Python

From Jupyter Notebook to production code

a Kedro introduction
Have you ever had issues to share your Jupyter Notebooks? Ever had troubles with code that "works on my machine" only? Do you consider your Research and Development smooth and straightforward? Is your code scalable? Tough questions, I know. But if you've mentally answered 'no' to any of those you could use a tool to help with some of the pain-points of your workflow. Kedro is an open-source Python library that helps data scientists write data pipelines following software engineering best practices from the start. Known as the Django of ML/DS projects, Kedro is an opinionated framework based on cookiecutter data science that allows for modularity and scalability on data science projects. In this talk, I will explore the workflow of a Kedro project, introduce some of the most outstanding features of the framework, such as the Data Catalog and show how to convert a Jupyter Notebook into a Kedro project, allowing for scalability and team collaboration.
Talk structure Intro (5 min) The problem(s) (10 min) The solution (5 min) Demo - convert Notebook to Kedro project (15 min) Q&A (5 min) Audience This talk focus on data engineers, machine learning engineers and data scientists who wish to learn how to write code beyond the Jupyter Notebook. The audience is expected to know the basics of Python and Jupyter Notebooks. All levels are welcome. Key takeaways By the end of this talk the attendees are expected to understand the basics setup of a Kedro project, know how to convert a Jupyter Notebook into a Kedro project and to visualise the created data pipelines using Kedro Viz.

Additional information

Type devroom

More sessions

2/7/21
Python
D.python
We will talk about different approaches in teaching Data Science with the Python programming language. As a case study, we will use our own experience in providing Data Science education with Python across different audiences in the Asia Pacific region and share approaches and principles that worked for us. The lecture will serve as an anchor for more conversations and discussions for adapting pedagogy that is most effective for various contexts and settings.
2/7/21
Python
D.python
In 2020, with funding from Mozilla and CZI, the pip packaging team improved pip for all users. These focused on improving the depedenecy resolution for Python packages, and the user experience for all Python users. We carried out usability testing, user research and improved error and information messages for pip. This talk will be about these improvements - we'll explain how the new dependency resolver works, what it can (and can't do!), how we improved the user experience of pip, the ...
2/7/21
Python
D.python
Mypy has been around since 2012, and in recent years its gaining wide spread adoption. As the framework continues to evolve and improve, more and more useful features are being added. In this talk I'm presenting some hidden gems in the type system you can use to make your code better and safer!
2/7/21
Python
D.python
We made the Web accessible to human. What about making the web (of data) accessible to computers? Publish open data could be struggling. Deposit a CSV file on a web server is not enough. The data model used should be explicitly defined. The Linked Open Data (https://www.w3.org/standards/semanticweb/data) solves this with: * standardized implementation format (RDF) * standardized data access (Content negociation, SPARQL endpoint) * standardized data indentification (URI, data model as data) For ...
2/7/21
Python
D.python
Everybody hates mundane tasks, they are boring, repetitive and time-consuming. That’s why I love building bots, they can finish my tasks for me by working 24/7. But to build a bot to interact with the users, you have to write in async. If you are afraid of async, don’t worry! Today I am telling you how I learn using async and how I avoid checking in 500+ people in a conference by building a bot with Discord.py.
2/7/21
Python
D.python
As part of the schul-frei project of Teckids e.V. we curate free software and offer it to educational institutions. Besides the general equipment of schools with free software, an equal involvement of students in the development is for us important. One of the solutions presented by the project schul-frei is AlekSIS, a web-based school information system that is being developed jointly by Teckids e.V. and students of the Katharineum in Lübeck. The django-based platform provides data structures ...
2/7/21
Python
D.python
Pinax is an open-source ecosystem of reusable Django starter projects, apps, and themes for building websites. When developers began building Pinax in 2007, they had fun adding to it, but eventually Pinax had grown to become around 80 projects and apps. Without a strategy in place to make Pinax as easy as possible to maintain, the maintainers began to suffer burnout. I was hired to work on Pinax in the fall of 2017. In my talk, I'll outline the critical problems I've discovered and the solutions ...