In the pipeline

From the latest news to upcoming events and interesting topics, “In the Pipeline” is overflowing with interesting updates for the Kedro community.

In the pipeline: November 2023

Voxel v1 white

This article describes the “code smells” typical to a data science project developed in a Jupyter notebook. It explains how some of Kedro's features can improve the code within your notebook.

Supercharge your data science notebook with Kedro

In the pipeline: October 2023

"In the Pipeline" is overflowing with the September's Kedro news.

In the pipeline: September 2023

Glass v8

In this blog post, Diego Lira explains how to use Databricks Connect with Kedro for a development experience that works completely inside an IDE. 

How to integrate Kedro and Databricks Connect

"In the Pipeline" is overflowing with the latest Kedro news, upcoming events, and interesting topics.

In the Pipeline: July 2023

Voxel v1 black

This post illustrates the extensibility of Kedro with a new dataset for realtime data processing using Spark Structured Streaming. 

A new Kedro dataset for Spark Structured Streaming

Glass v9

This post explains how to use a newly-released dataset for managed Delta tables in Databricks within your Kedro project.

How to use Databricks managed Delta tables in a Kedro project

We explain how to write an efficient custom Kedro pipeline runner that continues executing a set of nodes after encountering non-catastrophic failure points.

Get up to speed: how to build a custom Kedro runner

Fourth anniversary balloon

As we commemorate the 4th anniversary of our open-source journey, we look back on some key milestones with gratitude to the Kedroid community who have supported us. 

Kedro celebrates 4 years as an open-source project

Glass v6 resin

We are thrilled to announce that the Kedro brand has had a makeover. This article shares the inspiration that went into the rebrand and explores the exciting changes that lie ahead.


Unveiling Kedro's striking new branding

small square tie fighter

We've launched collaborative experiment tracking in Kedro-Viz 6.2.0 to enable a team of users to log and compare each others' experiments. This tutorial explains how to configure and use the feature.

Collaborative experiment tracking in Kedro-Viz

small square violet pringles

On World Product Day 2023, Yetunde Dada explains how to build your technical skills as a product owner to enhance your effectiveness and success in the role.


How to become a more technical product owner

small square ice dunes

Polars is an open-source library that provides fast dataframes for Python. This blog post explains how can you use Polars instead of pandas in Kedro for your data catalog and data manipulation.

A Polars exploration into Kedro

small square planet spin

Amazon EMR works with open-source big data frameworks like Apache Spark to help you tackle vast amounts of data. This post explains how to combine Amazon EMR, Kedro, and Apache Spark. 

Seven steps to deploy Kedro pipelines on Amazon EMR

small square glass waves 2

In the Pipeline: May 2023

small square exploding sugarcubes

In this blog post, we introduce a set of principles that summarise our development philosophy and steer our decision-making. 

Seven development principles for opinionated teams

small square yellow neon

We announce a new member of the Kedro Technical Steering Committee as we move towards Kedro's 4th open source anniversary.

News from the Kedro Technical Steering Committee 

small square mint knot

In recent research, we learned that Databricks is the dominant machine-learning platform used by Kedro users. We investigated aspects of our users' data science workflows to prioritise seamless Kedro usage, and we are collaborating with Databricks on our findings.

How do data scientists combine Kedro and Databricks?

This post explains how Kedro can guide an analytics team to follow best practices and avoid technical debt.

Introducing your new team lead…Kedro

small square ball and spoke

In this post, we describe recent releases to Kedro, Kedro-Viz and some new Kedro datasets. We also show off our new blog design.

A new home for the Kedro blog and some recent releases

small square mint holes

This article focuses on experiment tracking, explaining why it is valuable to data science projects and describing the features recently released for experiment tracking in Kedro projects. 

kedro

In the pipeline: November 2023

Recent blog posts

Supercharge your data science notebook with Kedro

A new Kedro dataset for Spark Structured Streaming

All blog posts