DDataflow: An Open-Source Tool for End-to-End Tests of Machine Learning Pipelines (collaboration with Jean Carlo Machado)

GetYourGuide Tech Blog 7 Dec 2023

DDataflow is an open-source tool for end-to-end testing of machine learning pipelines. It addresses the challenge of slow feedback loops in ML development by providing efficient, configuration-based sampling of production data, enabling data science teams to iterate quickly and catch integration bugs early.

Data EngineeringTestingML PipelinePySparkOpen Source
GetYourGuide tech blog image for the DDataflow article