PyData Tel Aviv 2022

Interactivity, traceability, transparency and efficiency are becoming increasingly important, yet challenging, in today’s data-rich analysis applications. Inevitably, data analysis pipelines are often heavily parametrized, and we lack good ways to trace which specific parameters affect a focal downstream result and to evaluate the effects of changing parameters in ways that are interactive, transparent and computationally efficient. In the talk, we will introduce “Quibbler” - a new open source, pure-python package for building inherently interactive, yet traceable, transparent and efficient data analysis applications. Founded on a data-flow paradigm, Quibbler allows processing data through any series of analysis steps, while automatically tracking functional relationships between downstream results and upstream parameters. Quibbler facilitates and embraces human interventions as an inherent part of the analysis pipeline: input parameters, as well as algorithmic exceptions and overrides, can be specified interactively, and any such interventions are automatically recorded and documented. Changes to upstream parameters propagate downstream, pinpointing which specific data items, or even slices thereof, are affected, thereby vastly saving unnecessary recalculations. Importantly, Quibbler does not require learning any new programming syntax; it seamlessly integrates into any standard Python analysis code. We are just launching Quibbler as an open-source project, and are eager to see it being used and integrated within a range of data science applications. We are of course also looking for feedback, suggestions and help.