PyData Tel Aviv 2022

The Tale of Flexibility - Generalized Average for Hyperparameter Tuning and Pooling
12-13, 14:15–14:45 (Asia/Jerusalem), Track 1

From summarizing distributions to pooling operators and measuring the goodness of fit, averaging plays a unique and universally recognized role in the field of machine learning. In the talk we present Generalized Average (GA) - a continuous and fully differentiable average operator that allows for flexible interpolation between min, max and three different types of averages: arithmetic, geometric and harmonic. We share the results of two lines of experiments: (1) using GA in hyperparameter tuning for false-positive-averse cases (e.g. fraud detection) and (2) using GA as a pooling operator in Graph Attention Networks to improve the model’s flexibility. Finally, we present an open-source Python package with our implementation of GA. The talk is addressed to machine learning practitioners, who are interested in enriching their toolbox.


Imagine that you work at a fraud prevention company. You’re aiming at preventing fraudulent transactions. At the same time you want to avoid blocking the legitimate ones to keep the customers happy. Depending on the performance of your model you might want to penalize false positives or false negatives stronger to find a sweet-spot between financial benefits and customer satisfaction.

Generalized Average (GA) is a neat and easy way to help you with this task. Moreover, GA can be used as a custom loss function and a flexible goodness of fit metric.

The goal of the talk is to introduce you to the concept of GA and give you practical tools that will allow you to apply it to your own projects.

The talk is addressed to people who want to enrich their modeling toolkit.

To fully enjoy the content, it’s recommended that you:
* Have good understanding of basic algebra
* Have working knowledge of Python
* Have good understanding of machine learning process, including hyperparameter tuning
* Have good understanding of deep learning fundamentals

The goal of this talk is to give you practical understanding of how to think about averaging more broadly and how to leverage Generalized Average in your own projects.

graduate phd in applied Math form thr tehchnion in 2013. Worked as data satintist in auto mobile cyber, log analysis and more

R&D Machine Learning Engineer and Researcher at Ironscales and independent Machine Learning Researcher at TensorCell. Before joining Ironscales Alex has built end-to-end machine learning systems for Fortune Global 100 and 500 companies.

International speaker, blogger, currently working on a book on causality in Python. Interested in NLP, causality, probabilistic modeling, representation learning and graph neural networks.

Loves traveling with his wife, passionate about vegan food, languages and running.

Website: https://alxndr.io

LinkedIn: https://www.linkedin.com/in/aleksandermolak/