NannyML

Hey, it’s Wiljan from NannyML. Indeed, a lot has happened since our last launch here. I have been mostly talking to data scientists for the past few months. So we can understand their pains and needs better, allowing us to build a better product for them. We have also been talking to business users, model owners, and other stakeholders. AI models are almost never isolated services and most of the time part of a bigger business process. One of the questions I often get is why we need to monitor AI models after they have been deployed. I thought I share my answer here: AI models are trained using historical data. Yet, they are tasked to make predictions on data of the present. Today's data might be very different from yesterday's data. The world is constantly changing and so does the data it generates. Assumptions and patterns that were true during training won’t hold forever after deployment. AI models are bound to deteriorate. It’s a natural process and a matter of when not if!

Hey Product Hunt! Hakim here from NannyML 👋 Today we are launching NannyML v0.8.0. In the past 6 Months we have added: - Regression Support (with Direct Loss Estimation) - Command Line Interface - Database exports and cloud integration - Official docker container - A ton of new univariate data drift tests (Jensen-Shannon Distance and L-infinity distance) - A bunch more improvements All of these things bring you closer to having a complete view of your ML performance in production, catching failures, and resolving them as fast as possible. Most notable is regression support. NannyML now allows you to detect silent model failure on all ML models with tabular data that are classification or regression, integrated anywhere, with any stack. After the launch of NannyML Open Source 6 months ago here on product hunt, we have put our heads down and kept iterating on the library with your input. 8 major releases have happened since, alongside 15 minor releases. Your input has been vital to this velocity and has allowed us to continuing to build a library loved by 1000’s of data scientists worldwide. I hope you like these new features and I am looking forward to hearing all of your feedback! - H

Hey, Niels the only engineer here. It’s been amazing to see how much our team has achieved since our last launch. We launched mission-critical features such as support for regression and expanded the range of drift detection methods. We turned a library into a CLI and a container, exporting our metrics into databases and dashboards, pushing forward to give NannyML a well-deserved place within the fast-moving MLOps landscape. We’ve completely overhauled and reinvented ourselves to make NannyML as easy to use as we can. I’m already looking forward to tackling the next set of challenges together with this group of exceptional people. Give NannyML a try, I hope you love it as much as I do.

Congrats on the launch! 🎉🚀 I've been following Hakim and the team for a while now and I'm really impressed with their progress and also the overall approach. Although I'm not currently an active AI/ML practitioner, I understand the problem they are solving and it is definitely a big one and will become only more and more evident in the future. Btw the CLI looks amazing! The dog with 🕶 is awesome :D

Hey, Jakub here, research data scientist at Nanny! Over a year ago, when I found out that NannyML is trying to estimate performance of ML models without having targets I was pretty sure it’s impossible so I quit my job and joined them. I was proven wrong when we came up with CBPE that estimates performance of classification models without ground truth. Now we have a completely different (yet similarly simple) algorithm to estimate performance of regression models. I’m twice as wrong as I was after releasing CBPE. When I joined Nanny I also said that detecting concept shift without targets is impossible, I guess you now know what we are doing now :) Reach me out to discuss anything!

@pymaster210 great product

Hello, I am Nik, a Research Data Scientist at NannyML. I am very happy to share that my journey with NannyML continues. 6 months ago we launched our open source library supporting model monitoring for binary classification models. We have added a lot more features since then and are now launching support for monitoring of regression models! We will not stop here. Please join us in enhancing your Model Risk practices and helping to improve your AI decision making.