How Booking.com increases the power of online A/B experiments with CUPED

Link: https://booking.ai/how-booking-com-increases-the-power-of-online-experiments-with-cuped-995d186fff1d

#ab #statistics #booking

​​Reimagining Experimentation Analysis at Netflix

Short article on how Netflix built A/B test culture and why they considered that a good resource investment.

Link: https://medium.com/netflix-techblog/reimagining-experimentation-analysis-at-netflix-71356393af21
More info on theirs A/B test platform: https://medium.com/netflix-techblog/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15

#Netflix #ExperimentDesign #AB

​​Practitioner’s Guide to Statistical Tests
CoreML team at VK

If you want to learn how to choose the right statistical test from the many available and run it on your own data you can find the answer at this article.

The two most essential things in A/B tests are the design of the experiments and accurate analysis of the experiments’ results. In this article, the authors stuck to the most common design and compare various statistical analysis procedures, from the very standard t-test and Mann-Whitney test to state-of-the-art approaches like the reweighted bootstrap.


article: https://medium.com/@vktech/practitioners-guide-to-statistical-tests-ed2d580ef04f
github: https://github.com/marnikitta/stattests

#statistic #ab #tests #vktech

Reliable ML track at Data Fest Online 2023
Call for Papers

Friends, we are glad to inform you that the largest Russian-language conference on Data Science - Data Fest - from the Open Data Science community will take place in 2023 (at the end of May).

And it will again have a section from Reliable ML community. We are waiting for your applications for reports: write directly to me or Dmitry.

Track Info

The concept of Reliable ML is about what to do so that the result of the work of data teams would be, firstly, applicable in the business processes of the customer company and, secondly, brought benefits to this company.

For this you need to be able to:

- correctly build a portfolio of projects (#business)
- think over the system design of each project (#ml_system_design)
- overcome various difficulties when developing a prototype (#tech #causal_inference #metrics)
- explain to the business that your MVP deserves a pilot (#interpretable_ml)
- conduct a pilot (#causal_inference #ab_testing)
- implement your solution in business processes (#tech #mlops #business)
- set up solution monitoring in the productive environment (#tech #mlops)

If you have something to say on the topics above, write to us! If in doubt, write anyway. Many of the coolest reports of previous Reliable ML tracks have come about as a result of discussion and collaboration on the topic.

If you are not ready to make a report but want to listen to something interesting, you can still help! Repost to a relevant community / forward to a friend = participate in the creation of good content.

Registration and full information about Data Fest 2023 is here.

@Reliable ML