The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon)
Transcribed with NVIDIA Parakeet · Timestamps stay in sync with the live audio
I'm very clear that I'm a big fan of test everything. Which is any code change that you make. Any feature that you introduce. has to be in some experiment because again I've observed this sort of surprising result that even small bug fixes, even small changes can sometimes have
surprising unexpected impact. And so I don't think it's possible to experiment Too much? You have to allocate sometimes to these high risk, high reward ideas. We're gonna try something.
That's most likely to fail. But if it does win It's gonna be a home run. And you have to be ready. to understand and agree that most will fail. And I've it's amazing how many times I've seen
We already have this episode, so unlocking it is cheap and permanent: the full transcript, segment and word-level timestamps, and .txt, .srt, .vtt and word-level JSON downloads, as many times as you like, forever.
No account needed, we create one from your checkout email. With credits it is 1 credit; packs start at 10 episodes for $5, and monthly plans are cheaper still.