featured in #441
Fuzz Testing Is the Best Thing To Happen To Our Application Tests
- Andrei Pechkurov tl;dr: The team at QuestDB faced challenges with segfaults, data corruption, and concurrency bugs. To address these, the team implemented fuzz testing, an automated software testing technique that provides invalid or unexpected data to a program to monitor for exceptions. This article details the process of introducing fuzz testing, revealing critical issues and leading to more robust database performance. The team also collaborated with SQLancer, a tool for testing SQL Database Management Systems, to uncover issues in their SQL engine.featured in #441
A/B Testing Examples From Airbnb And YC's Top Companies
- Ian Vanagas tl;dr: Ian provides a comprehensive look at A/B testing examples from various successful companies, including Monzo, Instacart, Coinbase, Airbnb, and Convoy. It explores different approaches to A/B testing, such as Monzo's low-risk "pellets" strategy, Instacart's complex sampling problem-solving, Coinbase's scaling of tests, Airbnb's interleaving and dynamic p-values, and Convoy's Bayesian approach.featured in #437
A Software Engineer's Guide To A/B Testing
- Lior Neu-ner tl;dr: This guide provides an introduction to A/B testing for software engineers. It explains the basics of A/B testing, including how to devise, implement, monitor and analyze tests, and answers common questions about A/B testing. The guide also lists conditions under which you may want to avoid A/B testing, such as lack of traffic, high implementation costs, and ethical considerations. The post concludes with a launch checklist for A/B tests.featured in #434
featured in #431
Why We Test In Production (And You Should To)
- Ian Vanagas tl;dr: "Testing in production successfully is a multi-step process, and this post goes over what it is, why we do it, and how to do it well." Ian covers various types of production testing, such as usage tracking, feedback, monitoring, load testing, and integration testing.featured in #428
featured in #423
When And How To Run Group-Targeted A/B Tests
- Lior Neu-ner tl;dr: Tests are run when one user interaction with your product impacts how others use it. “Suppose Slack wants to improve the usage of a new video calling feature. Improving the feature's discoverability for a single user will increase their own usage with it, but since they use it with their coworkers, their coworkers will also discover it.”featured in #421
8 Annoying A/B Testing Mistakes Every Engineer Should Know
- Lior Neu-ner tl;dr: (1) Including unaffected users in your experiment. (2) Only viewing results in aggregate (aka Simpson's paradox). (3) Conducting an experiment without a predetermined duration. Lior discusses these and 5 more anti-patterns.featured in #416
So You Want To Build End-To-End Tests
- Rebecca Stone tl;dr: A well-built test suite runs faster, gives more reliable results, and makes long-term maintenance easier and cheaper. After thousands and thousands of end-to-end tests, QA Wolf has picked up a few tricks to build fast, stable, and accurate ones. This guide teaches you how.featured in #414