/Tests

When And How To Run Group-Targeted A/B Tests

- Lior Neu-ner tl;dr: Tests are run when one user interaction with your product impacts how others use it. “Suppose Slack wants to improve the usage of a new video calling feature. Improving the feature's discoverability for a single user will increase their own usage with it, but since they use it with their coworkers, their coworkers will also discover it.”

featured in #421

8 Annoying A/B Testing Mistakes Every Engineer Should Know

- Lior Neu-ner tl;dr: (1) Including unaffected users in your experiment. (2) Only viewing results in aggregate (aka Simpson's paradox). (3) Conducting an experiment without a predetermined duration. Lior discusses these and 5 more anti-patterns.

featured in #416

So You Want To Build End-To-End Tests

- Rebecca Stone tl;dr: A well-built test suite runs faster, gives more reliable results, and makes long-term maintenance easier and cheaper. After thousands and thousands of end-to-end tests, QA Wolf has picked up a few tricks to build fast, stable, and accurate ones. This guide teaches you how.

featured in #414

An Example Of LLM Prompting For Programming

- Martin Fowler tl;dr: Martin shows us how ChatGPT produces useful self-tested code. The initial prompt primes the LLM with an implementation strategy asking for an implementation plan rather than code. Once that plan is in place, it’s refined and the author uses it to generate useful sections of code.

featured in #407

Why Engineering Teams Struggle To Scale Their Test Coverage

- Kirk Nathanson tl;dr: We talk to a lot of engineering leaders about QA and end-to-end testing. Something we hear all the time is how difficult it’s been to scale their automated test coverage beyond a few key workflows. Here are the three obstacles that are faced by companies of all sizes.

featured in #406

Unit Testing Analytics Code

- Matt Kaye tl;dr: “I face lots of pushback when it comes to unit testing. Usually the objections come in the form of either not knowing why you might test, since the code is just so simple and straightforward that nothing could go wrong, or not understanding the value added. In my opinion, both of these objections come from the same place.”

featured in #405

How Lyft Uses Load Testing To Ensure Reliable Service During Peak Events

- Remco Van Bree tl;dr: “We have come to realize that load testing in production is a powerful tool to prepare systems for unexpected bursty traffic and peak events. We’ll explore why Lyft needed a custom performance testing framework that worked in production, how we built a cross-functional solution, and how we’ve continued to improve this testing platform.”

featured in #404

Keeping The Cloudflare API 'All Green' Using Python-Based Testing

- Elie Mitrani tl;dr: This article discusses Scout, an automated system running Python tests verifying the end to end behavior of Cloudflare’s APIs. Scout evaluates APIs in production-like environments, green lights a production deployment and monitors the behavior of APIs in production. This article dives deep into how it operates.

featured in #399

Testing Practices I Follow

- Eliran Turgeman tl;dr: “I’ll share my practices for writing tests and talk about when I write tests. Disclaimer: This is not groundbreaking advice, if you’re an experienced software engineer the following might be obvious to you.”

featured in #393

The Only Guide To Automated End-To-End Testing You’ll Ever Need

tl;dr: If you’re looking for automated test coverage, there are a few options to consider: in-house, traditional outsourcing, or Test Coverage as a Service. We wrote this guide to help you find a solution that aligns with your product, team size, budget, and overall testing needs.

featured in #390