There is more of this guide.
Practice
23 questions- Experiment designYou want to test a feature that changes how teams collaborate inside an organization. What's your randomization unit and why?
- Experiment designA new feed feature shows huge engagement gains in week one and flat results in week three. How would you interpret that?
- Experiment designYou're testing a marketplace feature that affects both buyers and sellers. What randomization scheme avoids contamination?
- Experiment designA teammate wants to A/B-test virality features that depend on other users in the test. What's the risk and how do you handle it?
- Experiment designYou want to A/B-test an algorithm change that affects sessions, but users have multiple sessions. What's your randomization unit?
- Experiment designA long-running feature test still shows a positive effect after three months. Should you trust that and how would you check?