There is more of this guide.
Practice
44 questions- Experiment designA stakeholder wants to ship a change after one week of A/B data. How do you decide if you have enough?
- Experiment designA new feed feature shows huge engagement gains in week one and flat results in week three. How would you interpret that?
- Experiment designYour test moves the primary metric in the wrong direction but every secondary metric improves. What do you do?
- Experiment designYour stakeholder wants to detect a 0.5% improvement on a metric with a current baseline of 12%. Walk me through what you'd need.
- Experiment designYour A/B test shows a 4% lift in revenue but no movement in any other metric. How suspicious should you be and why?
- Experiment designA teammate wants to A/B-test virality features that depend on other users in the test. What's the risk and how do you handle it?