Coach note

Interviewers ask this kind of question to surface how you think, not what you remember. The strongest answers are specific, calmly told, and end on what changed.

Experiment design · Evaluating AI features

You're A/B testing two LLM-backed variants whose outputs differ run to run, even on identical inputs. What does that non-determinism do to your experiment design and your read of the results?

Type your answer

0of ~220 wordsAbout a minute spoken

Voice isn’t supported in this browser — type your answer in the box.

Create a free account to get more critiques.

Your answer

Your answer appears here as you speak.

You're A/B testing two LLM-backed variants whose outputs differ run to run, even on identical inputs. What does that non-determinism do to your experiment design and your read of the results?

Practice more