Tune a batch job and an IDE button differently
Your task
Your team runs 2 features on the same model. One is a nightly job that reads about 8,000 crash reports and writes a short summary of each for the morning triage meeting. The other is an "explain this stack trace" button in the IDE, used while a developer waits. A teammate configured both the same way: the same model tier, streaming on, and a low cap on output tokens.
What would you set differently for each, and why?
It covers
- Model tier
- Batch vs interactive