seniorReportedcurated
Describe a time you disagreed with a product or engineering stakeholder about using an agent, fine-tuning, or RAG.
30-second answer
- The disagreement was testable, not stylistic: the product lead wanted a multi-agent workflow for renewal briefs, and I argued the task was deterministic retrieval, summarisation and CRM validation.
- So I proposed a bake-off on 75 historical accounts, scoring field accuracy, reviewer edits and p95 latency, which moved the decision from who argued better to what the data showed.
- The agent produced more polished narratives but hallucinated account attributes in 12% of cases at 19 seconds p95, against 3% and 6.4 seconds for the constrained flow.
- We shipped the simpler flow with human approval on outbound content, and it cut analyst prep time 46%.
- The process change was to require a bake-off on real historical data before an architecture debate reaches leadership, because the option you reject usually has a real advantage worth naming.
- The lesson has a boundary: the bake-off decided this task, and a genuinely open-ended agent task would need a different test rather than a rerun on the same 75 accounts.
Likely follow-ups
Open a question to rehearse the answer.
What would have changed your mind?
- If the agent's field accuracy had matched the constrained flow, the narrative quality would have been worth paying latency for.
- A requirement for multi-hop synthesis across systems, or a reviewer edit rate that showed the constrained flow missing the analyst's real task, would also have moved me.
- The bar was set before the run, so changing my mind would have required the data to cross it, not a better demo.
How did the product lead react, and what did you do to keep the relationship?
- I separated the person from the proposal, agreed the scoring criteria with them first, and let the result rather than my opinion carry the argument.
- I gave them the decision on narrative polish once the factual bar held, so the outcome was a shared design rather than a defeat.
- That kept the relationship intact and produced a rule we both use now, which is a better outcome than winning the meeting.
Red flags
- Frames it as winning an argument rather than reducing complexity
- Runs no experiment and argues purely from principle
Concepts this tests
Tags
Source: www.onetworesume.com
More in Behavioural & project deep-dives
- Walk me through an AI feature you shipped end to end and your exact role. What decisions were yours?
- Tell me about a time a model surprised you in production. How did you find out, and what did you change?
- Tell me about a model-quality regression you introduced or discovered after deployment. What did you do?