The Interview Should Test Who Owns the Patch

The first answer is only a baseline

AI-assisted interviews are often framed as a choice between banning tools and letting a model do the work. A better question is what the interviewer can observe after a candidate gets a first answer. A polished patch alone says little about how it was produced; the candidate’s choices while checking and adapting it are more revealing. That is an assessment design argument, not a claim that any one exercise predicts job performance.

A July 2026 HackerRank vendor guide proposes a “multi-step build with a twist”: let a candidate build with AI, then introduce a changed requirement S1. Canva’s engineering team described a related policy in a June 2025 post, saying it expected candidates in several engineering tracks to use AI and redesigned tasks around more complex, ambiguous challenges S3. These are organizations’ accounts of their own approaches, not independent evidence that the formats improve hiring outcomes.

Use a controlled change, not a hidden trap

Give the candidate a small, realistic repository task with clear initial acceptance criteria. Tell them in advance which AI tools are allowed, what resources are available, and how long they have. After a first implementation, introduce one prewritten change that affects behavior—not an arbitrary puzzle. Ask the candidate to explain what changed, update or add a test, then inspect the resulting diff and describe a failure case.

A commenter in a Hacker News discussion describes a CSV exercise extended by five lines after one rule changes S2. That is a practitioner anecdote, not a validated hiring method, but it illustrates a useful constraint: the follow-up should be concrete enough to see whether the candidate revisits assumptions and tests, rather than merely asking for a fresh prompt.

Tell candidates that requirements may evolve, even if you keep the exact change undisclosed. That preserves the exercise without turning tool-policy discovery or interviewer surprise into the test. Keep the repository, timebox, initial task, follow-up, and AI rules consistent across candidates.

Score observable decisions

Write the rubric before the interview. Record whether the candidate clarifies the changed contract, chooses a test that distinguishes old from new behavior, checks the AI-generated diff, and can explain a trade-off or remaining risk. Do not use prompt style, typing speed, or amount of generated code as stand-ins for those judgments. If the candidate spots an unsafe assumption and asks for a decision, count that as useful evidence too.

YC Startup School’s hiring session labels its interview segment “sell first, assess second” S4. Apply that here: explain why the exercise resembles the role, disclose the rules, and leave room for the candidate to ask questions. The goal is not to prove that one interview predicts an engineer’s future. It is to gather a consistent, reviewable sample of how someone takes responsibility for a patch when the contract moves.