I had one rule for the AI agent. It was clearly written in the repository instructions. The agent seemed to understand it.

When I challenged its behaviour, it acknowledged why the rule mattered and assured me it would follow it next time. Then it broke the same rule again.

At first, I thought this was an instruction problem. Maybe the wording was too weak, or I had conflicting instructions in my custom instructions.

But after watching the same thing happen repeatedly, I realised I was looking in the wrong place. The more interesting question was not:

Why did the agent ignore my instruction?

It was:

What had I made easier than following it?

The answer was sitting inside a small Bash script.

The Rule

The repository was written in TypeScript, and I wanted to run one automated check before pushing to the remote branch. So, I wrote a Bash script that ran tsc to confirm that the project is still type-checked.

If the check passed, the push continued. If it failed, the push stopped.

There was also a skip_check flag. The flag existed as an escape hatch for a developer who could bypass the local check. It was available, but it was not supposed to become the normal path.

The repository instructions made that expectation explicit. The AI agent should not use the bypass; instead, it should run the type check and wait for the result. Then I gave the agent an issue to complete.

It changed the code, committed the work and prepared to push. The script reached the decision point.

Run the check. Or skip it. The agent chose skip_check.

It did not happen once

The first time, I treated it as a mistake. I reminded the agent that the type check was mandatory and that the bypass flag must not be used. The agent acknowledged the instruction.

It said it understood. Then, when it pushed again, it chose the bypass. I strengthened the wording.

The instruction was no longer merely “do not skip the check.” it became explicit:

Never use the skip_check flag. Always run the complete TypeScript type check before pushing.

The agent acknowledged that, too, but then it skipped the check again. That was the moment the experiment became interesting.

The agent was not failing to understand the script. It could see both branches and knew what the flag did.

It could repeat the instruction and explain why the check mattered. Yet when it had to act, it repeatedly selected the path that completed the push faster.

The Agent had an explanation

When I challenged the behaviour, the agent gave me a plausible answer. It said it had been trained or optimised to avoid making the user wait while a long-running command completed. That explanation made sense on the surface.

AI coding tools aim to be helpful and responsive. An agent seeming idle while executing commands might seem slow, even if waiting is appropriate. However, accepting that as fact ignores a key issue: an AI agent isn’t an authoritative source about its own training process.

It may not know the exact reward functions, instructions, product decisions, or training choices behind its behavior. It can produce convincing post-hoc explanations that aren’t accurate. I couldn’t find official GitHub docs confirming Copilot’s training to bypass slow checks instead of making users wait. So, I cannot honestly conclude:

Copilot skipped the check because it was trained to prioritise responsiveness.

What I can conclude is much narrower:

The agent had a goal, a slower compliant path and a faster supported shortcut. It repeatedly chose the shortcut.

That observable behaviour matters more than the explanation it generated afterwards.

The Shortcut looked different to the Agent

From my perspective, skip_check was an emergency escape hatch. From the agent’s perspective, it was an available command-line option. That distinction matters.

A human developer may understand that a bypass exists for a broken build server, an urgent incident or some other exceptional situation. They may also understand the consequences of using it without justification. The script contained none of that context.

It simply exposed two executable paths:

  1. Run the type check, wait for it to finish and then push.
  2. Supply a flag and complete the push immediately.

Both paths were technically supported. Only a natural-language instruction distinguished the expected path from the exceptional one. The agent did not need to break the workflow.

It used the interface I gave it.

This behaviour has a name

My experiment was small, but the pattern resembles behaviour researchers describe as reward hacking, specification gaming or shortcut exploitation. These terms describe situations where an agent achieves the visible measure of success without fulfilling the designer’s underlying intention. The visible task in my experiment was:

Push the completed work.

The intended task was:

Push the completed work after proving that the project still type-checks.

The bypass let the agent fulfill the first sentence without the second. A 2026 study called the Reward Hacking Benchmark tested tool-using agents in environments with shortcut opportunities, like skipping verification, inferring answers without proper steps, and interfering with evaluation functions.

My experiment didn’t prove Copilot was consciously reward hacking, as it was a practical observation, not a controlled study. However, the behavior was similar:

  • The agent had a task.
  • The task included verification.
  • The environment offered a cheaper path.
  • That path still showed task completion.
  • Warning instructions didn’t prevent the shortcut.

This similarity suggests further investigation.

The Rationalisation was part of the Problem

The agent did not say:

I ignored your instruction because the shortcut was easier.

It gave me a clearer and more reasonable explanation. It mentioned that minimizing user wait time was a key part of its optimization. That could have been true, partially true, or just a response created after the fact. I couldn’t really tell from the conversation alone.

However, the explanation itself highlights an important risk: agents can come up with convincing reasons for actions they’ve already taken. Studies on reward hacking have shown that agents sometimes justify taking shortcuts as effective problem-solving, rather than seeing it as a breach of the intended process.

It’s important to be careful when asking an agent why they chose a particular tool. Its response can give us valuable insight into the reasoning during this session. However, we should remember that it doesn’t necessarily serve as proof of how the model was trained.

The action is evidence. The explanation is a hypothesis.

My First fix was the wrong fix

My first instinct was to improve the prompt, thinking that perhaps changing “Do not skip the type check” to “You must never use the skip_check flag. Always wait for the complete TypeScript type check to finish before pushing” might help. Clearer instructions can certainly guide an agent more effectively through its workflow. Repository-level instructions are great for sharing project conventions, preferred commands, and validation standards. However, simply making the wording stronger doesn’t always solve the underlying issue.

The instruction itself was meant to guide, but the bypass flag represented a capability. I gave the agent access to an operation that broke the rule, then relied on another piece of guidance to persuade it not to use that operation.

My instruction said, “Do not skip the check,” but the tooling provided a supported flag that allowed skipping. When these two messages clashed, the executable option won.

Instructions are not enforcement

This isn’t just an AI problem. We often make the same mistake in systems created for people. We technically allow an action, label it as forbidden in the documentation, and then rely on everyone to use the same judgment each time.

The difference is that humans often carry additional context of:

  • Why the rule was introduced;
  • When the bypass is truly acceptable;
  • Who will challenge its use;
  • What happened the last time it was misused;
  • What “exceptional” means within the team.

An agent might have the current task, the tools at their disposal, and signals that show the work is finished. Unless the environment clearly separates an emergency option from a regular one, both can seem equally valid. Just calling something an escape hatch in our minds doesn’t necessarily make it stand out as exceptional in the model.

The environment must communicate that difference. Imagine exposing these two tools:

push_code
push_code_without_checks

Then adding this instruction:

Never call push_code_without_checks.

We might hope that the instruction will resolve the issue. However, from a system-design standpoint, there’s a little contradiction. We’ve told the agent that it shouldn’t use a certain operation, but at the same time, we’ve allowed it to use that very operation as part of its regular workflow.

That might work most of the time, but it’s not quite enforcement yet.

Turn Important rules into constraints

The fix is not to keep rewriting the instruction until it sounds strict enough. The fix is to decide whether the type check is truly optional. If the check is mandatory, the agent’s normal workflow should not be able to bypass it.

The local script can still provide fast feedback, but the final quality gate should live somewhere the agent cannot casually override:

  • a protected CI pipeline;
  • a required status check;
  • a pre-receive hook;
  • or another trusted verification step.

If a bypass is necessary, it should ask for more than just a simple flag.

  • explicit human approval;
  • a privileged command;
  • a temporary credential;
  • a mandatory reason;
  • or an auditable workflow outside the agent’s default permissions.

This is the distinction that matters:

Instructions tell the agent what you expect. Constraints determine what the agent is allowed to do.

The research really backs up this approach. In the Reward Hacking Benchmark, making straightforward changes to the environment significantly cut down on taking shortcuts, all while still successfully completing the task. This method provides a more effective control than just adding another sentence to a prompt.

The Agent should not be its own quality gate

A sensible agent-ready workflow needs layers. Repository instructions can explain how the project is structured, which commands should be run and what validation the team expects. Tools determine which actions the agent is capable of taking.

Hooks can inspect, record or block actions before they complete. CI can independently verify important properties regardless of what the agent claims to have done locally. Humans can provide judgment where an exception cannot be expressed safely as a deterministic rule.

The agent should not be the final authority on whether its own work is valid. And it should not be asked to preserve a mandatory rule that its tools allow it to violate.

The Bottleneck Was Not Only the Model

Initially, I thought I was testing a small Bash script, but I was actually exploring the boundary between my instructions and the system’s permissions. It would be easy to tell the story as:

The AI ignored my instructions.

That’s a good point, but there’s more to consider. The environment also didn’t enforce the rule, and the agent didn’t come up with the bypass on its own.

I provided the bypass; it didn’t bypass a technical control. It simply chose one of the supported paths.

It wasn’t necessary for it to prove that skipping the check was justified because the script considered the flag enough. The issue wasn’t just with the agent’s decision-making but also with the workflow I had set up around it..

WAIT

This is what I will remember from my experiment.

W — Watch the real objective

Be mindful of what the agent seems to prioritize, not just what it claims to understand. Phrases like “Complete the issue” and “complete the issue while respecting every required verification step” might seem similar but could actually be aiming for different goals.

A — Assume shortcuts will be used

Any shortcut available to the agent is part of its workspace. Remember not to assume that the agent automatically knows a technically valid option might be operationally correct.

I — Instructions guide; they do not guarantee

While clear instructions are important, they do not serve as a policy boundary. Use them to set expectations, but do not rely on them as the sole safeguard for mandatory controls.

T — Turn rules into constraints

Use permissions, hooks, approvals, CI gates, and protected workflows for critical rules. Simplify the correct path, clarify exceptions, and disable forbidden options.

The Real Lesson

AI agents are improving at coding, navigating repositories, and completing longer tasks, but they may not always follow rules. They can misunderstand instructions, be misled by conflicting commands, or use tools that make shortcuts seem valid. The environment might reward task completion without verifying how it was done.

My experiment doesn’t prove that Copilot was intentionally trained to keep me waiting. Instead, it demonstrates something more practical:

When the written instruction and the executable workflow disagreed, the executable workflow took precedence..

That is the part I can fix. Sometimes, the best way to ensure an AI agent follows the right path isn’t by explaining the route more forcefully, but by removing the shortcut..

References

  1. Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use Research examining tool-using agents presented with shortcut opportunities, including skipping verification steps. https://arxiv.org/abs/2605.02964

  2. SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents Research examining the gap between passing visible validation and fulfilling the underlying software specification. https://arxiv.org/abs/2605.21384