Every Mistake Becomes a Rule:
How diffray Learns from Your Feedback

Product leader Aakash Gupta summarized it perfectly: "Every mistake becomes a rule." The longer a team works together with AI, the smarter it becomes.

This is exactly the philosophy diffray is built on. Today, we'll show you how it works under the hood.

The Problem: Context Pollution Kills Review Quality

Before we talk about rules, we need to understand the main technical challenge of AI code review — context pollution.

Anthropic's research shows that LLMs, like humans, lose focus as the context window fills up. Corrections accumulate, side discussions pile up, outdated tool outputs linger. The result is predictable:

JetBrains Research (December 2025) quantified this: agent contexts grow so rapidly that they become expensive, yet don't deliver significantly better task performance. More context ≠ better results.

The Solution: Specialized Subagents with Isolated Context

Boris Cherny uses subagents as "automated encapsulations of the most common workflows." His philosophy:

Instead of one omniscient reviewer, his code review command spawns multiple parallel agents with distinct responsibilities:

This adversarial layer is crucial. Secondary agents challenge findings from the first pass, eliminating false positives through structured skepticism.

The result, in Cherny's words: "finds all the real issues without the false ones."

How It Works Technically

When the main agent delegates to a subagent, a fresh context window spawns containing only the task description and relevant parameters. The subagent may explore extensively—consuming tens of thousands of tokens searching through code—but returns only a condensed summary of 1,000-2,000 tokens.

This preserves the primary agent's focus while enabling deep analysis.

At diffray, we use 10 specialized agents, each focused on a specific domain: security, performance, code style, architectural patterns, and more. Each agent operates in an isolated context and returns only substantive findings.

Rule Crafting: Turning Feedback into Knowledge

Now for the main event. Subagents solve the context problem. But how do you make AI learn from your corrections?

The CLAUDE.md Pattern

In Claude Code, teams maintain a CLAUDE.md file in their repository—a kind of "constitution" for the project. The file is automatically loaded into context at every session.

But there's a critical limitation. HumanLayer research shows that Claude Code's system prompt already contains ~50 instructions, and frontier LLMs reliably follow only 150-200 instructions total. Instruction-following quality decreases uniformly as count increases.

This means: you can't just dump 500 rules and expect magic.

Three Levels of Knowledge

Effective rules encode knowledge at three levels:

## Tech Stack
- Backend: Python 3.11, FastAPI, SQLAlchemy
- Frontend: React 18, TypeScript, TailwindCSS
- DB: PostgreSQL 15

## Why We DON'T Use ORM for Complex Queries
History: ORM generated N+1 queries in reports.
Decision: Raw SQL for analytics, ORM only for CRUD.

## Before Committing
- Run `make lint` — must pass with no errors
- Run `make test` — coverage must not drop

The Problem with Manual Approaches

Manual rule maintenance works... as long as your team is small and disciplined. In reality:

How diffray Automates Rule Crafting

diffray flips the process on its head. Instead of manually writing rules, you just give feedback on reviews.

The Learning Loop

Types of Rules in diffray

Practical Example: From Annoyance to Rule

Imagine: diffray leaves a comment on your PR:

You know this is a legacy module scheduled for rewrite next quarter. Fixing types now would be a waste of time.

You reply: "This is legacy, typing will be addressed during Q2 refactoring"

What happens next:

But importantly: the rule isn't permanent. The expiration date means that after Q2, diffray will start checking types in that directory again.

The Metric: Reducing False Positive Rate

The key measure of AI code review effectiveness is false positive rate. How many comments out of 100 were useless?

Typical industry benchmarks:

How we achieve this:

Getting Started: Three Steps

Every interaction makes diffray smarter. After a month, you'll have a personalized AI reviewer that knows your conventions better than a new developer after onboarding.

Conclusion: AI That Grows with Your Team

The philosophy of "every mistake becomes a rule" isn't just a catchy phrase. It's an architectural principle that separates toy tools from production-ready solutions.

diffray is built on three pillars:

The result: AI code review that gets better with every PR. Not because the model was updated, but because it learns from your team.