Descending: Why You Cannot Delete an Idle AI Agent Layer

A system I built has an autonomous agent in it that answers nothing.

I measured it. Five requests went through the diagnostic path. The agent rung answered none of them. Four of the five were resolved by the cheaper rungs underneath it, and the fifth went unanswered whether the agent was there or not. The agent cost 26 percent more per request and contributed nothing.

The system is the reference implementation for a book I was writing at the time, which means I measured the overhead, printed the number in a chapter, and then left the rung in place while I wrote five more chapters on top of it.

This is an article about how you decide whether a layer like that one is safe to remove.

The interesting part is what did not stop me. It was not effort. Removing the rung was one line in a table: the ladder is a list of tuples, and taking the top entry off the diagnostic path is a single edit that takes effect on the next deploy. It was not ownership. I own the whole system. It was not tooling, or approval, or a migration plan, or a deprecation window.

I had the measurement, the authority, and a one-line change, and I did not make it. Nor has anyone I have asked since.

The thesis: adding and removing are not the same kind of claim

The standard explanation for why software never gets simpler is cost. Removal is harder than addition because dependencies accumulate, the people who understood the component leave, and coordinating the change across a large system is expensive. That literature is real. Read it.

It is not what stopped me, though my case is unusual and I should say how. I own the system, the removal is one line, and nobody would have reviewed it. There is also a confound worth naming before a reader finds it: the rung is a teaching artifact in a book about all eight rungs, so deleting it carried a cost that has nothing to do with architecture. That is dependency gravity with a manuscript as the dependent. This is not proof that the two barriers are independent. It is the case that made me notice the second one.

Something else stopped me. The blocker sits inside the claim I would have had to make. My codebase had nothing to do with it:

Adding a layer is authorized by a single case. Removing one is authorized by evidence your logs structurally cannot contain, because your system has never once run without the layer. Both decisions rest on judgement in the end, and only one of them is ever asked to admit it.

To justify building the agent rung, I need one case. One reproducible request that the multi-step reasoning rung below it could not answer is a finished argument. Finding a hundred more would not make it more finished.

To justify removing the agent rung, I need the absence of such a case, and not merely in what I have observed. I need it across all traffic the system will ever see. "No request needs this rung" is a claim about requests nobody has made yet. A log is a record of requests somebody already made. Those two sets do not overlap, and that is the whole problem.

That is why the removal did not happen. Not because it was expensive, but because it was never authorized, and I could not say what would authorize it.

What an idle AI architecture layer actually costs

An idle architecture layer bills you every month. The rungs of a generative AI (GenAI) system are priced very differently from each other.

Anthropic reported in June 2025 that agents use roughly 4 times the tokens of a chat interaction, and that multi-agent systems use roughly 15 times. On the BrowseComp evaluation, token usage by itself explained 80 percent of the variance. Those absolute numbers belong to a model generation and will move. The ratio is the durable part: each rung you climb multiplies the cost of the one below it.

How much of your bill that idle rung carries depends entirely on how often it fires. Mine ran on every diagnostic request and still only added 26 percent, because the rungs beneath it were doing expensive work too. A top rung that fires on a fraction of traffic and multiplies its cost is the case that hurts.

Gartner predicted in June 2025 that over 40 percent of agentic AI projects would be cancelled by the end of 2027, citing cost and unclear business value. Gartner's stated drivers are cost, unclear business value, and inadequate risk controls. It does not break out how many of those projects could have been rescued by removing a layer rather than cancelling the project, and I suspect the number is not small. That is my inference, not theirs.

And by default the ladder only goes one way. Part 1 of this series gave a rule for climbing the ladder: start at Level 0 and move up only when the current level fails. That rule is sound. It also has only one direction built into it. "Move up when the current level fails" is closed by a single failure. It is an existence-claim rule, and it has no downward form. Part 1 named LLM-washing, meaning a deterministic problem wrapped in a model call because the team is in AI mode, and gave you a test for avoiding it before you build. It did not tell you what to do about the LLM-washing already in production, because the test that catches it going up does not run in reverse.

Adding a layer carries a universal claim too, and nobody checks it

There is an obvious objection to everything above, and it is the right one to raise.

Adding a layer also asserts something universal. "This agent rung will not degrade any future request" quantifies over unobserved traffic exactly as hard as "no request needs this rung" does. Added latency, new failure modes, a wider prompt-injection surface, non-determinism on a path that used to be deterministic: none of that is settled by the one case that justified building it. An agentic RAG system that costs ten times what it should is that unexamined risk claim coming due. My asymmetry is narrower than I stated it. It holds for the benefit claim on each side. The risk claim runs the other way.

That narrowing is what makes the practical point sharp rather than blunting it. Both directions carry an unverifiable universal claim. Only one of them gets audited.

An addition is authorized by a single case, ships with its risk claim unexamined, and is expensive to reverse once other things grow against it. A removal is asked for a standard of evidence that does not exist, and as an unroute it is the cheapest reversal in the system. Engineers apply scrutiny in inverse proportion to reversibility. That is the risk direction being backwards, and it is not a claim about logic. It is a claim about which arguments get made in review, and I have not sat in a design review where somebody asked what would have to be true for the new rung to be unnecessary in a year.

The wrong way: reading a zero off a dashboard

Most attempted descents are authorized by a bar chart of height zero.

The counter says the rung fired zero times. Everybody in the room agrees that zero means unused. Somebody says we should turn it off. Somebody else, correctly but unhelpfully, says it makes them nervous. The meeting ends without a decision, or it ends with a deletion nobody can defend.

The problem is that a counter of zero carries no information about how much traffic produced it. Zero looks the same at every scale. The chart cannot distinguish a rung that survived heavy traffic untouched from a rung that has barely been asked anything at all.

Put a confidence interval on it and the two cases stop looking alike. Standard library only:

import mathfrom statistics import NormalDistNORM = NormalDist()def exact_upper_zero_events(successes: int, trials: int, alpha: float = 0.05) -> float:    """One-sided Clopper-Pearson upper bound at level alpha. Zero events only.    Raises rather than returning a wrong number if you call it on data that    has events, because the closed form silently does not apply there.    """    if trials <= 0:        raise ValueError("trials must be positive")    if successes != 0:        raise ValueError("closed form is valid only for successes == 0")    return 1.0 - alpha ** (1.0 / trials)def wilson_upper(successes: int, trials: int, conf: float = 0.95) -> float:    """One-sided upper Wilson bound.    At zero events Wilson sits below the exact bound, so it is the friendlier    of the two. Prefer the exact bound when the number argues for a deletion.    """    if trials <= 0 or not 0 <= successes <= trials:        raise ValueError("need 0 <= successes <= trials, trials > 0")    z = NORM.inv_cdf(conf)    z2 = z * z    centre = (successes + z2 / 2) / (trials + z2)    spread = (z / (trials + z2)) * (        (successes * (trials - successes) / trials + z2 / 4) ** 0.5    )    return min(1.0, centre + spread)def rule_of_three(trials: int) -> float:    """The 3/n shortcut for the upper 95% bound when no events are seen.    Asymptotic. Unusable below roughly n = 30; the table shows why.    """    if trials <= 0:        raise ValueError("trials must be positive")    return min(1.0, 3.0 / trials)

Run that across a range of traffic volumes, all of them showing zero escalations to the rung:

All bounds one-sided 95 percent.    observed   exact upper   Wilson upper      3/n--------------------------------------------------       0 / 5        0.4507         0.3511   0.6000      0 / 20        0.1391         0.1192   0.1500     0 / 100        0.0295         0.0263   0.0300    0 / 1000        0.0030         0.0027   0.0030

My five requests are the top row. The rung answered nothing, and that same measurement does not rule out a rung that is genuinely needed on 45 percent of all diagnostic requests.

The reasoning fails here, and a bigger sample would fail the same way. Five requests could never have settled the question at any level of care, because the measurement cannot separate a rung nobody needs from one that gets called on every other page.

The table hides a trap, and two of its lines are worth quoting in a meeting.

The estimator is a choice, and one of them flatters you. At zero events the Wilson bound sits below the exact Clopper-Pearson bound at every row here: 0.3511 against 0.4507 at five requests, and the gap runs to 22 percent further down. Wilson is the friendlier number. When the number is being used to argue for a deletion, quote the exact bound and say which one you used. I drafted this article with Wilson before I checked, which is how I know the temptation is real.

The rule of three is a trap at small n. The familiar shortcut says that with zero events in n trials, the upper 95 percent bound is about 3/n. It is asymptotic. At five requests it reports 0.60 against an exact 0.4507, overstating by a third, and below three requests it returns a number above 1.0 unless you clamp it. By a hundred requests it reports 0.0300 against an exact 0.0295 and is free. Use it for the large-traffic illustration, never for the case you actually have.

Clean traffic does not get you to zero. A thousand requests with no escalations bounds the true rate at 0.30 percent. The point estimate is still zero, and the bound is not a forecast. What the data cannot do is rule out a rate as high as three in a thousand. If three in a thousand is a rate you would have built the rung for, this measurement has not settled anything.

The Induction Gap: what production logs can and cannot prove

The distance between what a log can establish and what a removal decision requires is thoroughly understood in one field and unnamed in another.

If you have done off-policy evaluation, you have already met it, as a positivity violation - sometimes called an overlap violation. You cannot estimate the value of a policy that takes actions your logging policy never took, because the importance weight is undefined wherever the propensity is zero. That is not an analogy for the situation here. It is the same object. Your logging policy always routes to the rung. The candidate policy never does. The propensity of the candidate action under the logging policy is exactly zero, so no estimator built from that log recovers the candidate's value, at any sample size.

What nobody does is apply it to architecture decisions, and that is the only thing I am claiming a name for.

The Induction Gap is the positivity violation as it appears in a layer-removal decision: the difference between the existence claim your telemetry can support ("these requests were handled without the rung") and the universal claim your removal asserts ("no request needs the rung"). Telemetry closes the first by construction, because it is a record of things that happened. It cannot close the second at any volume, because your logging policy never generated a single observation of the world you are proposing to create.

This is where teams get stuck. They respond by collecting more data. More data narrows the interval and leaves the shape of the claim untouched. Drive the upper bound from 45 percent down to 0.3 percent and you are still holding an existence claim while trying to write a universal one.

Readers will reach for older ideas here. The closest three do a different job.

Chesterton's Fence says do not remove what you do not understand. That is a knowledge requirement and it is satisfiable: read the code, interview the author, understand the fence completely. The Induction Gap survives all of that. I understand my agent rung perfectly, because I wrote it, and understanding it tells me nothing about whether next month's traffic will need it.

Absence of evidence is not evidence of absence (Altman and Bland, BMJ, 1995) states the same instinct statistically. It is a warning. The Induction Gap says where the problem comes from and which step to take next.

Hume's problem of induction is the ancestor of all of it. Hume's point is that induction never reaches certainty in any direction. The point here is narrower and more useful: the shape of the claim flips when you reverse the decision, so the same telemetry that closes the argument going up cannot close it coming down. That is what makes this an engineering problem rather than a philosophical one.

The operational test is one question: would this number look different if the layer were genuinely necessary? If the answer is no, your number measures your routing. Route differently and the number changes.

Which zero are you actually looking at?

A counter reading zero is at least three different facts. They render identically on every dashboard I have seen, and they call for opposite actions.

Rows two and three are facts about your instrumentation. Teams read them as facts about their architecture.

The middle row is the common one and it is nearly invisible. An unrouted component emits nothing, so a dashboard built from decisions draws the same flat line for "never chosen" and for "never offered". Nothing on the chart marks which one you are looking at.

Why dependency graphs point at the wrong layer to remove

My system has a rung above the agent one. That multi-agent rung is held by nothing, and it is the most expensive architecture in the design, the one that took longest to build. No other rung imports it. Nothing routes to it. Structurally it is the only layer I could delete today without breaking a single test.

It is also the layer I have the least evidence about, and for exactly the same reason. Nothing routes to it, so nothing depends on it, and nothing has learned anything about it. It has no counter at all.

The two rankings invert, and they invert for one reason. The import graph ranks removability. The evidence ranks warrantedness. In a ladder architecture those are anti-correlated by design, because the expensive top rung is deliberately the least-wired one.

I am not going to say a deletability tool would have been wrong, because the article's own argument is that the question is not answerable from the codebase. That is the point. A tool reading the import graph is answering a different question confidently, and the confidence is what does the damage.

The tooling instinct here is strong, so state the rule plainly. Low coupling means a layer is easy to remove. It says nothing about whether removing it is correct. Those are different questions. Your codebase answers only the first one.

What is actually provable: non-inferiority testing and what it costs

Statistics solved this years ago. That objection is correct, so here is what the solution costs. It also assumes an evaluation capability that most teams have not built.

An ordinary significance test can only fail to find a difference, which is why "we saw no difference" is such weak grounds for deletion. Non-inferiority testing is built for exactly this claim. You pre-specify a margin: the largest drop in answer rate you are willing to accept. Then you test whether the short ladder is worse by more than that. If it is not, you have licensed a real conclusion.

So non-inferiority testing works. The interesting part is the bill. Same module as above, so NORM is already in scope:

def noninferiority_n(    baseline: float, margin: float, alpha: float = 0.025, power: float = 0.80) -> int:    """Requests per arm to show the short ladder is not worse by more than margin.    One-sided, assuming the true difference is zero. Sensitive to baseline:    feed it the ends of the baseline's own interval before quoting a number.    """    z_alpha = NORM.inv_cdf(1 - alpha)    z_beta = NORM.inv_cdf(power)    return math.ceil(        (z_alpha + z_beta) ** 2 * 2 * baseline * (1 - baseline) / margin**2    )

My answer rate without the rung is 0.80. That figure is four requests out of five, so before quoting anything from it, run its own interval through the function:

  baseline 0.376 ->  1,474 requests per arm  baseline 0.800 ->  1,005 requests per arm  baseline 0.964 ->    218 requests per arm

Three numbers, and the spread is the finding. A baseline of four-out-of-five has a Wilson interval running from 0.376 to 0.964, so the honest answer is "somewhere between two hundred and fifteen hundred per arm," not 1,005. Printing 1,005 with a thousands separator would be the same error this article is about, committed one section after naming it. I have five requests, which is short of the low end by a factor of forty.

That number comes with conditions, and the conditions are the whole argument.

It requires a margin. A margin is a business judgement. No volume of data will produce one for you. Refusing to state one is the actual failure in most descent arguments. Teams hide that refusal behind statistical vocabulary.

It requires counterfactual execution, and a passive log contains none. That does not make a live rollout the only instrument, and the cheaper grade is the one most readers can act on this week.

Replay and shadow execution push real requests through the shortened ladder without exposing anybody. Replay runs logged requests offline. Shadow runs the short ladder beside the long one, serves the long one's answer, and records where they diverge. Neither needs routing control, online evals, or a single affected user.

What neither recovers is the behavioural half. Whether a user escalates, retries, or gives up is not in a shadow run, because nobody was ever served the shorter answer. For that you need the rollout. Every measurement in this article is a replay, which is the cheap grade, and I will not present it as the expensive one.

It licenses a claim only over the traffic distribution you sampled. The tail that motivated building the rung is, by construction, the part least likely to show up in a sample that size.

The bounded claim is provable. It costs several hundred to a couple of thousand diagnostic requests I do not have, and a deployment I can undo in one config change.

The bounded claim is the one you actually need

I have been avoiding something, and a careful reader will have spotted it by now.

Nobody in production ever needed the universal claim. No engineer requires "no request will ever need this rung." What they require is that the expected cost of being wrong is smaller than the cost of the rung, which is a bounded claim, and bounded claims are provable. A reader who has got this far is entitled to say: so the answer is run an experiment, which I already knew.

The section above survives it unchanged. The margin is a judgement nobody can measure for you, and the evidence still has to come from counterfactual execution rather than logs. Neither gets easier because the claim got smaller.

Then there is the objection that framing invites. A Bayesian will point out that a prior over the unobserved tail, plus a loss function, settles this today at five requests with no experiment at all. That is correct, and it relocates the problem rather than dissolving it. Eliciting a prior over traffic you have never seen is the same act of judgement as choosing the margin, wearing different notation.

The claim is narrower than "you cannot prove it," and more useful. The decision needs one number that measurement cannot supply, and one kind of evidence that logging cannot supply. The first is a judgement you have to own. The second is the part you can actually fix, and fixing it is cheaper than everyone assumes.

How to remove a layer safely: substitute reversibility for evidence

When a claim cannot be closed by evidence, stop trying to close it and change what you are asking permission for. Reversibility does the same work in workload placement, for the same reason.

Do not ask permission to delete the rung. Ask permission to stop routing to it, in a change that can be undone without a deploy. Then let the traffic answer the question.

This is a two-stage move and the stages are not the same act:

Unroute. Take the rung off the ladder. In mine that is a single line. It ships on the next deploy, and putting it back demands no understanding of the system at all. That is exactly the property you want at 3am, when the person restoring it will not be you. The code stays.

Unbuild. Delete the module. This one is irreversible in any practical sense, and it is the step everybody argues about. It is also the step nobody has to take on a deadline, because dormant code that receives no traffic bills you almost nothing.

Splitting those two is what makes the descent tractable. People read the unroute as a safety measure. It does the work of an experiment. The first stage generates the evidence the decision was missing. Run the short ladder for a week and you have built the sample the argument required. A passive log could never have supplied it, because what you need is a record of what happens without the rung, and the rung was always there. Replay gets you part of the way. Only the rollout gets you the rest. Either way the instrument is the change itself, run somewhere it can be taken back.

Both paths, end to end:

flowchart TD
    Z["Counter on the rung reads zero"] --> Q{"Is the population<br/>stated?"}

    Q -->|"No - read as a verdict"| W1["Delete the rung"]
    W1 --> W2["Unobserved traffic arrives"]
    W2 --> W3["Incident, then rebuild<br/>under time pressure"]

    Q -->|"Yes - declared before reading"| R1["Unroute: take the rung<br/>off the ladder"]
    R1 --> R2["Run live traffic against<br/>the short ladder"]
    R2 --> D{"Escalations appear<br/>in the stated population?"}

    D -->|"Yes"| R3["Restore the route.<br/>You now have the case<br/>that justifies the rung"]
    D -->|"No, and the population could have detected them"| R4["Unbuild: delete the module"]

    style Z fill:#95A5A6,color:#FFFFFF
    style Q fill:#FFD93D,color:#2C2C2A
    style W1 fill:#E74C3C,color:#FFFFFF
    style W2 fill:#E74C3C,color:#FFFFFF
    style W3 fill:#E74C3C,color:#FFFFFF
    style R1 fill:#4A90E2,color:#FFFFFF
    style R2 fill:#4A90E2,color:#FFFFFF
    style D fill:#FFD93D,color:#2C2C2A
    style R3 fill:#6BCF7F,color:#2C2C2A
    style R4 fill:#6BCF7F,color:#2C2C2A

Both branches of the lower half are wins. If escalations appear, you have obtained the reproducible case that authorizes keeping the rung, which is the artifact you never had. If none appear across a population large enough to have detected them, you have earned the deletion.

The upper branch is the one that looks decisive. It is also the one where the rebuild gets argued in an incident channel.

Why dead code tooling cannot find an idle AI layer

At this point somebody reasonably asks why this is hard when large companies delete code at enormous scale.

Meta's SCARF system has automatically deleted more than 100 million lines of code across over 370,000 change requests, including committing the deletions and dropping the tables behind them. That is real, it is published, and it is far beyond what most teams manage.

SCARF works from an augmented dependency graph that merges compiler-derived static reachability with runtime logs, with textual reference search as a fallback safety net for dynamic languages. It deletes where a proof of unreachability exists, and where that proof is machine-checkable it can auto-merge.

An overbuilt architecture rung has no such proof available. The rung is reachable. It executes on every request that routes to it, returns results, and the results are correct. Nobody needs it. There is no static analysis that concludes "this layer is reachable, functioning, and unnecessary", because unnecessary is a property of the traffic, and static analysis only ever sees the program.

Dead code removal is a solved problem. Live code removal is not, and an idle rung is live code.

The same distinction explains why the tooling everyone points at does not help. Dependency analysis finds unreachable code. Coverage finds unexecuted code. Neither has any opinion about code that executes correctly and changes no outcome. No static tool does. The dynamic ones that do exist - experimentation platforms, holdout groups, per-route outcome attribution - are wired at feature granularity, not architecture-layer granularity, which is why nobody points them at a rung. That granularity mismatch is the same gap the 37.3 percent figure below measures.

Why "removal is just expensive" does not explain it

The received account of why systems never simplify is Lehman's second law of software evolution: as a program evolves, its complexity increases unless work is done to reduce it. Lehman himself suggested it might be an analogue of the second law of thermodynamics, and the framing has been repeated for forty-five years.

It is on weaker empirical ground than its reputation suggests. A systematic literature review of the laws of software evolution reports little and mixed empirical validation for law II, while the laws on continuing change and continuing growth held up across the studies that tested them. The scope is also narrower than most citations admit: Lehman asserted the laws only for E-type programs, the ones embedded in a real-world environment that they in turn change.

I am not claiming the cost account is wrong. Coordination cost, dependency gravity, and knowledge decay are all real, and Google's chapter on deprecation in Software Engineering at Google is the best statement of them. Read it in full. It contains the line this whole argument circles: "hope is not a strategy." Google is describing advisory deprecation, which has no deadline and relies on teams voluntarily migrating. My point is adjacent. For an architecture-layer removal, hope is not merely a bad strategy. Absent an intervention, it is the only thing on offer.

What would actually settle whether the two barriers are independent is a two-by-two: cases where removal is cheap and evidence is absent, and cases where removal is expensive and evidence is available. I have one instance in each of two cells and nothing in the other two, so I am not going to claim it is settled.

What the record does show is that an existence proof unblocks decisions that an absence of escalations does not. McDonald's ended its automated drive-through order-taking test across more than 100 restaurants in June 2024, and Taco Bell re-scoped its voice system at busy sites during 2025. Both moved once the failures were undeniable. The caveat matters though: both were vendor pilots with defined endpoints, which are structurally cheaper to end than a layer with years of dependents. They show a closeable argument getting closed. Coordination cost may still have mattered in both.

Organisational friction slows a descent down. The Induction Gap stops it from starting. Those are different failures and they need different fixes.

What the Klarna reversal actually shows

Klarna's numbers were excellent. The outcome was bad anyway.

Klarna reported in early 2024 that its AI assistant was handling 2.3 million conversations, about two-thirds of its customer service chats, cutting average resolution time from 11 minutes to under 2, and doing the work of roughly 700 agents. Those are the company's own figures, never independently audited. By 2025 it was rehiring human agents and rebalancing toward a hybrid model, the chief executive saying the focus on cost had produced lower quality that was not sustainable.

Two readings are available and I cannot rule either out. Either the figures were accurate and the aggregate still authorized a bad decision, or the figures were vanity-shaped - resolution time measured over the chats the assistant chose to handle, deflections counted as resolutions - and the reversal is just a correction. I should also concede the structural objection: Klarna ascended a ladder rather than descending one. It added an AI rung, and the human rung atrophied through a hiring freeze as a side effect.

What survives both readings is the part I care about. The chart was an aggregate over the traffic the assistant selected, which is clause one of the Removal Receipt violated at the source. If the population is chosen by the thing being evaluated, the number it produces cannot authorize anything, and it makes no difference whether the number was honest.

Klarna's descent was executed as an unbuild, with headcount reduced through a hiring freeze and attrition. That is the irreversible stage. Getting back meant new hiring, a hybrid product, and a reversal announced to the same press that covered the launch.

Can production trace data authorize removing a layer?

The most recent academic work on descending this ladder is a July 2026 preprint on what it calls progressive crystallization: converting repeatedly validated agent behaviours into deterministic workflows, reported as moving from 0 to 45 percent deterministic execution over eight months with more than a 70 percent reduction in per-incident agent cost. It is a single-author preprint with self-reported numbers from one production system, so treat the figures accordingly.

Taken at face value it contradicts this article, because it claims an evidence-based promotion mechanism driven by observed agent behaviour - exactly the passive-log authorization I have said cannot work.

The design says otherwise. Its execution taxonomy has three stages, and the middle one is hybrid, which means the agent path remains available while the deterministic path handles what it can. The abstract contains no confidence intervals, no sample-size justification, and no discussion of what licenses a demotion decision.

The retained fallback is doing the epistemic work here. The evidence is along for the ride. That is an unroute with the route still warm, described as evidence-based promotion. The paper reads as a counterexample and works as a confirmation: the most rigorous recent attempt at this problem reached for reversibility without naming it as the thing that made the descent safe.

A note on "rung": Pearl's ladder of causation and this one

If you work in causal inference, "rung" is Judea Pearl's word. I kept it anyway, because the collision turns out to be useful.

Pearl's ladder of causation has three rungs: association, intervention, and counterfactual. A usage dashboard is association. It reports what co-occurred in the regime you observed. An unroute is intervention. You change the system and watch what happens. "Would any request ever have needed this layer?" is counterfactual, which is the top rung and is not available to you at all.

A dashboard cannot authorize a removal because a removal is an intervention question asked of association data. The scream test moves you up a rung of evidence. No amount of care on the dashboard gets you there.

I should not oversell the mapping. An unroute on a slice of traffic is a weak intervention, not a clean do-operator, because you are still choosing which traffic sees it. Rung two is generous.

How to decide whether to remove an AI architecture layer

The 7 GenAI Architectures, the book this series accompanies, calls the artifact that authorizes a descent a Removal Receipt and gives it four clauses. Use them in this order, because the order is what stops the argument going wrong:

1. Declare the population before you look. Which requests, over what window, on which ladder. The order matters because reading the number first corrupts the choice. You will not see it happen. Narrow the window to where the rung was reachable, set aside the outage, discount the one incident everybody agrees was atypical, and the zero you finish with is a zero you assembled.
2. Report the counter with its denominator. Never a bare zero. "0 of 5" and "0 of 5,000" are different facts and must not render identically.
3. Compute the smallest effect your sample could have caught. If the honest answer is that it could not have caught even the largest effect arithmetically available, say that out loud, and stop treating the zero as a finding.
4. Keep the first step reversible without shipping code. Unroute, do not unbuild. The module stays.

As the book puts it, three of those four without the fourth is "a preference with a chart attached." Clauses one to three will usually be thin, because most teams arrive at this decision with a handful of observations in hand. Clause four is what makes a thin case actionable regardless.

Two operational rules go with it:

Order the ladder so the expensive rung is the removable one. If your most costly rung is also the one wired into the most call sites, you have made the descent structurally harder than it needed to be. Decide this while building, not while cutting.

Assume you cannot run the test yet. The LangChain State of Agent Engineering survey, fielded over late 2025 with 1,340 respondents, found only 37.3 percent running online evaluations at all, rising to 44.8 percent among teams already in production. Treat that as a proxy for the routing-plus-measurement capability rather than a measurement of it, since the two are not the same thing. Either way the instrument is missing in most teams. Build the ability to route a slice of live traffic through a shortened ladder first. You will use it again long after this one deletion.

Replace the evidence requirement with a reversibility requirement

Complexity in a GenAI system accumulates for a reason deeper than laziness and wider than cost. Every rung you added was authorized by a proof. No rung you might remove can be, because the two decisions are different kinds of claim, and telemetry only speaks to one of them.

Waiting does not help either. The evidence you are waiting for does not exist. That is how a 26 percent overhead survives being measured, written down, and published.

Replace the evidence requirement with a reversibility requirement. Declare the population, print the denominator, unroute, and let a week of traffic tell you what no amount of history could. The rung you can safely delete is not the one you have the most data about. It is the one you have made it cheapest to put back.

My agent rung is still routed, and I want to be precise about why, because "I know and I have not acted" is not the same thing as "the argument does not work on its author."

I have run the cheap grade. Replaying the five recorded requests against the shortened ladder resolves the same four of them and spends 21 percent fewer tokens. That is genuine counterfactual evidence. It cost no traffic, no rollout, and no risk, and anybody reading this can run the equivalent against their own logs this week.

What I have never had is the other half. Nobody has been served an answer from the short ladder, so nothing in my data says whether a user would have escalated, retried, or given up. Replay cannot produce that, and more replay will not either.

The unroute is blocked exactly where this article says it is blocked: on the one kind of evidence that only exists once you make the change. Nerve has nothing to do with it. The cheap grade told me the rung is idle across the traffic I have. It cannot tell me the rung is unnecessary, and unnecessary is what a deletion claims.

I could ship it anyway. That would be a defensible call, made on judgement with the measurement absent. That is the honest description of most architecture decisions, and the thing this article is asking you to say out loud instead of dressing it up as a chart.

References

• Popper, K. (1959). The Logic of Scientific Discovery. Hutchinson. (German original: Logik der Forschung, 1934.) See also: Thornton, S. "Karl Popper." Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/popper/
• Hume, D. (1739). A Treatise of Human Nature, Book 1, Part iii, Section 6. See also: Henderson, L. "The Problem of Induction." Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/induction-problem/
• Wilson, E.B. (1927). "Probable Inference, the Law of Succession, and Statistical Inference." Journal of the American Statistical Association 22(158):209-212. https://www.tandfonline.com/doi/abs/10.1080/01621459.1927.10502953
• Brown, L.D., Cai, T.T. and DasGupta, A. (2001). "Interval Estimation for a Binomial Proportion." Statistical Science 16(2):101-133. https://projecteuclid.org/journals/statistical-science/volume-16/issue-2/Interval-Estimation-for-a-Binomial-Proportion/10.1214/ss/1009213286.full
• Hanley, J.A. and Lippman-Hand, A. (1983). "If Nothing Goes Wrong, Is Everything All Right? Interpreting Zero Numerators." JAMA 249(13):1743-1745.
• Jovanovic, B.D. and Levy, P.S. (1997). "A Look at the Rule of Three." The American Statistician 51(2):137-139. https://www.tandfonline.com/doi/abs/10.1080/00031305.1997.10473947
• Altman, D.G. and Bland, J.M. (1995). "Statistics Notes: Absence of Evidence Is Not Evidence of Absence." BMJ 311:485. https://pmc.ncbi.nlm.nih.gov/articles/PMC2550545/
• Piaggio, G., Elbourne, D.R., Pocock, S.J., Evans, S.J.W. and Altman, D.G. (2012). "Reporting of Noninferiority and Equivalence Randomized Trials: Extension of the CONSORT 2010 Statement." JAMA 308(24):2594-2604. https://doi.org/10.1001/jama.2012.87802
• Georgiev, G. (2017). "The Case for Non-Inferiority A/B Tests." Analytics-Toolkit. https://blog.analytics-toolkit.com/2017/case-non-inferiority-designs-ab-testing/
• Dudik, M., Langford, J. and Li, L. (2011). "Doubly Robust Policy Evaluation and Learning." ICML 2011. arXiv:1103.4601. https://arxiv.org/abs/1103.4601
• Sachdeva, N., Su, Y. and Joachims, T. (2020). "Off-policy Bandits with Deficient Support." KDD 2020. arXiv:2006.09438. https://arxiv.org/abs/2006.09438
• Pearl, J. and Mackenzie, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books.
• Lehman, M.M. (1980). "Programs, Life Cycles, and Laws of Software Evolution." Proceedings of the IEEE 68(9):1060-1076.
• Herraiz, I., Rodriguez, D., Robles, G. and Gonzalez-Barahona, J.M. (2013). "The Evolution of the Laws of Software Evolution: A Discussion Based on a Systematic Literature Review." ACM Computing Surveys 46(2), Article 28. https://dl.acm.org/doi/10.1145/2543581.2543595
• Winters, T., Manshreck, T. and Wright, H., eds. (2020). Software Engineering at Google, Chapter 15, "Deprecation." O'Reilly. https://abseil.io/resources/swe-book/html/ch15.html
• Shackleton, W. et al. (2023). "Dead Code Removal at Meta: Automatically Deleting Millions of Lines of Code and Petabytes of Deprecated Data." ESEC/FSE 2023. https://dl.acm.org/doi/10.1145/3611643.3613871
• Meta Engineering (2023). "Automating dead code cleanup," 24 October 2023. https://engineering.fb.com/2023/10/24/data-infrastructure/automating-dead-code-cleanup/
• Chesterton, G.K. (1929). The Thing, chapter "The Drift from Domesticity." https://www.chesterton.org/taking-a-fence-down/
• Hirning, D. (2025). "Moving from a 'Scream Test' to holistic lifecycle management." Microsoft Inside Track, 20 November 2025. https://www.microsoft.com/insidetrack/blog/microsoft-uses-a-scream-test-to-silence-its-unused-servers/
• Ministry of Testing. "Scream Testing." Software Testing Glossary. https://www.ministryoftesting.com/software-testing-glossary/scream-testing
• Kumar, R. (2026). The 7 GenAI Architectures: A Field Guide to Choosing the Right AI System - Before You Overbuild It. Chapter 16, "Descending: Rightsizing an Overbuilt System." https://www.amazon.com/dp/B0HF8PHF2H
• Hadfield, J. et al. (2025). "How we built our multi-agent research system." Anthropic, 13 June 2025. https://www.anthropic.com/engineering/built-multi-agent-research-system
• Schluntz, E. and Zhang, B. (2024). "Building Effective Agents." Anthropic, 19 December 2024. https://www.anthropic.com/engineering/building-effective-agents
• Gartner (2025). "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027." Press release, 25 June 2025. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
• Malik, A. (2026). "Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production." arXiv:2607.07052 (preprint). https://arxiv.org/abs/2607.07052
• LangChain (2026). State of Agent Engineering (survey, n=1,340, fielded 18 November to 2 December 2025). https://www.langchain.com/state-of-agent-engineering
• CNBC (2024). "McDonald's to end AI drive-thru test with IBM," 17 June 2024. https://www.cnbc.com/2024/06/17/mcdonalds-to-end-ibm-ai-drive-thru-test.html
• Nation's Restaurant News (2025). "Taco Bell is adjusting its Voice AI plans." https://www.nrn.com/restaurant-technology/taco-bell-is-adjusting-its-voice-ai-plans
• Forbes (2025). "Klarna Reverses AI Push, Says Customers Prefer Human Support," 18 May 2025. https://www.forbes.com/sites/quickerbettertech/2025/05/18/business-tech-news-klarna-reverses-on-ai-says-customers-like-talking-to-people/

Related Articles

AI Engineering

• Cost Governance and Budget Allocation Across Agent Types: Token Spend Is Infrastructure Spend
• Agentic AI Observability: Why Traditional Monitoring Breaks with Autonomous Systems
• Why Your Agentic RAG System Costs 10x More Than It Should

System Design

• The 7 GenAI Architectures Every AI Engineer Should Know