Here is a graph that cannot loop forever. It has a recursion limit of 8. The whole reproduction is below, so you can run it rather than take my word for any of this.
from typing import NotRequired, TypedDictfrom langgraph.graph import StateGraph, STARTfrom langgraph.checkpoint.memory import InMemorySaverclass L(TypedDict): n: NotRequired[int]def spin(state: L): return {"n": state.get("n", 0) + 1}lg = StateGraph(L)lg.add_node("spin", spin)lg.add_edge(START, "spin")lg.add_conditional_edges("spin", lambda s: "spin") # unconditional self-looploop_app = lg.compile(checkpointer=InMemorySaver())lcfg = {"configurable": {"thread_id": "t-loop"}, "recursion_limit": 8}for attempt in range(1, 5): try: # None input on every attempt after the first: resume this thread. loop_app.invoke({"n": 0} if attempt == 1 else None, lcfg) except Exception as e: snap = loop_app.get_state(lcfg) print(f"attempt {attempt}: {type(e).__name__} at n={snap.values.get('n')}, " f"checkpoint step={snap.metadata.get('step')}")Here it is looping forever. Each line is one invoke on the same thread, the way a
supervisor retries a failed run. The first call passes the initial state, and every
later call passes None, which is how you tell LangGraph to resume this thread from
its last checkpoint.
attempt 1: GraphRecursionError at n=8, checkpoint step=8attempt 2: GraphRecursionError at n=18, checkpoint step=18attempt 3: GraphRecursionError at n=28, checkpoint step=28attempt 4: GraphRecursionError at n=38, checkpoint step=38TOTAL supersteps burned across 4 resumes with recursion_limit=8: 38Thirty-eight supersteps against a limit of eight. Nothing is broken. No exception was
swallowed, no configuration was overridden, and the limit was enforced correctly every
single time. Each invoke stopped at its bound. The bound just moved.
This is not a bug report. recursion_limit behaves exactly as its implementation says
it does. The problem is what its implementation says.
The thesis: your guards are recomputed, not remembered
Every bound the runtime hands you by default is a pure function of checkpointed state.
The recursion limit is derived from the step counter in checkpoint metadata. The refund
guard from Part 3 reads refund_allowed
and refund_checked_for out of graph state. A human approval is a resume value delivered
back into the same run. All of them ask the state a question and act on the answer.
None of them stores how much of the bound has already been used in a way the next derivation has to respect. So the next derivation starts clean.
That is the whole article. A checkpointer's job is to return your run to a known good past, and your guards are computed from that past. Recomputing them returns them to the condition they were in before they were spent - not because anything failed, but because nothing in the mechanism was ever asked to remember that they were spent.
The two bodies of work that should have caught this both miss it, and they miss it in the same way. Both assume that the danger in replay is that replay goes wrong.
Jack Vanlightly's account of determinism in durable execution is the clearest statement of the orthodox position. His worked failure is a double charge caused by control flow diverging on replay: the first execution falls inside a promotional date window and charges a discounted card, and the replay runs after the window closes, takes the other branch, and charges again. His prescription follows from the diagnosis. Control flow must be deterministic, side effects must be idempotent or tolerant of duplication.
The security literature makes the same assumption from the other side. ACRFence, which I will come back to in detail, roots its attacks in model non-determinism: a restored agent "re-synthesizes subtly different requests after restore," and servers accept them as new transactions because "all existing protection mechanisms share the same assumption: the caller will send identical requests on retry."
Divergence is the enemy in both accounts. Make replay faithful and the problem goes away.
It does not go away. The failure in this article needs no divergence and no non-determinism. Set temperature to zero, make every branch deterministic, replay the run perfectly, and it still happens - because a guard that faithfully re-derives the same verdict from the same restored state is a guard that authorises the same action twice. The faithfulness is the bug.
How LangGraph recomputes recursion_limit on every resume
The step budget is computed in SyncPregelLoop.__enter__ and its async twin, in
libs/langgraph/langgraph/pregel/_loop.py. Two lines, after the checkpoint has loaded:
self.step = self.checkpoint_metadata["step"] + 1self.stop = self.step + self.config["recursion_limit"] + 1And the halt, checked on each tick:
if self.step > self.stop: self.status = "out_of_steps" return FalseRead the second line again. self.stop is not a property of the thread. It is
recomputed on every entry into the loop, relative to wherever the checkpoint left the
counter. Resume a thread parked at step 8 with recursion_limit=8 and you get
stop = 9 + 8 + 1 = 18. Resume it at 18 and you get 28.
The counter is doing its job. It is monotonic, it persists, and it correctly records how far this thread has travelled. It is the budget that resets, because the budget is derived from the counter at entry rather than stored alongside it.
Do the subtraction and the useful form falls out. The allowance on any entry is
stop - step_entry + 1 = (step_entry + recursion_limit + 1) - step_entry + 1 = recursion_limit + 2 ticksThe starting offset cancels. Every entry gets recursion_limit + 2 ticks, no matter
where the counter is. That is the whole mechanism, and it is a cleaner statement than
anything about where stop lands.
It also explains the 8-versus-10 split in the transcript, which is not what I first
assumed. A fresh run spends two of its ticks on work that is not the loop node - applying
the input and entering the graph - so 8 of the 10 execute spin. A resume with None
input spends none of them, so all 10 do. Instrumenting the node confirms it: 8 executions
on the first invocation, then 10, 10, 10.
Two smaller details, because both are easy to get wrong. The check itself does not raise:
it sets status to "out_of_steps" and returns False, and GraphRecursionError
surfaces further out, in Pregel.stream() after the loop exits. And a fresh run starts
from checkpoint_metadata["step"] = -2, which is an undocumented internal sentinel - it
is in the source, not in the manual, so do not build on it.
Why the counter looks fine and the bound is not
Part 1 said that "range(max_steps) restarts at
zero on every resume, so a run that crashes repeatedly has no effective length bound at
all." For the hand-rolled loop it was describing, that is exactly right - a Python for _ in range(25) starts at zero every time you call the function, and its conclusion is the
one this article spends its length confirming.
What I want to kill is the inference, not the sentence. Part 3
forwarded that deferral as "max_steps resetting to zero on resume," and it is a short
step from there to assuming LangGraph's counter does the same thing. It does not.
The counter does not reset to zero. It continues from the checkpoint, exactly as you
would want. If it did reset to zero, the bug would be obvious the first time anyone
printed langgraph_step after a resume, and someone would have filed it years ago.
What actually happens is subtler and harder to see: the counter is honest and the budget is not. A monotonic counter that everyone can inspect sits right next to a bound that is silently re-granted, and the two are related by a line of arithmetic in a context manager. Nothing you print will look wrong.
Self-Restoring Bound
I am naming this, because it is not specific to recursion_limit and it needs a handle
that generalises.
A Self-Restoring Bound is a limit, grant, or quota whose remaining allowance is re-derived from checkpointed state on every entry, rather than stored as a spent total that the derivation has to respect.
Two conditions:
- The allowance is computed, not carried. Something reads state at entry and works out how much is left, instead of reading a running total of how much is gone.
- Something re-enters that derivation. A resume, a retry, an interrupt, a second turn on the same thread, a supervisor re-invoking a failed run.
That produces two species, and they are worth separating because they fail differently.
The spend record is inside the rollback surface - or there is no spend record at all.
ACRFence's approval token is the full case: consumption is recorded, in state, and the
rewind erases the record. Part 3's refund grant is the degenerate case, and it is the more
common one: nothing ever marks it spent, so recovery has nothing to unmark. refund_allowed
is set, the refund fires, the process dies before the write lands, and the grant is sitting
there exactly as it was. Either way the guard re-derives True.
The human approval belongs here by effect but not quite by mechanism, and the difference is worth keeping straight: a resume value is not rolled back, it is replayed forward into the task from pending writes. Nothing is restored. The answer is simply re-delivered, which gets you to the same place.
The spend record survives, but the allowance is recomputed relative to it rather than
against it. This is recursion_limit. Nothing is rolled back at all - the step counter
is durable, monotonic and completely honest, which is what makes it so hard to see. The
budget is derived by adding to that counter instead of subtracting from a cap, so it
arrives full no matter what the counter says.
The second species does not even need a crash. A clean run followed by an ordinary second turn on the same thread re-enters the derivation and gets a fresh allowance. Recovery is the most common way to re-enter a bound, not the only way, and calling this a recovery-only problem would understate it.
Miss either condition and you are fine. A bound derived from an absolute wall-clock deadline carries its own spend record in the clock, because the clock moved even though the checkpoint did not. A grant whose consumption is recorded in a database the checkpointer does not own is a running total the derivation cannot ignore.
A relative per-invocation timeout is not in that safe set, and it is the one people
reach for. timeout=600 is re-derived at entry by adding to the clock rather than
subtracting from a cap, which is species two word for word.
This is the fourth bound this series has had to name. Part 1 separated the length bound (recursion_limit caps how far a run
goes) from the shape bound (which caps where it can go). Part 3 added the alphabet
bound (which fixes what actions exist, and says nothing about when). A Self-Restoring
Bound is not a fourth kind of bound. It is a property any of the three can have, and
recursion_limit - the original length bound - has it.
I should say plainly that a fuse that resets itself is an old idea and a good name for it is already taken. US patent 4024488, filed in 1975 and granted in 1977, describes a "self-restoring type current limiting device": a material that vaporises under overcurrent, limits the current, and then "is rapidly self-restored to its original electrically conductive state." Different field, no confusion, and honestly a fair picture of what is happening here. The protection trips, the fault clears, and the protection quietly makes itself available again.
Recursion limits, refund grants, and interrupt approvals: the same bug three times
Once you have the name, the pattern shows up in places that look unrelated.
The runtime's own bound. recursion_limit, as measured above. The spend record is
the step counter, which lives in checkpoint metadata - inside the rollback surface,
because the checkpointer is what writes it.
An authorisation. Part 3 shipped a refund guard that went through four wrong versions before it worked. Here is the version that survived peer review:
authorised = ( state.get("refund_allowed") is True and state.get("refund_checked_for") == (args.get("ticket_id"), args.get("cents")))Both conditions are pure reads of restored state. Neither is consumed by a successful
dispatch. check_entitlement writes the grant, the checkpoint makes it durable, the
guard reads it and allows the refund, billing.refund() moves the money - and if the
process dies before that superstep's checkpoint lands, recovery restores a state in
which the grant is still sitting there unspent. The guard is re-entered and returns
True, correctly, for the second time.
That is a fifth defect in a guard I corrected four times and then shipped. It is not visible from inside Part 3's frame, because Part 3 was reasoning about a single run.
A human approval. interrupt() parks a run, a human approves, Command(resume=...)
delivers the answer. The approval is now a resume value the runtime replays back into the
task on re-entry - held in pending writes under a reserved channel rather than as an
ordinary state key, though the distinction does not save you. It is a
Self-Restoring Bound with a person attached to it. Worse, LangGraph restarts the node
from the top on resume, so this is not a rare crash-window problem. It is the documented
normal path.
The interrupt docs are explicit about the mechanics: "When execution resumes (after you
provide the requested input), the runtime restarts the entire node from the
beginning - it does not resume from the exact line where interrupt was called. This
means any code that ran before the interrupt will execute again." There is a section
headed "Side effects called before interrupt must be idempotent," and it names the
anti-patterns directly, including writing an audit-log record before the interrupt,
which "will create duplicate records on each resume."
This is not a hidden footnote. It is in the manual. What the manual does not say is that the approval itself is one of the things that comes back.
You must resume on the same thread. The docs are explicit: "You must use the same
thread ID when resuming that was used when the interrupt occurred." Resume on a new
thread_id and you do not get an error, you get a fresh run with empty state.
The resume value is the return value. Command(resume=...) supplies what the
interrupt() call evaluates to. It is also the only Command form meant to be passed
into invoke or stream; update, goto, and graph are for returning from node
functions.
Multiple interrupts in one node match by position unless you opt out. With a scalar
Command(resume=value), LangGraph keeps a list of resume values per task and "matching is
strictly index-based, so the order of interrupt calls within the node is important."
Reorder two interrupt() calls between deploys and parked threads resume with the answers
swapped - the approval intended for one question delivered as the answer to another. The
escape is the map form: 1.x surfaces Interrupt(value=..., id=...) and accepts
Command(resume={interrupt_id: value}), which matches by identity. Use the map form for
any node with more than one interrupt.
In practice: interrupt() first in the node, one irreversible action after it, and
nothing else in that node.
That ordering stops pre-interrupt side effects from repeating. It does not make the approval one-shot, and I want to be explicit about that because this is the scariest case in the article and it needs the same treatment as the refund. If the node re-runs after the approval was delivered - a later crash in that node, or the partial-resume path in #6208 below - the resume value is re-supplied from the task scratchpad and the effect fires again.
So give the approval a ledger row too. Mint an approval_id when the interrupt is
created, derived the same deterministic way, and consume it immediately before the
irreversible action. It is the same compare-and-swap, and it turns "a human said yes" from
a value that comes back into a grant that gets spent.
Someone got to the authorisation case first
The authorisation instance of this belongs to someone else, and it is recent enough that I nearly published a duplicate of it.
ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore (Zheng, Yang, Zhang and Quinn, arXiv:2603.20625, March 2026) names two attack classes. The second is Authority Resurrection, and their example is a manager's single-use approval token: the agent obtains it, executes a deletion, the token is marked consumed, and then a rewind restores the agent "to the state just after approval was granted." Their statement of the mechanism is the one I would have written: "The agent now holds the token but has no memory of having used it," and so "has effectively escalated a legitimate, scoped authorization into an unauthorized action on a different target."
Their threat model covers the ordinary case too, not only the malicious one. They list "Crash-Induced Restore, where an external attacker triggers a crash after an irreversible action, causing the framework to auto-restore" alongside deliberate rollback abuse by an insider. So I cannot claim the non-adversarial case as untouched ground either.
What is left, and what this article is actually about, is narrower than it first looked:
- The generalisation past authority. A recursion limit is a bound but not an authority. ACRFence's two attack classes are both about tool effects and credentials. The step budget is neither, and it is re-granted by the same mechanism.
- The independence from non-determinism. Their stated root cause, and the assumption their defence is built around, is that a restored agent emits a different request. Their Action Replay result measured the identical case too - "all 10 checkpoint-restore trials produced duplicate commits" - so I cannot claim they never saw it. What survives is narrower: their remedy classifies a post-restore call by whether it is semantically equivalent to the recorded one, and on a match it returns the recorded response. The failure here needs no classification, because the call is identical and the guard authorising it is deterministic. Their Replay branch would handle it correctly. It would do so by answering from a record that lives outside the checkpoint, which is the same move this article ends up making.
- The LangGraph mechanics. Their testbed is Claude Code CLI backed by Qwen3-32B.
LangGraph appears in their survey table, citing a maintainer's acknowledgement, and was
never experimentally tested. Everything measured in this article was run against
langgraph 1.2.9.
The older ancestor has a name too. One-shot capabilities are about fifty years old. Miller, Yee and Shapiro's Paradigm Regained (2003) observes that raw capabilities grant "only unconditional, full, perpetual access to the objects they designate." The revocation patterns that answer it go back to Redell's 1974 thesis, in the tradition Saltzer and Schroeder set out the year after. The closest contemporary neighbour is PORTICO, which names lingering authority for a capability still exposed after the episode that justified it has closed. That is authority outliving its scope. A Self-Restoring Bound is authority outliving its use. The classical fix for perpetual authority is to mark it spent.
That fix is exactly what durability defeats. If you mark it spent in graph state, the mark is inside the rollback surface, and recovery unmarks it.
Halfway: move the counter into state
The obvious response is to stop relying on recursion_limit and count the steps yourself.
class Spin(TypedDict): n: NotRequired[int] steps: NotRequired[int]def spin(state: Spin): used = state.get("steps", 0) + 1 if used > 8: raise RuntimeError("step budget exhausted") return {"n": state.get("n", 0) + 1, "steps": used}This is better than what it replaces, and it is still not good enough. The gap between those two is the instructive part, so I want to be exact about it rather than wave it away.
It is better because steps is a running total the derivation cannot ignore. It carries
across invocations. A supervisor that re-invokes forever no longer gets a full budget
every time, so the loop actually terminates - which is more than recursion_limit does.
On the article's own opening scenario, this fixes the bug.
It is not good enough because steps is a state key, and state keys are what the
checkpointer restores. Crash at step 7, resume, and steps comes back as whatever the
last durable checkpoint said. Under the default durability="async" that may be several
supersteps behind where the process actually got to. The count is not wrong in a
direction you can bound: you asked for a cap of 8 and you have a cap somewhere in
[8, 8 + whatever the crashes ate], and nothing reports the difference.
An approximate safety bound is not a safety bound. If the number exists to stop a runaway loop from spending real money, "8, or possibly 11, and you will not be told which" is not a control you can put in a design document. You have also added a state key, which means Part 2's reducer rules now govern it: pick its reducer deliberately, because the default is last-write-wins.
The fix is not to abandon the counter. It is to move it somewhere the checkpointer does not restore, which costs about the same and gives you an exact number.
This matters more than a toy example, because Part 3
recommended ToolCallLimitMiddleware(thread_limit=20) to you as "a genuinely useful"
per-thread bound, and that middleware is the Halfway pattern. Its source declares
state_schema = ToolCallLimitState and reads its counter with
state.get("thread_tool_call_count", {}), so the count is an ordinary graph state key. It
does what the hand-rolled version does: survives a clean resume, rolls back with the
checkpoint on a crash, and under-counts by an amount nothing reports.
That is still better than recursion_limit, and I am not withdrawing the recommendation.
But "thread_limit" reads like a per-thread guarantee and it is a per-thread estimate. If
you are using it to bound spend rather than to catch a runaway loop, move the counter out.
Right way: put the spend record outside the rollback surface
The fix is structural. The record of having spent the bound has to live somewhere the checkpointer does not control.
This is the same shape as ACRFence's stateful validation, which they measured at 0 of 2 token reuses succeeding against 2 of 2 with stateless validation. I am not claiming the mechanism. What is added here is where it sits relative to the checkpointer, and the observation that the grant identifier can stay in graph state while only the consumption record has to move out - which is what makes this cheap enough to adopt.
The grant identifier can stay in state. That part is safe, and here is why: rolling back refund_grant_id just hands you the same identifier again, and the
ledger you check it against still says it is spent. Only the consumption record has to
be outside.
CREATE TABLE grant_ledger ( grant_id TEXT PRIMARY KEY, thread_id TEXT NOT NULL, tool_call_id TEXT NOT NULL, ticket_id TEXT NOT NULL, cents INTEGER NOT NULL, consumed_at TIMESTAMPTZ, effect_id TEXT, -- written when we decide to act effect_result JSONB, -- written when the effect returns UNIQUE (thread_id, tool_call_id));CREATE TABLE thread_steps ( thread_id TEXT PRIMARY KEY, steps INTEGER NOT NULL);The effect_id and effect_result columns are for the reconciliation problem further
down; ignore them until then. What matters here is what the grant is keyed on, because I
got this wrong the first time and the wrong version is the version most people would
write.
check_entitlement is a tool the model calls, so it sits inside the replay window like
everything else. If a replay re-runs it and a freshly minted random grant_id produces a
new row, unconsumed, the compare-and-swap below succeeds against it and the refund
goes out twice with a perfectly healthy ledger. That is ACRFence's Action Replay, where
the agent "re-issue[s] the payment with a fresh reference ID." A ledger keyed on a random
identifier walks straight into it.
So the identifier has to be derived. The obvious derivation is from the arguments -
thread_id:ticket_id:cents - and it is a trap. It cannot tell "the same intent replayed"
from "a genuinely new intent that happens to look identical," so a customer legitimately
owed a second refund of the same amount on the same ticket can never receive one.
Derive it from the tool call instead:
import uuidGRANT_NS = uuid.UUID("6f9619ff-8b86-d011-b42d-00c04fc964ff") # any fixed namespacedef mint_grant(conn, thread_id: str, tool_call_id: str, ticket_id: str, cents: int) -> str: """Keyed on the call, so a replayed dispatch collides and a new decision does not.""" grant_id = str(uuid.uuid5(GRANT_NS, f"{thread_id}:{tool_call_id}")) with conn.cursor() as cur: cur.execute( """ INSERT INTO grant_ledger (grant_id, thread_id, tool_call_id, ticket_id, cents, consumed_at) VALUES (%s, %s, %s, %s, %s, NULL) ON CONFLICT (thread_id, tool_call_id) DO NOTHING """, (grant_id, thread_id, tool_call_id, ticket_id, cents), ) return grant_idtool_call_id is the right key because it is stable in exactly the cases where stability
is what you want, and unstable in exactly the cases where it is not. I checked both halves
rather than assuming:
serde round-trip of an AIMessage carrying a tool call before: call_abc123 after : call_abc123 stable: Truethrough the checkpointer emit:call_abc123 dispatch:call_abc123 after restore from get_state: call_abc123 stable: TrueUnlike the tuple further down, a tool call id is a string and survives serialisation as itself. So a replayed dispatch of a decision the model already made carries the same id and lands on the same row. A model that runs again and re-decides emits a new id, gets a new row, and is treated as the new attempt it actually is.
That last sentence is also the limit of this fix, and it is worth being blunt about.
Keying on the call protects the dispatch, not the decision. If the replay window reaches
back far enough to re-run the model and check_entitlement authorises again, you get a
new call id, a new grant, and a second refund. The ledger cannot see that those two
decisions were about the same thing. What stops that one is the entitlement check itself
returning False the second time, which means your billing system - not your agent - is
the last line of defence. Build accordingly.
The commit boundary matters here too: mint_grant has to be committed before consume()
runs in its own transaction, or the consume finds no row and refuses a legitimate refund.
The consume step is a compare-and-swap, not a read followed by a write. It succeeds at
most once per grant_id, for the lifetime of the row, no matter how many times it is
called:
def consume(conn, grant_id: str, thread_id: str, ticket_id: str, cents: int) -> bool: """Mark a grant spent. True at most once per grant_id, per committed transaction.""" with conn.cursor() as cur: cur.execute( """ UPDATE grant_ledger SET consumed_at = now() WHERE grant_id = %s AND thread_id = %s AND ticket_id = %s AND cents = %s AND consumed_at IS NULL """, (grant_id, thread_id, ticket_id, cents), ) return cur.rowcount == 1The ticket and amount are in the WHERE clause deliberately. This keeps Part 3's
argument-binding property: the grant authorises one refund, on one ticket, for one
amount. A model that supplies different arguments does not match the row and does not
consume it. thread_id is there for the same reason - a grant identifier that leaks into
another thread's state should not be spendable there.
One thing this function does not do is commit. It relies on the caller's
with pool.connection() block committing on clean exit, and that commit has to land
before the effect fires. Lift it onto a raw connection without autocommit and it returns
True and then rolls back, which is the worst possible outcome: the guard believes the
grant is spent and the database disagrees.
AtlasState gains exactly one key for this: refund_grant_id, NotRequired[str],
written by check_entitlement. refund_checked_for from Part 3 goes away, and its job
moves into the WHERE clause.
The guard then replaces a state comparison with a ledger consumption:
from langgraph.config import get_configclass LedgerRefundGate(AgentMiddleware): def __init__(self, pool): self.pool = pool # a psycopg_pool.ConnectionPool, not the checkpointer def wrap_tool_call(self, request, handler): if request.tool_call["name"] != "issue_refund": return handler(request) args, state = request.tool_call["args"], request.state grant_id = state.get("refund_grant_id") thread_id = get_config()["configurable"]["thread_id"] if not grant_id: return ToolMessage( "refund not permitted: no entitlement grant on this thread", tool_call_id=request.tool_call["id"], ) # Coerce before the database sees it. A model-supplied "4999" against an # INTEGER column raises inside the guard, which turns a refusable call into # an aborted run. try: ticket_id, cents = str(args["ticket_id"]), int(args["cents"]) except (KeyError, TypeError, ValueError): return ToolMessage( "refund not permitted: malformed ticket_id or cents", tool_call_id=request.tool_call["id"], ) with self.pool.connection() as conn: spent = consume(conn, grant_id, thread_id, ticket_id, cents) if not spent: return ToolMessage( "refund not permitted: this grant is already spent, or does not " "authorise this ticket and amount", tool_call_id=request.tool_call["id"], ) return handler(request)The rename is deliberate. This is not Part 3's RefundGate with a new body - it replaces
only the wrap_tool_call half of that pair, and Part 3's wrap_model_call half stays
exactly as it was. Emission-side narrowing still does the work it always did, which is
keeping the tool off the model's menu when it has no business being there. It was never
the guard, and it is not the guard now.
Part 3's ordering rule still applies and is easy to lose here: wrap_* handlers nest
first-outermost, so a wrap_tool_call refusal wants to be first in the middleware
list. Put LedgerRefundGate last and any outer wrap_tool_call that returns without
calling handler() bypasses the consume entirely, which is Part 3's round-four correction
being un-learned.
The argument coercion is not decoration either. Part 3's guard compared against state, so
a malformed argument simply failed to match and the call was refused. This guard hands the
argument to a database with a typed column, so a model that emits "4999" as a string
raises inside the guard rather than returning a refusal. That aborts the run before
consume() and before the effect, so it fails closed and no money moves - but it converts
a refusable call into an outage, and it will be diagnosed as an agent bug rather than a bad
argument. Coerce, and refuse.
Both guards, run against the same restored state, with the second and third calls standing in for a crash-replay of the first:
PART 3 GUARD (state only) - crash between effect and checkpoint: first dispatch -> REFUNDED 4999 on 8812 replayed identically after crash -> REFUNDED 4999 on 8812 replayed identically after crash -> REFUNDED 4999 on 8812 refunds actually issued: 3PART 4 GUARD (spend ledger) - identical replay: first dispatch -> REFUNDED 4999 on 8812 replayed identically after crash -> REFUSED replayed identically after crash -> REFUSED refunds actually issued: 1PART 4 GUARD - a DIFFERENT call riding the same grant (grant still unconsumed): different ticket, same grant -> REFUSED refunds actually issued: 0One thing that transcript is quietly hiding, and it is the cost of the fix rather than a
flaw in it. REFUSED is a ToolMessage, it goes into the message history, it gets
checkpointed, and the model reads it. On a genuine crash-replay the model is now told the
refund failed - for a refund that succeeded - and it will say so to the customer. You have
traded a duplicate refund for a wrong answer, which is the better trade and is still a
trade.
That is what the effect_result column is for. On a consume miss, look the row up. If it
is consumed for this same tool_call_id and carries a recorded result, return that
result instead of a refusal - the replay then gets the truth. A call that does not match
the row still gets refused, because that one is a fork rather than a replay, which is
exactly the distinction ACRFence's replay-or-fork semantics draws.
Two conditions on that branch, and I had both wrong in a draft of this section:
- Match on the call, not on the arguments. "Consumed for this ticket and amount" would hand the recorded result to a genuinely new refund request that happened to look identical, telling the model a fresh request had already succeeded.
- A recorded failure is not a recorded result. If the effect ran and the provider
declined, the row is consumed with a failed outcome. Returning that as a replay answer
is correct; leaving the row terminal is not, because the thread can then never retry.
Clear
consumed_atwhen you record a failed effect, so a legitimate retry can re-consume it. Otherwise a single decline strands the customer permanently, and the reconciliation sweep will not notice because the row does have an effect recorded against it.
That second one is the sharper trap, because it is not produced by a crash. An ordinary provider decline is enough.
The same restructuring gives you a real cross-invocation budget. Count against thread_id
in a row the checkpointer cannot roll back:
class ThreadBudgetExceeded(RuntimeError): """Raised when a thread exhausts its cross-invocation model-call budget."""class ThreadModelCallBudget(AgentMiddleware): """A bound that survives resume, because its counter is not graph state.""" def __init__(self, pool, cap: int): self.pool, self.cap = pool, cap def before_model(self, state, runtime): # get_config(), not runtime.config - Runtime exposes context/store/ # stream_writer/previous, and has no .config attribute. thread_id = get_config()["configurable"]["thread_id"] with self.pool.connection() as conn, conn.cursor() as cur: cur.execute("SELECT steps FROM thread_steps WHERE thread_id = %s", (thread_id,)) row = cur.fetchone() if row and row[0] >= self.cap: # Check before incrementing, so a tripped thread stops climbing. raise ThreadBudgetExceeded( f"thread {thread_id} used {row[0]} model calls, cap {self.cap}" ) cur.execute( """ INSERT INTO thread_steps (thread_id, steps) VALUES (%s, 1) ON CONFLICT (thread_id) DO UPDATE SET steps = thread_steps.steps + 1 """, (thread_id,), ) return NoneThree caveats, because this one is not a drop-in replacement for recursion_limit.
before_model fires per model call, not per superstep, so this is a model-call
budget. For an agent that is the number you actually care about, since model calls are
what cost money, but it is not the same quantity.
before_model also only exists for create_agent. The self-looping StateGraph at the
top of this article has no middleware at all, so there the equivalent is the same
check-and-increment at the head of the node body, or a wrapper node in front of it.
And check before incrementing. Increment-then-check leaves a tripped thread climbing forever, so a thread that trips the cap once is permanently dead even after you fix whatever caused it. You will also want a reset path, because "permanently dead" is rarely what you meant.
The payoff is the scenario this article opened with. A supervisor that re-invokes on
failure hits before_model on the first model call of the new invocation, reads a counter
that the crash did not touch, and raises immediately. The loop that burned 38 supersteps
against a limit of 8 stops at the cap instead, on the first retry that exceeds it.
recursion_limit stays where it is. It is a fine per-invocation guard against a graph
that misroutes inside one run. It was never a per-thread cost control, and the fix is not
to raise it.
The atomicity you cannot have
There is a hole in the pattern above and it does not close.
consume() and billing.refund() are two different systems. There is no transaction
that spans both. Vanlightly puts it plainly: "the durable function is not an atomic
transaction." So the crash can land between them, and you have to choose which way that
failure goes.
Consume first, then act. A crash between the two loses the refund. The grant is marked spent, the money never moved, and a customer is owed something the system now believes it has settled. The error is detectable, because you hold a consumed grant with no recorded effect, and it is reversible, because you can re-drive it.
Act first, then consume. A crash between the two duplicates the refund. That is the company's money out the door twice and a reconciliation entry nobody asked for. It is also harder to detect, because nothing in your system recorded the first one.
Choose the first one, and choose it by the general rule rather than by the example: prefer the direction whose error is detectable and reversible. For a refund that means consume-first. For an outbound charge it flips - a lost charge is revenue you can re-drive, a duplicate charge is a customer's money and a chargeback. The rule is stable; which side it lands on depends on who is out of pocket when it goes wrong.
That only works if the sweep can actually run, and for that the ledger has to record
intent and outcome, not just consumption. Add an effect_id and an effect_result to the
row: write the intent when you consume, write the result when the effect returns. Now a
consumed grant with no result is a real, findable work item, and the sweep re-drives it.
Without those columns "consumed, never acted" and "consumed, acted, record lost" are the
same row and the sweep has nothing to reconcile against.
Re-driving is also where provider idempotency keys finally earn their place, keyed on the
grant_id. Under consume-first a replay never reaches the provider at all, so the key
does nothing on the normal path - it matters when the sweep re-drives and needs the
provider to collapse a possible duplicate. It still does not help with a different call
riding a live grant, which is why the ledger is the load-bearing part.
And note the expiry, because it interacts badly with this part's own premise of workflows that span days. Stripe's documentation states that idempotency keys are pruned after at least 24 hours, and "we generate a new request if a key is reused after the original is pruned." A thread that parks on a human approval on Friday and resumes on Monday has an idempotency key that no longer means anything. Check the ledger row, and treat the provider key as a short-lived optimisation layered on top of it.
The tuple that passes every test and fails after a resume
While verifying the guard I broke it. The fix changes a recommendation Part 3 made, and the failure mode is bad enough to need its own section.
Part 3 stores the authorised pair as a tuple, ("8812", 4999), and compares it against
the call's arguments. It also says this, which I need to retract:
LangGraph's default serde preserves tuples; a JSON-backed one does not.
The first half is false. The default serializer does not preserve tuples. Run it:
in=('8812', 4999) type=tuple -> out=['8812', 4999] type=list equal=Falsein=['8812', 4999] type=list -> out=['8812', 4999] type=list equal=Truein={'b', 'a'} type=set -> out={'b', 'a'} type=set equal=Truein=('nested', ('x', 1)) type=tuple -> out=['nested', ['x', 1]] type=list equal=FalseSets survive, because JsonPlusSerializer has an extension handler for them. Tuples do
not have one. The default path is msgpack, msgpack has no tuple type, and a Python tuple
packs as an array and unpacks as a list.
Now the part that makes this genuinely dangerous:
inside next node: ('8812', 4999) type=tuple == ('8812', 4999) ? Trueafter get_state: ['8812', 4999] type=list == ('8812', 4999) ? FalseWithin a single run, the next node receives a tuple, because the value is handed along in memory and never goes through the serializer. It only comes back as a list after a real checkpoint round-trip.
So a guard that compares tuples passes every test you write in one process. It passes the
node-to-node test. It passes the eight-case table Part 3 shipped. And then it fails closed
for any comparison that crosses an invocation boundary: the grant written on Tuesday never
matches the call replayed on Wednesday, because one side is a tuple and the other came
back as a list. Within a single invocation it keeps working, which is what makes the bug
so hard to corner - the next time check_entitlement runs it writes a fresh tuple and the
guard matches again. Part 3 correctly predicted that consequence and then told you it
would not happen to you on the default serializer.
Store the pair as a string and compare strings:
update["refund_checked_for"] = f"{ticket_id}:{cents}" # for anyone staying on Part 3's shapeChange all three sites together - the writer, the predicate, and the guard - because an
encoding that matches in two places out of three fails closed. Better, adopt the ledger
above and the comparison moves into a WHERE clause where the database does the
coercion for you.
There is a general rule here that outlives this bug. Anything you compare across a resume boundary has to survive serialisation as itself. In-process equality proves nothing about post-restore equality, and the in-process test is the one you will naturally write.
Where the durability boundary actually is
Everything above depends on knowing what is durable and when. The short version: a node body is not a durable unit.
Checkpoints are written at superstep boundaries. The persistence docs describe a checkpoint as "a snapshot of graph state at a specific moment, saved at each super-step boundary." Nothing inside a node body is checkpointed. But the superstep boundary is not the finest-grained durable record, and this is where I have to be careful, because it is easy to state the window one boundary too wide.
As each task in a superstep completes, its writes land in checkpoint_writes. On resume,
the loop loads those pending writes and matches them by task id, and a task that
already has writes is not re-executed - its writes are simply applied. The docs put it
plainly: LangGraph stores pending writes "from any other nodes that completed
successfully at that super-step" so that on resume "we don't re-run the successful
nodes." That skips the task itself, not just its siblings.
So the exposure is narrower than "the node body." It runs from node entry to the moment that task's write actually reaches the checkpointer. If the process dies after the refund task's write has landed, the guard is not re-entered and the refund is not re-issued. Pending writes already close that half.
What they do not close is the part that matters:
- The gap itself is real and racy. Under the default
durability="async"the write is handed to a background submission rather than a synchronous durable point, so "the write landed" is not something the node body gets to observe. interrupt()ignores all of this. The node restarts from the top on resume regardless of pending writes, which is why the approval case is the worst one.- A task is not a tool call.
ToolNodeexecutes all of a turn's tool calls inside one task, and Part 3 showed those running in the model's emitted order within that single task. If tool B fails, the whole task's writes are discarded and tool A runs again on resume. A per-task durable record is not a per-effect durable record. recursion_limitdoes not involve pending writes at all. Its allowance is recomputed at loop entry, before any of this applies.
How wide the window gets is a configuration choice most teams never make. The durability
parameter takes three values, and the default is the middle one:
| Mode | What it does | Crash window |
|---|---|---|
"exit" | Persists only when execution exits - success, error, or interrupt | Widest. An entire run's progress can be lost |
"async" | Default. Persists asynchronously while the next step executes | The docs concede "a small risk that LangGraph does not write checkpoints if the process crashes during execution" |
"sync" | Persists synchronously before the next step starts | Narrowest |
durability arrived in v0.6.0 and replaced checkpoint_during, which is deprecated with
backwards compatibility promised until v2.0.0. durability="async" corresponds to the
old checkpoint_during=True, and "exit" to False.
The default matters for this article. Under "async", the checkpoint for superstep N may
still be in flight when the process dies during superstep N+1. That is a wider replay
window than most people picture when they say a run is checkpointed, and it is what you
get unless you pass the argument.
There is no published measurement of what "sync" costs relative to "async". The docs
say only that "a higher durability mode adds more overhead." I am not going to invent a
percentage. Measure it on your own graph if the answer matters.
flowchart LR
subgraph NODE["One node body - nothing here is durable"]
direction TB
N1["read state"] --> N2["billing.refund<br/>irreversible effect"]
N2 --> N3["compute the update"]
end
NODE --> W["per-task write<br/>lands in checkpoint_writes"]
W --> CP["superstep boundary<br/>checkpoint written"]
CP --> NEXT["next superstep"]
style N1 fill:#98D8C8,color:#2C2C2A
style N2 fill:#E74C3C,color:#FFFFFF
style N3 fill:#98D8C8,color:#2C2C2A
style W fill:#FFD93D,color:#2C2C2A
style CP fill:#6BCF7F,color:#2C2C2A
style NEXT fill:#4A90E2,color:#FFFFFF
The red box is the whole exposure. Everything from node entry to the per-task write is a region in which an irreversible effect can happen with no durable record that it did.
The runtime gives you three ways to shrink it, and none of them closes it. Move the effect
into its own node, so the region contains nothing else. Set durability="sync", so the
write is a synchronous durable point rather than a background submission. Or wrap the
effect in a @task from langgraph.func, which the durable-execution docs recommend
directly - "wrap any non-deterministic operations ... or operations with side effects
(e.g., file writes, API calls) inside tasks to ensure that when a workflow is resumed,
these operations are not repeated." A @task memoises its result as a pending write, which
is a genuine sub-node durable boundary.
All three narrow the same gap: the distance between the effect returning and its record becoming durable. None of them make that distance zero, because the effect happens in another system and the record happens in yours. That is why the ledger below is still load-bearing after you have done all three.
Choosing a checkpointer: SQLite for dev, Postgres for production
The checkpointer is what defines the rollback surface, so pick it deliberately.
| Backend | Package | Use |
|---|---|---|
InMemorySaver | langgraph-checkpoint | Tests only. The docs are explicit: "When the process restarts, all checkpoints are lost" |
SqliteSaver / AsyncSqliteSaver | langgraph-checkpoint-sqlite | Local development, single process |
PostgresSaver / AsyncPostgresSaver | langgraph-checkpoint-postgres | Production |
setup() must run once before first use. It checks a checkpoint_migrations table and
applies any missing migration scripts in order. It is not optional and it is not
automatic.
The Postgres connection needs two non-default settings, and both fail in confusing ways
if you miss them. Connections must be created with autocommit=True, or the checkpoint
table creation does not persist. They also need row_factory=dict_row, or reads raise a
TypeError when the checkpointer addresses columns by name.
PostgresSaver.from_conn_string is a context manager rather than a constructor - a point
Part 1 already made and worth repeating, because
holding the returned object past the with block is a popular way to get a closed
connection in production.
Also note that ShallowPostgresSaver and its async twin are deprecated. The migration
path is PostgresSaver with durability="exit", which tells you something about what
"shallow" meant: fewer writes, wider crash window.
Three tables carry the state: checkpoints, checkpoint_blobs, and checkpoint_writes.
The last one is the per-task write log that makes partial recovery possible, and it is
the closest thing LangGraph has to a durable record inside a superstep.
It is not only crashes
Nodes re-run for several reasons, and only one of them is a crash:
- Resume after
interrupt(). The node restarts from the top. Documented, normal, and the most common path in any human-in-the-loop system. - Retry policy on node error. The node runs again by design.
- Superstep scheduling. A node reachable by more than one path can execute twice in a single run with no failure at all. Issues #4026 and #6005 are both this, and both are closed.
- Partial resume of a multi-interrupt node. A LangGraph maintainer, Caspar Broekhuizen, filed #6208 and states it directly: "a node with two interrupts will rerun after only one resume." A fix was merged on 6 October 2025 and reverted two days later. The issue is still open.
- Platform-level stale-run sweeps. #7417, open since April 2026, reports that on LangGraph Cloud "when a tool call takes longer than ~3 minutes, it gets silently re-dispatched from the last checkpoint while the original is still running." The reporter attributes it to a heartbeat and sweep interval shorter than the tool duration. That analysis is the reporter's, not a maintainer's, but the duplicate work and cost are observed.
I should correct one of my own references here as well. Part 2 deferred "duplicated side effects on fan-in (issues #4026, #6005)" to this article, framed as a replay problem. It is not. Both are scheduling duplication - a node reachable twice inside one superstep graph, with no crash and no resume involved. They are still worth reading, because they prove nodes run more than once for reasons unrelated to recovery, but they are not evidence for this article's thesis and I should not have filed them as if they were.
How a checkpoint replays an irreversible side effect
flowchart TD
A["check_entitlement runs<br/>grant written into graph state"] --> B["superstep boundary<br/>checkpoint lands, grant is durable"]
B --> C["guard reads state<br/>grant is unspent, so: allow"]
C --> D["billing.refund<br/>money leaves the account"]
D --> E["crash before this task's write<br/>reaches checkpoint_writes"]
E --> F["recovery restores<br/>the last durable checkpoint"]
F --> C
D -.->|"not restored, not undone"| G["the money is still gone"]
style A fill:#4A90E2,color:#FFFFFF
style B fill:#4A90E2,color:#FFFFFF
style C fill:#FFD93D,color:#2C2C2A
style D fill:#E74C3C,color:#FFFFFF
style E fill:#95A5A6,color:#2C2C2A
style F fill:#7B68EE,color:#FFFFFF
style G fill:#C2185B,color:#FFFFFF
The cycle C -> D -> E -> F -> C is the failure, and every arrow in it is correct
behaviour. The dotted edge is the asymmetry that makes it expensive: state is restored,
the world is not.
Multi-day threads and the deploy problem
This part of the series promises "workflows that span days and survive deploys." Both halves need qualifying.
The threads themselves are fine. A thread_id is a durable cursor, and a run that parks
on an interrupt in one process resumes in another days later. That works.
What does not survive is a change to the shape of the state underneath a parked thread. LangChain's own guidance on schema changes across deployments is blunt about it:
Completed threads can survive topology changes (node renames, additions, removals). Interrupted threads cannot.
The reason is mechanical. An interrupted thread is waiting to enter a specific node. Rename or remove that node and it has nowhere to resume to. Add a required field to the state schema and the checkpoint has no value for it. Remove a field and the checkpoint carries one the schema no longer defines.
The prescribed mitigations are: always give new fields defaults and read them with
state.get("field", default); set a checkpoint time-to-live so old threads expire rather
than resurrect into an incompatible schema; and for breaking changes, "drain existing
threads before deploying or accept that in-progress threads will need to be restarted."
Sit that last sentence next to "workflows that span days and survive deploys" and the tension is obvious. Draining a queue of threads parked on human approvals means waiting for humans. There is no built-in migration path: the request for state schema versioning has been open on the JavaScript repository since September 2024, with no maintainer response, and the Python side has no equivalent mechanism either.
The practical read: threads that span days are a commitment to schema stability for the lifetime of the longest-lived thread. If approvals can sit for a week, your state schema is frozen for a week, or you accept that some threads die on deploy. Decide which, and write it down, because the default is to discover it during an incident.
The checkpoint is also an input
One more thing belongs here, because it changes how you should think about the checkpointer's trust boundary.
In June 2026 Check Point Research published a chain from SQL injection to remote code
execution in the LangGraph checkpointer. The list() method built SQL with unescaped
filter keys, so an attacker-controlled filter could break out of a JSON path string and
inject a UNION SELECT returning attacker-chosen msgpack. Deserialising that payload
reaches getattr(importlib.import_module(...), ...)(...), and from there to os.system.
Three CVEs came out of it: CVE-2025-67644 (langgraph-checkpoint-sqlite, fixed in 3.0.1),
CVE-2026-28277 (langgraph, fixed in 1.0.10), and CVE-2026-27022
(langgraph-checkpoint-redis, fixed in 1.0.2). An earlier one, CVE-2025-64439, covered
the same deserialisation risk in the serializer's JSON fallback mode.
The mitigation now documented on the checkpointer packages is to set
LANGGRAPH_STRICT_MSGPACK=true or pass allowed_msgpack_modules, restricting
deserialisation to known-safe types.
The general point outlasts the specific CVEs. A checkpoint is not only a recovery artefact. It is data your process reads back and reconstructs objects from, which makes your checkpoint store a deserialisation trust boundary and part of your attack surface. Treat write access to it the way you treat deploy credentials.
The audit: where does each bound's spend record live?
The audit is one question, asked once per bound.
For every limit, grant, quota, and approval in your system, name the thing that records it as spent, and say whether a checkpointer restores that thing.
| Bound | Spend record | Inside the rollback surface? | Verdict |
|---|---|---|---|
recursion_limit | Step counter in checkpoint metadata | Yes | Self-restoring. Per-invocation only |
| Entitlement grant in state | The state key itself | Yes | Self-restoring |
Approval via interrupt() | Resume value in pending writes | Yes | Self-restoring |
| Grant id in state + ledger row | Ledger row in your database | No | Safe if the grant id is derived deterministically - a re-minted random id forks a fresh unconsumed row |
| Per-thread step counter in your database | Ledger row | No | Safe |
| Provider idempotency key | Provider's key store | No, but expires | Safe under 24h, then not |
| Absolute deadline (timestamp stored at episode start) | The clock, which no checkpointer owns | No | Safe against self-restoration - though it is the canonical unsafe thing under divergent replay, which is the class Vanlightly's determinism requirement exists for |
Relative per-invocation timeout (timeout=600, asyncio.wait_for around a run) | Nothing | n/a | Self-restoring, species two. Same shape as recursion_limit: four resumes against a 10-minute timeout gives you 40 minutes |
ToolCallLimitMiddleware(thread_limit=...) | Middleware counter in the graph's state schema | Yes | Survives a clean resume; rolls back with the checkpoint on a crash. Better than recursion_limit, still approximate - see the Halfway section |
Then the checklist:
- Do not use
recursion_limitas a cost control. It bounds one invocation. If a supervisor retries, multiply by the retry count and keep multiplying. - Put every consumption record outside the checkpointer. The grant identifier can live in state. The fact that it was used cannot.
- Make consumption a compare-and-swap, not a read then a write.
UPDATE ... WHERE consumed_at IS NULLand check the affected row count. Make sure it commits before the effect fires. - Derive grant identifiers deterministically, and put a uniqueness constraint behind them. A re-run that mints a fresh random id forks a new unconsumed row and the whole ledger buys you nothing.
- Bind the grant to its arguments in the same statement, so a different call cannot ride a live grant. Coerce those arguments before the database sees them, or a type error inside the guard aborts the run instead of refusing the call.
- Consume before you act when a lost effect is the detectable, re-drivable error - and check which side that is for your effect, because it flips between refunds and charges. Record intent and outcome, or the reconciliation sweep has nothing to match.
- Give human approvals a ledger row too. Structural hygiene stops pre-interrupt side effects repeating; it does not make the approval one-shot.
- Shrink the effect-to-durable-record window with all three levers: the effect in its
own node,
durability="sync", and@taskaround the effect. None of them closes it. - Never place a side effect before an
interrupt()in the same node. It re-runs on every resume, and this is documented behaviour rather than an edge case. - Set
durabilityexplicitly. The default is"async", which is not the narrowest window. Choose it on purpose. - Compare only values that survive serialisation. Tuples do not. Test equality after a real checkpoint round-trip, never node-to-node in one process.
- Use the id-map resume form for any node with more than one
interrupt(), so answers match by identity rather than by position. - Give new state fields defaults, set a checkpoint time-to-live, and know whether your longest-parked thread outlives your deploy cadence.
- Treat the checkpoint store as a trust boundary. Set
LANGGRAPH_STRICT_MSGPACKand keep the checkpointer packages patched. - Do not assume one run per thread. The ledger makes the effect single-shot. It
does not make the run single-shot, and open-source LangGraph gives you no
thread_idmutual exclusion - two supervisors resuming the same thread will both execute the whole graph.
The bound you can see is not the bound you have
Go back to the transcript at the top. Thirty-eight supersteps against a limit of eight, and every one of those four enforcement decisions was correct.
That is the uncomfortable part, and it is the reason this failure survives review. There is no broken component to find. The checkpointer restored the state it was asked to restore. The step counter recorded the steps that were taken. The guard evaluated its predicate against the state in front of it and returned the right answer. Each piece is individually defensible, and the system still spends a bound it already spent.
The durable-execution literature told us to make replay faithful. The security literature told us to worry about the agent asking for something different after a restore. Both are correct advice, and neither one touches this, because a perfectly faithful replay of a perfectly deterministic guard is sufficient on its own. The property that saves you is not determinism. It is that the record of having spent something lives somewhere the recovery mechanism cannot reach.
So the question to carry out of this article is not "is my agent deterministic" or "are my tools idempotent." It is narrower and easier to answer: for every bound in this system, what writes down that it was used, and does the next derivation have to respect it? If nothing does, the bound is a suggestion with good intentions, and it will arrive back at full the next time anything re-enters it - a crash, a retry, or an ordinary second turn.
What Part 5 covers
Part 5 moves to context and memory: treating the context window as the scarce resource it is, separating in-thread state from cross-thread user memory, and building memory that extracts and compacts rather than accumulating.
There is a thread from this article into that one. Everything here treats checkpointed state as something restored faithfully and completely. Memory systems deliberately do the opposite - they summarise, drop, and rewrite. A bound derived from state that gets compacted rather than restored is a different problem from one that gets restored exactly, and it gets its own treatment.
Part 7 gets the observability question this article keeps raising and deferring: none of these failures are visible without tracing that spans resume boundaries, and a run that resumes four times looks like four runs unless you have built otherwise.
References
- Zheng, Y., Yang, Y., Zhang, W., & Quinn, A. (2026). ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore. arXiv:2603.20625. https://arxiv.org/abs/2603.20625
- Santos-Grueiro, I. (2026). Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agents. arXiv:2606.22504. https://arxiv.org/abs/2606.22504
- Miller, M. S., Yee, K.-P., & Shapiro, J. (2003). Paradigm Regained: Abstraction Mechanisms for Access Control. http://erights.org/talks/asian03/paradigm-revised.pdf
- Redell, D. D. (1974). Naming and Protection in Extendible Operating Systems. PhD thesis, University of California, Berkeley. Cited here via Miller, Yee & Shapiro (2003); I have not read the thesis directly.
- Saltzer, J. H., & Schroeder, M. D. (1975). The Protection of Information in Computer Systems. Proceedings of the IEEE, 63(9). https://cgi.cse.unsw.edu.au/~cs9242/19/papers/Saltzer_Schroeder_75.pdf
- Vanlightly, J. (2025, November 24). Demystifying Determinism in Durable Execution. https://jack-vanlightly.com/blog/2025/11/24/demystifying-determinism-in-durable-execution
- LangGraph. Persistence. LangChain Docs. https://docs.langchain.com/oss/python/langgraph/persistence
- LangGraph. Durable execution. LangChain Docs. https://docs.langchain.com/oss/python/langgraph/durable-execution
- LangGraph. Interrupts. LangChain Docs. https://docs.langchain.com/oss/python/langgraph/interrupts
- LangGraph. Durability type reference. https://reference.langchain.com/python/langgraph/types/Durability
- langchain-ai/langgraph.
libs/langgraph/langgraph/pregel/_loop.py. https://github.com/langchain-ai/langgraph/blob/main/libs/langgraph/langgraph/pregel/_loop.py - LangChain Support. Managing State Schema Changes Across LangSmith Deployment Versions. https://support.langchain.com/articles/1785884356-managing-state-schema-changes-across-langsmith-deployment-versions
- Broekhuizen, C. (2025, September 26). Do not re-execute a node that interrupted unless all of its interrupts have been resumed. langchain-ai/langgraph #6208. https://github.com/langchain-ai/langgraph/issues/6208
- Napoli, M. A. (2026, April 5). Long tool calls (~180s+) silently re-executed from checkpoint on LangGraph Cloud. langchain-ai/langgraph #7417. https://github.com/langchain-ai/langgraph/issues/7417
- Burns, B. (2024, September 30). Feature Request: Support for State Schema Versioning & Migration. langchain-ai/langgraphjs #536. https://github.com/langchain-ai/langgraphjs/issues/536
- langchain-ai/langgraph #4026 (2025, March 26, closed). In the fan-out and fan-in with extra steps, there are execution steps that do not meet expectations. https://github.com/langchain-ai/langgraph/issues/4026
- langchain-ai/langgraph #6005 (2025, August 25, closed). LangGraph's defer=True logic doesn't solve much and is fundamentally broken. https://github.com/langchain-ai/langgraph/issues/6005
- Porat, Y. (2026, June 11). From SQLi to RCE - Exploiting LangGraph's Checkpointer. Check Point Research. https://research.checkpoint.com/2026/from-sqli-to-rce-exploiting-langgraphs-checkpointer/
- Stripe. Idempotent requests. https://docs.stripe.com/api/idempotent_requests
- US Patent 4024488 (1977). Self-restoring type current limiting device. https://patents.justia.com/patent/4024488
Related Articles
- State Architecture for Agent Networks: The Resume Is the Dangerous Part
- LangGraph or a While Loop? You Already Have a Runtime
- LangGraph Compaction Deletes the View, Not the Record
- Human-in-the-Loop at Production Scale: The Checkpoint Membrane


