The Transcript Is the Bill

The transcript is the bill. An agent job is not one request, and pricing it as one request is how a free lane disappears without a crash. Every tool turn reprints the prior messages. The model does not remember. You pay to remind it.

You already treat retries as a cost center. This is adjacent and worse. A retry repeats a call. A loop reprints a growing document. The second turn is not twice the first. It is the first plus the tool dump plus the next instruction, stuffed back through the same tokenizer.

That reprint is the unit you should budget. Not “jobs.” Not “agents.” Tokens that reappear because your orchestrator is polite enough to send the whole story again.

The multiplier hides in the polite client

Most agent wrappers look inexpensive in a dashboard because the dashboard counts HTTP calls. One call, one row. The row does not show that call three rehydrated calls one and two. Context is not a cache unless you built a cache. It is a payload you keep mailing to yourself.

Picture a meeting notes app that emails the entire thread every time someone types “ok.” You would not call that one email. You would call it a photocopier with a stamp. Tool-using loops do the same thing, only the paper is tokens and the stamp is your budget.

A tiny heuristic is enough to see it. Four characters per token is wrong in the edges and right enough to scare you.

# proposal: envelope tracer, not a production SDK
from dataclasses import dataclass, field


def approx_tokens(text: str) -> int:
    return max(1, len(text) // 4)


@dataclass
class Turn:
    role: str
    text: str
    tool: str | None = None

    @property
    def tokens(self) -> int:
        return approx_tokens(self.text)


@dataclass
class Envelope:
    cap: int
    spent: int = 0
    rejected: int = 0
    log: list[dict] = field(default_factory=list)

    def charge_prompt(self, transcript: list[Turn], new_output: str) -> bool:
        prompt = sum(t.tokens for t in transcript)
        completion = approx_tokens(new_output)
        cost = prompt + completion
        row = {
            "turn": len(self.log) + 1,
            "prompt_tokens": prompt,
            "completion_tokens": completion,
            "cumulative": self.spent + cost,
            "accepted": self.spent + cost <= self.cap,
        }
        if not row["accepted"]:
            self.rejected += cost
            self.log.append(row)
            return False
        self.spent += cost
        self.log.append(row)
        return True

Run a fake loop against it. Do not wait for a vendor invoice. The shape shows up in thirty lines.

SYSTEM = Turn("system", "You are a repo nurse. Call tools. Do not guess file contents.")
user = Turn("user", "Find why /health returns 500, then patch it.")
transcript = [SYSTEM, user]
env = Envelope(cap=8_000)

tool_blobs = [
    "search: 12 files, 9 false positives, 3 stack traces pasted in full",
    "read: 1_800 lines of middleware, comments included",
    "test: 40 lines of failure, 200 lines of captured logs",
]

for i, blob in enumerate(tool_blobs, start=1):
    thought = f"Turn {i}: I will inspect more context because the last tool was noisy."
    if not env.charge_prompt(transcript, thought):
        print("abort before reprint", env.log[-1])
        break
    transcript.append(Turn("assistant", thought, tool=f"tool_{i}"))
    transcript.append(Turn("tool", blob, tool=f"tool_{i}"))
else:
    print("loop finished inside cap", env.spent)

for row in env.log:
    print(row)

You will watch prompt tokens climb while completion tokens stay small. That is the tell. The model is not “thinking more.” You are mailing a thicker envelope. The third tool result did not cost third-tool money. It cost the whole novel again, plus a paragraph.

Charge the reprint, then decide the lane

Once you have a cumulative column, jobs become comparable. A one-shot completion with a 600-token prompt is a postcard. A four-turn repair agent with the same user sentence is a binder. Free capacity can hold postcards all afternoon. Binders contend. Binders also fail open if you only cap steps.

Step caps lie. Four tools can be cheap if each returns a hash. Four tools can be fatal if each returns a file. Token caps tell the truth because they meter the photocopy, not the choreography.

Put the abort next to the call site, not in a weekly report. A report is a eulogy. An abort is a budget.

def run_turn(env: Envelope, transcript: list[Turn], call_model, tools):
    # labeled example: wire this in front of your real client
    if env.spent >= env.cap:
        raise RuntimeError("envelope already spent")
    prompt_preview = sum(t.tokens for t in transcript)
    if prompt_preview >= env.cap:
        raise RuntimeError("reprint alone exceeds cap")
    output = call_model(transcript)  # your client here
    if not env.charge_prompt(transcript, output.text):
        raise RuntimeError("completion would cross cap")
    transcript.append(Turn("assistant", output.text, tool=output.tool))
    if output.tool:
        blob = tools[output.tool](output.args)
        transcript.append(Turn("tool", blob, tool=output.tool))
    return output

Keep the rejected counter. Discarded completions are still work the machine did. If you stream and the user hits stop, charge what left the server, not what landed in the UI. Dashboards that ignore aborts teach you the happy path. Cost lives in the stop button.

A one-line check on a laptop is enough to start:

python envelope_tracer.py
# expect prompt_tokens to rise each turn faster than completion_tokens

If you want a pytest around the rule, assert the shape, not a vendor number.

def test_reprint_grows_faster_than_completion():
    env = Envelope(cap=50_000)
    transcript = [Turn("system", "x" * 400), Turn("user", "fix it")]
    assert env.charge_prompt(transcript, "ok")
    transcript += [Turn("assistant", "ok"), Turn("tool", "y" * 800)]
    assert env.charge_prompt(transcript, "still looking")
    first, second = env.log[0], env.log[1]
    assert second["prompt_tokens"] > first["prompt_tokens"]
    assert second["prompt_tokens"] > second["completion_tokens"]

That test is the whole ops note. If prompt tokens do not outrun completion tokens, you do not have a loop problem. You have a single-call problem, and your old per-job budget still works.

When free capacity is the wrong bet

Free model access is a rehearsal room. A free server is a quiet box for the tracer. Neither one changes the photocopy math. If the reprint curve is steep, moving the same loop onto a spare machine only moves the queue. The binder is still a binder.

You can run the envelope against MonkeyCode’s free model access and free server option when you need a real tokenizer in the loop instead of len(text) // 4. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that lane to learn the turn where the reprint would have crossed the cap. Do not use it as a promise that an unbounded agent is suddenly cheap.

Free capacity is the wrong bet when the loop cannot abort. A job that must finish because a customer is staring at a spinner will sit behind whoever else is photocopying a repo. It is the wrong bet when tool fan-out is unbounded, when a search tool can paste nine files because the query was vague, and when the transcript carries secrets you should not park on a shared box. It is the wrong bet when you price success as “the agent returned,” because return values hide the cumulative prompt.

It is a reasonable bet when the task is throwaway measurement. A linter for your own tracer. A fixture that replays last week’s tool blobs. A comparison between “summarize the tool output” and “paste the file.” Those runs teach you whether to shrink the photocopies before you ever care which lane carries production traffic.

Shrink the photocopies on purpose. Summarize tool results before they re-enter the transcript. Replace file dumps with hashes plus the three failing lines. Cap search hits. Drop the system prompt’s novel if the tools already constrain behavior. Each of those cuts the reprint, which cuts every future turn, which is the only compounding discount this architecture has.

Do not confuse that with caching the same user prompt. Cache helps when the document is stable. Agent transcripts are not stable. They accrete. A cache key that includes the whole transcript is unique every turn, which is a polite way to say you still pay.

Who should not use this

Skip the envelope if you do not have a loop. A single completion with a known prompt size can use a boring max-tokens field and a boring bill. Skip it if you cannot abort; a cap you will override under pressure is decoration. Skip it if your tools return structured IDs and never prose. There is no photocopier in that design. You are already doing the right thing.

Also skip a shared free server if the transcript can contain customer code, credentials, or anything you would not paste into a ticket. Rehearsal is not a data-handling policy. Keep those jobs on iron you control, with the same abort, because the math does not care who owns the GPU.

The operational habit is small. Log prompt versus completion per turn. Cap the sum, not the step count. Abort in the client. Treat free capacity as a place to watch the curve, not a place to hide it. When the reprint is the bill, you stop arguing about models and start editing what you mail back to yourself.

If you try the tracer on a spare box, keep the envelope on. The number worth writing down is not tokens remaining. It is the turn index where the next photocopy would have failed the cap.