Read insight attributes as aliases, not as one emitter's vocabulary #1

2026-09-19T20:22:28-07:00

zzg commented

2026-09-19 20:22:28 -07:00

Stacked on #2 (module-path-gitea). Merge that one first; this branch is rebased on top of it, so the diff below is this change alone.

Fathom ingests OTLP from anything, and ingest already knew that emitters disagree about names. classify() keys on gen_ai.operation.name as well as Claude Code's span.type; the correlator's DefaultJoinKeys() lists gen_ai.tool.call.id alongside tool_use_id.

The insights analyzer did not. It read cost_usd, input_tokens, output_tokens, cache_read_tokens, model, stop_reason, tool_name and user_prompt — Claude Code's private vocabulary, exclusively.

So a trace from any other emitter rendered correctly in the trace view and reported no cost, no tokens, and every tool named "tool". That is a silent wrong answer, and the kind that presents as "you have no findings" rather than as a bug.

Demonstrated, not theorized

A pi run emitted with plain GenAI conventions, ingested into a real store. Fathom classified and correlated it perfectly — the data is all there:

SELECT sum(input_tokens)              -> -        (what the analyzer looked for)
SELECT sum(gen_ai.usage.input_tokens) -> 12043    (what is actually on disk)

DebugExtract against that same store, before and after:

	before	after
prompt	`""`	`"refactor the auth module and run the tests"`
model	`""`	`claude-sonnet-5`
cost	`$0.00000`	`$0.04218`
tokens	`0` (cached `0`)	`12560` (cached `11000`)
tools	`tool tool`	`read bash`
stop reason	`""`	`tool_use`

How

internal/insights/attrs.go resolves each fact through an ordered list of names, Claude Code's own first in every list. A trace carrying both vocabularies resolves to the name the analyzer was originally written against, so this cannot change any existing answer.

Resolution is first-match-wins, not sum-across-matches — an emitter stamping both names on one span must not have its tokens counted twice. TestClaudeCodeAttributesStillWin pins both properties.

Cost has one extra wrinkle: Claude Code reports it on the log records, a GenAI emitter reports it on the span. costOf() checks the span first and short-circuits, so a trace carrying both does not pay twice.

This is a read-time alias, not an ingest-time rewrite. What an emitter sent stays on disk verbatim — the same reason classify() keeps the original span name whatever kind it decides on.

Tests

internal/insights/attrs_test.go, 6 cases, written first and watched fail against a pi/costslayer-shaped trace. go test ./... and go vet ./... both clean.

Context

The emitter that surfaced this is costslayer's new OTLP layer (zzg/costslayer#4), but nothing here is specific to it — the aliases are the OTel GenAI conventions, so any conforming emitter benefits.

🤖 Generated with Claude Code

> **Stacked on #2** (`module-path-gitea`). Merge that one first; this branch is rebased on top of it, so the diff below is this change alone. Fathom ingests OTLP from anything, and ingest already knew that emitters disagree about names. `classify()` keys on `gen_ai.operation.name` as well as Claude Code's `span.type`; the correlator's `DefaultJoinKeys()` lists `gen_ai.tool.call.id` alongside `tool_use_id`. The insights analyzer did not. It read `cost_usd`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `model`, `stop_reason`, `tool_name` and `user_prompt` — Claude Code's private vocabulary, exclusively. So a trace from any other emitter rendered **correctly in the trace view** and reported **no cost, no tokens, and every tool named "tool"**. That is a silent wrong answer, and the kind that presents as "you have no findings" rather than as a bug. ## Demonstrated, not theorized A pi run emitted with plain GenAI conventions, ingested into a real store. Fathom classified and correlated it perfectly — the data is all there: ``` SELECT sum(input_tokens) -> - (what the analyzer looked for) SELECT sum(gen_ai.usage.input_tokens) -> 12043 (what is actually on disk) ``` `DebugExtract` against that same store, before and after: | | before | after | |---|---|---| | prompt | `""` | `"refactor the auth module and run the tests"` | | model | `""` | `claude-sonnet-5` | | cost | `$0.00000` | `$0.04218` | | tokens | `0` (cached `0`) | `12560` (cached `11000`) | | tools | `tool tool` | `read bash` | | stop reason | `""` | `tool_use` | ## How `internal/insights/attrs.go` resolves each fact through an ordered list of names, **Claude Code's own first in every list**. A trace carrying both vocabularies resolves to the name the analyzer was originally written against, so this cannot change any existing answer. Resolution is **first-match-wins, not sum-across-matches** — an emitter stamping both names on one span must not have its tokens counted twice. `TestClaudeCodeAttributesStillWin` pins both properties. Cost has one extra wrinkle: Claude Code reports it on the **log records**, a GenAI emitter reports it on the **span**. `costOf()` checks the span first and short-circuits, so a trace carrying both does not pay twice. **This is a read-time alias, not an ingest-time rewrite.** What an emitter sent stays on disk verbatim — the same reason `classify()` keeps the original span name whatever kind it decides on. ## Tests `internal/insights/attrs_test.go`, 6 cases, written first and watched fail against a pi/costslayer-shaped trace. `go test ./...` and `go vet ./...` both clean. ## Context The emitter that surfaced this is costslayer's new OTLP layer (`zzg/costslayer#4`), but nothing here is specific to it — the aliases are the OTel GenAI conventions, so any conforming emitter benefits. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

zzg changed title from ~~Read insight attributes as aliases, not as one emitter's vocabulary~~ to Read insight attributes as aliases, and move the module path to git.z.gd

2026-09-19 20:28:11 -07:00

zzg referenced this pull request

2026-09-19 20:31:29 -07:00

Move the module path to git.z.gd/zzg/fathom #2

zzg added 2 commits 2026-09-19 20:31:59 -07:00

Move the module path to git.z.gd/zzg/fathom 79fe343be5

The module was declared as github.com/zzg/fathom, which is not where
this repo lives and never has been. Nothing is published at that path,
so `go get github.com/zzg/fathom` resolves to a 404 while the import
paths in every file assert otherwise.

git.z.gd is a real module path rather than a cosmetic correction:
Gitea serves the go-import meta tag for it,

  <meta name="go-import" content="git.z.gd/zzg/fathom git https://git.z.gd/zzg/fathom.git">

and the repo answers unauthenticated over HTTPS, so `go get
git.z.gd/zzg/fathom` resolves where the old path could not.

Mechanical: the module line plus 46 import paths across 23 files. There
is no go.sum to churn, since fathom has no third-party dependencies.
go build, go vet, gofmt and the full test suite are clean under the new
path, and the built binary still runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Read insight attributes as aliases, not as one emitter's vocabulary 4d0bedd85a

Fathom ingests OTLP from anything, and ingest already knew emitters
disagree about names: classify() keys on gen_ai.operation.name as well
as Claude Code's span.type, and the correlator's join keys list
gen_ai.tool.call.id alongside tool_use_id.

The insights analyzer did not. It read cost_usd, input_tokens,
output_tokens, cache_read_tokens, model, stop_reason, tool_name and
user_prompt -- Claude Code's private vocabulary, exclusively. A trace
from any other emitter therefore rendered correctly in the trace view
and reported no cost, no tokens, and every tool named "tool". That is a
silent wrong answer, and the kind that presents as "you have no
findings" rather than as a bug.

Each fact now resolves through an ordered list of names with Claude
Code's own first, so a trace carrying both vocabularies resolves to the
name the analyzer was written against and no existing answer changes.
Resolution is first-match-wins rather than sum-across-matches, so an
emitter stamping both names on one span does not get its tokens counted
twice -- there is a test pinning that.

Cost gets one extra wrinkle: Claude Code reports it on the log records,
a GenAI emitter reports it on the span. costOf() checks the span first
and short-circuits, so a trace carrying both does not pay twice.

This is a read-time alias, not an ingest-time rewrite. What an emitter
sent stays on disk verbatim, the same reason classify() keeps the
original span name whatever kind it decides on.

Verified against a real store, not only in unit tests: a pi run emitted
by costslayer's new OTLP layer previously extracted to $0.00000 and
zero tokens, and now extracts to $0.04218, 12,560 tokens, 11,000
cached, both tool names, and the correct model and stop reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

zzg force-pushed insights-attr-aliases from 6bcb937bfb to 4d0bedd85a

2026-09-19 20:31:59 -07:00

Compare

zzg changed title from ~~Read insight attributes as aliases, and move the module path to git.z.gd~~ to Read insight attributes as aliases, not as one emitter's vocabulary

2026-09-19 20:32:11 -07:00

zzg referenced this pull request

2026-09-19 20:36:08 -07:00

Move the module path to git.z.gd/zzg/fathom #2

zzg commented

2026-09-19 20:37:04 -07:00