Files
zzgandClaude Opus 5 4d0bedd85a Read insight attributes as aliases, not as one emitter's vocabulary
Fathom ingests OTLP from anything, and ingest already knew emitters
disagree about names: classify() keys on gen_ai.operation.name as well
as Claude Code's span.type, and the correlator's join keys list
gen_ai.tool.call.id alongside tool_use_id.

The insights analyzer did not. It read cost_usd, input_tokens,
output_tokens, cache_read_tokens, model, stop_reason, tool_name and
user_prompt -- Claude Code's private vocabulary, exclusively. A trace
from any other emitter therefore rendered correctly in the trace view
and reported no cost, no tokens, and every tool named "tool". That is a
silent wrong answer, and the kind that presents as "you have no
findings" rather than as a bug.

Each fact now resolves through an ordered list of names with Claude
Code's own first, so a trace carrying both vocabularies resolves to the
name the analyzer was written against and no existing answer changes.
Resolution is first-match-wins rather than sum-across-matches, so an
emitter stamping both names on one span does not get its tokens counted
twice -- there is a test pinning that.

Cost gets one extra wrinkle: Claude Code reports it on the log records,
a GenAI emitter reports it on the span. costOf() checks the span first
and short-circuits, so a trace carrying both does not pay twice.

This is a read-time alias, not an ingest-time rewrite. What an emitter
sent stays on disk verbatim, the same reason classify() keeps the
original span name whatever kind it decides on.

Verified against a real store, not only in unit tests: a pi run emitted
by costslayer's new OTLP layer previously extracted to $0.00000 and
zero tokens, and now extracts to $0.04218, 12,560 tokens, 11,000
cached, both tool names, and the correct model and stop reason.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-19 20:31:53 -07:00

103 lines
3.9 KiB
Go

package insights
import "git.z.gd/zzg/fathom/internal/model"
// Emitters disagree about what to call the same fact, and the analyzer has to
// read facts rather than names.
//
// Ingest and correlation already knew this: classify() keys on
// gen_ai.operation.name as well as Claude Code's span.type, and the
// correlator's join keys list gen_ai.tool.call.id alongside tool_use_id. The
// analyzer did not, and read Claude Code's private vocabulary exclusively. A
// trace from any other emitter therefore rendered correctly in the trace view
// and reported no cost, no tokens, and every tool named "tool" — a silent
// wrong answer that presents as "no findings" rather than as a bug.
//
// The lists below are ordered, and Claude Code's own names come first in
// every one. A trace carrying both vocabularies resolves to the name the
// analyzer was originally written against, so adding these cannot change any
// existing answer. Resolution is first-match-wins rather than
// sum-across-matches, for the same reason: an emitter that stamps both names
// on one span must not have its tokens counted twice.
//
// This is a read-time alias, not an ingest-time rewrite. What an emitter sent
// stays on disk verbatim — the same reason classify() keeps the original span
// name whatever kind it decides on.
var (
costKeys = []string{"cost_usd", "gen_ai.usage.cost"}
inputTokenKeys = []string{"input_tokens", "gen_ai.usage.input_tokens"}
outputTokenKeys = []string{"output_tokens", "gen_ai.usage.output_tokens"}
// Anthropic's cache accounting is not in the GenAI conventions proper;
// these are the names the conventions' Anthropic mapping uses.
cacheReadKeys = []string{"cache_read_tokens", "gen_ai.usage.cache_read_input_tokens"}
cacheWriteKeys = []string{"cache_creation_tokens", "gen_ai.usage.cache_creation_input_tokens"}
// Request before response: the analyzer groups by what was asked for, and
// a response model can name a dated build of the same requested alias.
modelKeys = []string{"model", "gen_ai.request.model", "gen_ai.response.model"}
toolNameKeys = []string{"tool_name", "gen_ai.tool.name"}
stopReasonKeys = []string{"stop_reason", "gen_ai.response.finish_reasons"}
promptKeys = []string{"user_prompt", "prompt", "gen_ai.prompt"}
)
// promptEventNames are the log-record event names that carry a user's prompt.
var promptEventNames = []string{"user_prompt", "gen_ai.user.message"}
func isPromptEvent(name string) bool {
for _, n := range promptEventNames {
if name == n {
return true
}
}
return false
}
// firstStr returns the first key present with a non-empty string value.
func firstStr(a model.Attrs, keys []string) string {
for _, k := range keys {
if v, ok := a[k]; ok && v.Str != "" {
return v.Str
}
}
return ""
}
// firstFloat returns the first key present with a numeric value, and whether
// any was found. The two are distinct: a genuine zero cost and a missing cost
// are different facts, and collapsing them is how "this run produced nothing"
// becomes unanswerable.
func firstFloat(a model.Attrs, keys []string) (float64, bool) {
for _, k := range keys {
if v, ok := a[k]; ok {
if f, ok := v.Float(); ok {
return f, true
}
}
}
return 0, false
}
// firstNum is firstFloat truncated to an integer, for token counts.
func firstNum(a model.Attrs, keys []string) int64 {
if f, ok := firstFloat(a, keys); ok {
return int64(f)
}
return 0
}
// costOf resolves a cost for one span and its correlated records. Claude Code
// reports cost on the log records; a GenAI emitter reports it on the span
// itself. The span is checked first and short-circuits, so a trace that
// carries both does not pay twice.
func costOf(sp model.Span, logs []model.LogRecord) float64 {
if c, ok := firstFloat(sp.Attrs, costKeys); ok {
return c
}
var total float64
for _, l := range logs {
if c, ok := firstFloat(l.Attrs, costKeys); ok {
total += c
}
}
return total
}