Reading the scores
What the probe returns, and how to interpret it.
Every response carries a scores object alongside the usual OpenAI fields. It
maps a probe name to that probe's output.
The probe name is not a fixed string
hallucination is used throughout these docs because it is the usual probe,
but a model can report a different one, or several at once. Read the keys
(response.scores.keys()) rather than hardcoding one, and treat a missing key
as "not scored" rather than an error. If scores is missing entirely, see
Why am I not getting scores?.
Non-streaming
scores holds one array per probe, the same length as the completion:
{
"choices": [{ "message": { "content": "The Eiffel Tower opened in 1889." } }],
"tokens": ["The", " Eiffel", " Tower", " opened", " in", " 1889", "."],
"scores": { "hallucination": [0.01, 0.02, 0.02, 0.05, 0.03, 0.61, 0.01] }
}tokens comes back alongside it: the decoded text of each generated token, in
order, such that joining them reproduces content exactly. This is what makes
the scores usable — without it you would hold N scores against one opaque
string and have to re-tokenise with the identical tokeniser to line them up.
for token, score in zip(response.tokens, response.scores["hallucination"]):
if score is not None and score > 0.5:
print(f"flagged: {token!r}") # flagged: ' 1889'An individual score may be null
The score array is always the same length as tokens, so
scores[probe][i] always describes tokens[i]. An individual entry can
still be null when the probe's reading for that token was unavailable —
most often the last token of a response cut short by max_tokens.
A null means "no score for this token", not "a score of zero". Check for it
before comparing against a threshold, as above.
Streaming
Each chunk carries one number per probe, for that chunk's token:
stream = client.chat.completions.create(
model="google/gemma-4-31B-it",
messages=[{"role": "user", "content": "Tell me about Ada Lovelace."}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content or ""
score = (chunk.scores or {}).get("hallucination")
print(delta, score)Turning scoring off
Scoring is on by default. Pass include_scores: false for a plain completion:
{
"model": "google/gemma-4-31B-it",
"messages": [{"role": "user", "content": "Hello"}],
"include_scores": false
}Both scores and tokens come back null, and the probe surcharge is not
charged. In the Python SDK this must go through extra_body; see
include_scores for the details and
Pricing for what it saves.
Interpreting a score
A higher score means the probe considers the token more likely to be unsupported.
The score is not a probability of error
The scale is not calibrated to "chance this is wrong", and the useful cutoff is not 0.5. Pick a threshold from your own tolerance for false positives, and treat a flag as a prompt to verify a claim rather than a verdict. Email hello@telluvian.ai for the current operating point and its measured precision and recall.
Native vs routed
- Native — the probe reads the hidden states of the same forward pass that produced the token. Highest fidelity.
- Routed — the response is generated by another provider, then replayed through our model with each token forced so the probe can read hidden states it did not itself generate. Meaningful, but a reconstruction. It also adds latency proportional to output length.
Which mode a model uses is in GET /v1/models.