Calibration and Abstention

When a model says it's 80% confident, is it right 80% of the time?

That question sounds simple, but for most modern LLMs the answer is no, not really. The gap between stated confidence and actual accuracy has a name (calibration error), a clean definition, and a visualization that makes it obvious.

I'd been vaguely aware of this for a while before I sat down to dig in. The hero animation at the top summarizes the whole story in one loop: a base model close to the diagonal, RLHF blowing it up, temperature scaling pulling it back. Everything below is fleshing out why that happens and what you can do about it.

I want to work through calibration from first principles: what it means formally, how RLHF broke it, how to measure it, what you can do about it, and why the closely related problem of abstention (teaching a model to say "I don't know") is significantly harder than it looks.

One reason this matters more than it used to: as models get deployed in medical, legal, and financial contexts where people act on the outputs, the difference between a confident-sounding wrong answer and an explicit "I'm not sure" could be the difference between a good outcome and a harmful one.

Where this fits in the series

Arc 8 has been building up a picture of how to know whether a model is working. We looked at loss vs. benchmarks, eval harnesses, contamination, how to evaluate reasoning and tool use, and human preference evaluation. This post cuts across all of those: even with clean data, the right eval setup, and no contamination, how do you know whether to trust the model's answers?

Calibration is the quantitative version of that question. It turns a model from a black box that sometimes-outputs-correct-things into a system you can reason about under uncertainty.

If you need prior posts: cross-entropy and loss (Arc 1) covers how probabilities and log-probs work, and sampling strategies (Arc 5) covers softmax and temperature in the decoding context. This post assumes you're comfortable with both.

What calibration means

The formal definition is clean. A model is perfectly calibrated if, among all the predictions where it reports confidence $c$, it is correct exactly $c$ of the time. For every $c$:

If the model says "80% confident" on a thousand questions, it should get about 800 of them right. If it's saying "80% confident" but only getting 620 right, that's overconfidence. If it's saying "40% confident" but getting 680 right, that's underconfidence.

Calibration is aggregate: it's a property of the distribution of outputs, not any single prediction. It's also separate from accuracy. A model can be well-calibrated and mediocre (consistently accurate about its mediocrity), or well-calibrated and excellent (consistently confident when it should be). The word "calibrated" doesn't mean good at the task. It means honest about how good it is.

One more thing worth being precise about: calibration is meaningful only if you have some notion of what "correct" means. For factual QA, that's clear. For open-ended generation (write me a poem, summarize this document), there's no ground truth to calibrate against. Most calibration research and tooling applies to tasks with well-defined correct answers: classification, factual retrieval, multiple-choice benchmarks. When people talk about LLM calibration in practice, this is what they mean.

ECE: putting a number on it

The standard measure is Expected Calibration Error (ECE). The idea: bucket your predictions into $M$ bins by reported confidence, compute the accuracy and average confidence within each bin, take the weighted average absolute gap.

Where $B_m$ is the set of predictions whose reported confidence falls in the $m$-th bin, $|B_m|/n$ is the fraction of all predictions in that bin, $\text{acc}(B_m)$ is the fraction correct in that bin, and $\text{conf}(B_m)$ is the mean reported confidence in that bin. Lower is better. Zero means perfect calibration.

ECE is just the weighted average misalignment between confidence and accuracy across the full confidence range.

Let me work through a small example. Suppose you run a model on 1000 questions and look at its predictions in the 70%–80% confidence bin. You have 200 predictions there. Of those 200, 140 are correct (70%) and 60 are wrong. The mean confidence in the bin is 0.76. The gap is $|0.70 - 0.76| = 0.06$. This bin contributes $\frac{200}{1000} \times 0.06 = 0.012$ to the ECE. Do that for all ten bins and sum. A final ECE of 0.03 is good. 0.15 is a model you'd be nervous about deploying without further work.

One caveat that comes up a lot: ECE is sensitive to bin count and to how you handle bins with few samples. With $M = 10$ equally-spaced bins, the highest-confidence bin (90–100%) often has very few samples. Models rarely output predictions above 95% confidence on held-out benchmarks, even RLHF-tuned ones. Empty or sparse bins can distort the aggregate. Some researchers prefer adaptive calibration error (ACE), which uses equal-count bins (each bin has the same number of samples) instead of equal-width bins. It's slightly more complicated but more robust. We'll stick with standard ECE here because it's what most papers report.

The natural visualization for calibration is the reliability diagram: a bar chart where each bar covers one confidence bin, the bar height is the accuracy in that bin, and you overlay a diagonal line at $y = x$ for reference. Perfect calibration means every bar touches the diagonal. Bars systematically above the diagonal mean underconfidence (the model is right more often than it says it will be). Bars below the diagonal mean overconfidence.

What always strikes me about this diagram is how stark the RLHF story is. The base model tracks the diagonal reasonably closely. The RLHF-tuned model systematically diverges, especially for high-confidence predictions. The model learned to express confidence like a person giving a TED talk, and the calibration paid for it.

The good news is that temperature scaling mostly works. It's not perfect, but the ECE drops substantially, and the cost is nothing except a one-time calibration run on a held-out set.

Why base models are reasonably calibrated

This surprised me when I first dug into it. Kadavath et al. (2022) found that language models' intrinsic token probabilities (the log-softmax output) are reasonably well-calibrated on factual queries, even without any explicit calibration training. The models "mostly know what they know."

The pretraining objective is a surprisingly natural calibration signal. When you're predicting the next token from the distribution of the internet, you get thousands of gradient updates on each factual claim, each one nudging the model's probability toward the empirical frequency of the correct answer in the training data. A fact that appears in 70% of relevant documents ends up with a probability around 0.7. Not perfect (training distribution mismatch, memorization, and deduplication all mess with it), but a real signal.

"Token probabilities" here means the probability of the correct answer token(s), which is what evaluation harnesses actually use. When an eval harness scores a model on a multiple-choice question, it scores the log-probability of "A" vs. "B" vs. "C" vs. "D" and picks the highest. The model never generates a full text response; it just assigns probabilities to the four answer tokens. Those probabilities are the raw pre-RLHF signal, and they're the ones that are well-calibrated.

When a model generates a full-text answer ("The answer is A because..."), the confidence in that response has been RLHF-shaped. This is part of why evaluation harnesses measure different things than chatbot-style evaluation, and why calibration in chat contexts is harder than calibration in eval-harness contexts.

There's also a model-size effect. Larger pretrained models tend to be better calibrated than smaller ones on factual questions. Somewhat counterintuitive (you'd expect larger models to be more confident because they know more), but it makes sense when you think about gradient dynamics. Larger models fit the training data better, so their probability estimates reflect the training distribution more accurately. Smaller models have higher variance, which manifests as overconfidence in some domains and underconfidence in others. Base model calibration degrades as you scale down from frontier models, which is relevant when you're choosing a deployment model.

Why RLHF hurts calibration

RLHF and instruction-tuning are optimizing for human preference, not calibration. Humans in the feedback loop prefer confident, fluent responses. A model that says "I think the capital of France is Paris" gets worse ratings than one that says "The capital of France is Paris." Hedges feel wishy-washy. Directness feels authoritative.

So the model learns to be direct. And "direct" in practice means more confident, fewer qualifications, stronger claims. On questions the model knows, this is fine. On questions at the edge of its knowledge, or factual hallucinations, it now expresses the same confident tone as the things it does know. The confidence is stylistically uniform even when the underlying probability distribution isn't.

The result is that RLHF-tuned models are often meaningfully miscalibrated compared to their base versions. The ECE goes up, the reliability diagram bows below the diagonal, and the model has learned to sound certain.

This is the fundamental tension. RLHF makes models more useful (they follow instructions, they don't hedge uselessly on obvious things), but it erodes the honest uncertainty signal that base model token probs were accidentally providing.

A concrete way to see this. GPT-4's technical report shows calibration plots (Figure 8) on a subset of MMLU: the pre-trained base model had an ECE of roughly 0.007, while the post-trained (RLHF) model's ECE jumped to around 0.074. That's a tenfold degradation from a single round of post-training. The general pattern (RLHF hurts calibration substantially) has been replicated across other models and confirmed by independent studies.

I find it useful to think of RLHF as applying pressure in the direction of confident-sounding language, across all domains simultaneously. The model doesn't become uncertain about things it used to know. The underlying weights presumably still have the same probability for "France's capital." What changes is the style of the output. The probability distribution over tokens still exists; it's just decoupled from the confident-sounding generation style the model learned to produce.

Verbalized confidence and why it's different

One natural response to this: just ask the model how confident it is. "On a scale of 0-100%, how sure are you about this?"

This is called verbalized confidence, and it's worth distinguishing carefully from token probability confidence. They measure different things and they correlate poorly unless the model has been explicitly trained to make them match.

Token probability confidence is implicit and structural. It's the raw output logit, not a deliberate statement. Verbalized confidence is a learned behavior: the model generates a number as text, which means it's subject to all the same RLHF pressures, fine-tuning patterns, and prompt framing effects as any other generation. The model has no guaranteed introspective access to its own internal states. When it says "I'm 75% confident," it's pattern-matching on contexts where that number tends to appear, not reading a probability gauge from its own weights.

There's an asymmetry here. Token-level confidence does contain real information; the model genuinely assigns different probabilities to different tokens. But that probability is over the next token in the sequence, conditioned on everything before it. It isn't a well-formed epistemic probability over a claim. The aggregate across a generated response is a rough proxy for uncertainty, and a noisy one.

If you've ever pulled the logprobs from the OpenAI API and tried to use the sum (or product) of token log-probs as a confidence signal, you've encountered this. The per-token probability includes contributions from:

• Syntactic decisions (grammar is high-probability, so correct grammar inflates aggregate confidence)
• Common words ("the," "is," "a"), always high probability regardless of the factual claim
• The actual factual tokens that matter for uncertainty

A sentence like "The capital of France is Paris." has its total probability dominated by the syntactic structure, not by the factual part. "Paris" is high-probability partly because it's the right answer and partly because it's a common word that appears in many contexts after "capital of France is." This is why raw log-prob sum is a noisy signal: you're conflating uncertainty about facts with uncertainty about word choice.

One practical approach: instead of aggregating over all tokens, focus the confidence signal on the answer token(s) specifically. For a multiple-choice question with answer choices A/B/C/D, just look at the logit for "A", "B", "C", "D" directly. For open-ended factual questions, look at the first entity-level token (the first token of the named answer). This is what the evaluation harnesses we covered in an earlier post actually do when they compute model accuracy; they compare token probabilities at the relevant positions, not over the full generation.

Some recent work has tried to close this gap. Tian et al. (2023) showed you can train models to produce verbalized confidences that correlate reasonably with accuracy, but it requires explicit supervision. You need examples of the form "this claim is correct / incorrect" paired with the model's expressed confidence, not just human preference feedback.

Selective prediction: when to abstain

Abstention is harder than calibration. Calibration says: given that we're going to answer, how honest are our confidence numbers? Abstention says: should we answer at all?

The framework that makes this tractable is selective prediction. You define a policy: answer when model confidence exceeds a threshold $\tau$, otherwise abstain. As you vary $\tau$:

• Low $\tau$: answer almost everything, high coverage, lower average accuracy
• High $\tau$: answer only when very confident, low coverage, higher average accuracy

The Pareto frontier you trace as you sweep $\tau$ is the natural way to evaluate this. A better model (or a better-calibrated model) achieves higher accuracy at any given coverage level.

The coverage-accuracy trade-off makes the stakes concrete. In a medical QA system, you might decide you need 90% accuracy on answered questions. Then you want to know: what fraction of questions can you answer before you fall below that threshold? A poorly-calibrated model is a bad deal here. It has to throw away more questions to meet the accuracy bar, because its high-confidence predictions are often wrong.

The "answer everything" endpoint is where the model's raw average accuracy lives. The "abstain everything" endpoint is vacuously 100% (you can't be wrong if you never answer). The shape between those endpoints tells you how much you can gain by abstaining selectively. For a well-calibrated model the curve is steep; even abstaining on the bottom 20% of confidence predictions substantially improves accuracy. For a poorly-calibrated model the curve is flatter, because the confidence signal is noisy, and filtering by it doesn't help much until you've thrown away most of your coverage.

This is the practical case for calibration. Not "it's epistemically nice to be honest about uncertainty," though that's true. The concrete case is that calibration determines how useful your abstention policy can be. An overconfident model and a well-calibrated model might have the same average accuracy, but the well-calibrated model gives you a better trade-off on the operational question of "when do I trust this answer?"

Put another way, the Pareto curve is what you're buying when you improve calibration. You aren't buying higher accuracy. You're buying a more useful confidence signal, which lets you make better decisions about when to act on the model's output and when to escalate.

I think about it the same way I think about weather forecasts. A forecaster who says "70% chance of rain" and is right 70% of the time across all their 70%-forecasts is valuable not because they're always right, but because you can rationally decide to bring an umbrella based on their number. A model that says "I'm 90% confident" and is right 58% of the time on those predictions is useless as a decision input, because you can't calibrate your behavior to a number that doesn't mean anything.

What does abstention look like in practice?

Setting a threshold is conceptually clean but operationally messy. Here's what the actual deployment decision looks like.

Suppose you've built a medical QA assistant. You've run the model on a validation set of 2,000 questions with known correct answers. You've computed the coverage-accuracy curve. You've decided that for patient-facing outputs, you need 92% accuracy on answered questions. That's your risk threshold.

From the curve, you read off: at $\tau = 0.74$, you get 92% accuracy with 61% coverage. That means the system will decline to answer 39% of questions. That's a lot of abstentions. Your product team is going to ask: can we get it lower?

From here, the options diverge. You can:

1. Accept the threshold and route abstained questions to human reviewers. This is the safest path. Coverage of 61% with human backup might be acceptable if the workload is manageable.
2. Improve the model's calibration (temperature scaling, more calibration training). With better calibration, the same accuracy target might require less abstention, because the curve's shape improves, not its endpoint accuracy.
3. Use self-consistency: run 5 samples, require 4/5 agreement to answer. This often improves the coverage-accuracy trade-off at the cost of 5x inference compute.
4. Route by question type: if you know that certain query categories are consistently well-calibrated (e.g., pharmacology calculations) and others aren't (e.g., obscure case histories), apply the threshold only where you need it, and answer more freely on the reliable domains.

The right answer depends on your specific system. But notice what you need to make any of these decisions: a coverage-accuracy curve. Without it, you're setting a threshold by gut feel, and "gut feel" about model reliability in high-stakes contexts is how things go wrong.

How few-shot prompting distorts confidence

One wrinkle that's easy to overlook: few-shot prompting introduces its own calibration biases. When you provide $k$ examples in the prompt, the model has to fit the pattern of the examples you've shown. If all your examples happen to start with "A", the probability assigned to "A" as the first answer token is inflated because the in-context pattern points that way, not because the model believes the answer is A.

Zhao et al. (2021) showed that few-shot accuracy and calibration both depend heavily on example order, selection, and label balance. Two models with the same base accuracy can have dramatically different ECEs depending on how the prompt is constructed. If you're running your own calibration analysis, be consistent in your few-shot setup across models. If you're comparing numbers from different papers, be suspicious of comparisons that differ in few-shot count or example selection.

Practical approaches to calibration

A few tools in the toolbox, ranging from trivial to expensive.

Temperature scaling is the cheapest fix. You train the model normally, then hold out a small validation set and find a single scalar $T$ such that dividing all the logits by $T$ minimizes ECE. $T > 1$ spreads the distribution out (reduces overconfidence), $T < 1$ sharpens it. The whole thing is a one-dimensional optimization, no model retraining required.

The limitation is that it applies a uniform correction to all predictions. If the model is overconfident on entity facts but underconfident on procedural questions, a single $T$ can't fix both at once.

To be concrete about the math: if the model outputs logits $z_i$ for token $i$, the softmax probability is $p_i = \exp(z_i) / \sum_j \exp(z_j)$. Temperature scaling modifies this to $p_i = \exp(z_i / T) / \sum_j \exp(z_j / T)$. At $T = 1$ nothing changes. At $T = 2$ you divide all logits by 2, effectively flattening the distribution. The top token stays the top token, but the margin shrinks. At $T = 0.5$ (sharpening), the distribution concentrates more mass on the top token.

The calibration-optimal $T$ is found by minimizing ECE on a held-out validation set (not the same test set you'll evaluate on). For RLHF-tuned models, you typically find $T$ somewhere in the range 1.2–2.0. The optimization is just a one-dimensional line search; the full calibration run takes minutes even on large benchmarks.

Platt scaling is a refinement. It uses a linear function of the logit ($a \cdot z + b$) instead of a single temperature. Two parameters instead of one, which lets it also correct systematic additive bias, not just scale. Still no retraining. For most LLM use cases temperature scaling is sufficient, but if you're seeing a consistent over- or under-shift across the board in addition to scale issues, Platt scaling is worth trying.

Self-consistency is a different approach. Instead of calibrating a single generation, you sample the model multiple times with some randomness and look at agreement across samples. If the model consistently answers "Paris" to "capital of France?" across 20 samples, that's a strong signal of confidence. If it wavers between three different named executives, that's uncertainty, even if each individual generation sounds authoritative.

Self-consistency is effective but expensive. You're paying for $k$ generations when you wanted one. In latency-sensitive applications this is often not viable. The practical sweet spot is $k = 5$ to $k = 10$ samples. Below 5 the variance is high (two samples can agree for the wrong reasons). Above 10 you're spending a lot on diminishing returns.

Semantic entropy takes this idea further. Instead of checking literal string agreement, you cluster semantically equivalent answers and measure the entropy over meaning clusters. "Paris" and "the city of Paris" and "Paris, France" all mean the same thing, so that's low entropy (confident). But "Smith", "Johnson", and "Williams" as samples for the same question are genuinely different, which is high entropy (uncertain). This is more robust to paraphrasing than string comparison. Farquhar et al. use entailment classifiers (does A entail B, and B entail A?) to do the clustering.

Prompting for uncertainty is the lowest-tech approach. Something like "answer the question, and if you're not sure, say so" or few-shot examples that demonstrate hedging on uncertain questions. This doesn't give you a number, but for a chatbot where a human can follow up, behavioral hedging is often good enough. The downside is that RLHF priors work against you; you may need to fight the model's trained directness with fairly explicit prompting.

The broader picture: these approaches form a cost/reliability hierarchy.

• Prompting: free, unreliable, useful as a baseline or in low-stakes contexts.
• Temperature scaling: cheap one-time calibration run, works well for systematic over/under-confidence.
• Self-consistency: expensive at inference time, works independently of calibration quality.
• Semantic entropy: expensive and needs a clustering model, but most robust to paraphrasing.
• Calibration training (fine-tuning on labeled correct/incorrect examples): expensive, needs labeled data, but can solve problems post-hoc methods can't.

In practice, production systems often combine these: temperature scaling to fix the baseline, self-consistency for high-stakes inferences, and some form of domain-based routing to avoid asking the model about things it demonstrably hallucinates.

Computing ECE

Here's the minimal implementation. You need three things: a set of (confidence, correct) pairs, a bin count, and a summation loop.

import numpy as np
from scipy.optimize import minimize_scalar
 
def compute_ece(confidences, corrects, n_bins=10):
    """Expected Calibration Error."""
    confidences = np.array(confidences, dtype=float)
    corrects = np.array(corrects, dtype=float)
    n = len(confidences)
    ece = 0.0
    bin_edges = np.linspace(0, 1, n_bins + 1)
    for i in range(n_bins):
        lo, hi = bin_edges[i], bin_edges[i + 1]
        in_bin = (confidences >= lo) & (confidences < hi)
        if i == n_bins - 1:
            in_bin = (confidences >= lo) & (confidences <= hi)
        bin_size = in_bin.sum()
        if bin_size == 0:
            continue
        bin_acc  = corrects[in_bin].mean()
        bin_conf = confidences[in_bin].mean()
        ece += (bin_size / n) * abs(bin_acc - bin_conf)
    return ece
 
def temperature_scale(logits, T):
    """Apply temperature scaling to a batch of logits."""
    # logits: (N, vocab), raw pre-softmax scores
    # Returns softmax probabilities under temperature T
    scaled = logits / T
    exp_scaled = np.exp(scaled - scaled.max(axis=1, keepdims=True))  # stable softmax
    return exp_scaled / exp_scaled.sum(axis=1, keepdims=True)
 
def find_optimal_temperature(logits, labels, val_size=500):
    """
    Binary search for T that minimizes ECE on a validation split.
    logits: (N, num_classes), labels: (N,) true class indices
    """
    # Hold out a validation split to tune T (don't use your test set!)
    N = len(labels)
    idx = np.random.default_rng(42).choice(N, size=val_size, replace=False)
    val_logits, val_labels = logits[idx], labels[idx]
 
    def ece_at_temp(T):
        probs = temperature_scale(val_logits, T)
        confs = probs.max(axis=1)
        preds = probs.argmax(axis=1)
        corrects = (preds == val_labels)
        return compute_ece(confs, corrects)
 
    result = minimize_scalar(ece_at_temp, bounds=(0.1, 10.0), method='bounded')
    return result.x
 
def coverage_accuracy_curve(confidences, corrects, n_thresholds=50):
    """Pareto curve: sweep τ, compute (coverage, accuracy) at each threshold."""
    thresholds = np.linspace(0, 1, n_thresholds)
    results = []
    for tau in thresholds:
        answered = confidences >= tau
        coverage = answered.mean()
        accuracy = corrects[answered].mean() if answered.sum() > 0 else 1.0
        results.append({"tau": float(tau), "coverage": float(coverage),
                        "accuracy": float(accuracy)})
    return results
 
# --- Quick sanity check ---
rng = np.random.default_rng(42)
confs = rng.uniform(0, 1, 1000)
corr  = rng.random(1000) < confs          # well-calibrated
confs_over = np.clip(confs + 0.2, 0, 1)  # overconfident
 
print(f"ECE (calibrated):   {compute_ece(confs, corr):.3f}")       # ~0.02
print(f"ECE (overconfident): {compute_ece(confs_over, corr):.3f}") # ~0.18
 
# What does selective prediction look like?
curve = coverage_accuracy_curve(confs_over, corr)
# At τ=0.8 (answer when ≥80% confident), what's our coverage and accuracy?
high_conf = [p for p in curve if p["tau"] >= 0.79][0]
print(f"τ=0.8: coverage={high_conf['coverage']:.2f}, "
      f"accuracy={high_conf['accuracy']:.2f}")

The ECE number alone tells you that the model is miscalibrated, not how. For that you need the full reliability diagram to see whether overconfidence is uniform or concentrated in certain confidence ranges.

The find_optimal_temperature function is essentially a one-dimensional calibration run. In practice, you want to do this once, save the optimal $T$, and apply it at inference time by dividing your logits before the final softmax. If you're serving via an API that doesn't expose raw logits, you can still compute ECE from the probabilities the API returns, but you can't apply temperature scaling post-hoc.

A related gotcha: when measuring ECE for an LLM on a QA task, the raw logit probabilities aren't always available. You can use the API's logprobs parameter (most major APIs support this for top-N tokens), extract a verbalized confidence from the generated text, or use self-consistency sampling frequency as a proxy. Each measures something slightly different. Be explicit about which one your confidences array actually is.

Misconceptions

"A model that's often wrong is poorly calibrated." Calibration is about the relationship between confidence and accuracy, not accuracy alone. A model can be consistently mediocre (gets 40% of questions right) and still be well-calibrated, as long as it says "40% confident" on average for those questions. Calibration is about honesty, not competence.

"Asking the model to tell you its confidence will give you a calibrated number." Verbalized confidence is a generated text token, not a readout from an internal probability register. Unless the model was specifically trained on calibration-paired data, verbalized numbers are better understood as stylistic output (matching contexts where hedged/confident language appears in training) than as meaningful probability estimates.

"Abstention is just about setting a confidence threshold." The threshold is easy. The hard part is that confidence scores from poorly-calibrated models are unreliable as inputs to that threshold. A model that reports 90% confidence and is right 60% of the time will keep answering under almost any plausible threshold, because it never says it's less than 90% confident. Calibration and abstention are entangled: you can't do selective prediction well without reasonably calibrated confidences.

What's next

Calibration and abstention round out our tour of what evaluation actually looks like from the inside: measuring not just whether models are right, but whether they know when they're right. That closes Arc 8.

The next arc shifts from evaluating models to retrieving the context they need. It opens with embeddings from scratch: from Word2Vec to E5, building up how text becomes geometry, from count-based vectors through contrastive learning to the modern embedding models that power retrieval. Calibration shows up again in Arc 12, where it sits at the center of responsible scaling policy discussions. The atom you just learned is going to come back.

Additional reading (and watching)

• Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML 2017. The canonical treatment of ECE, temperature scaling, and the reliability diagram, originally for image classifiers but the framework transferred directly to LLMs.

• Kadavath, S., et al. (2022). Language Models (Mostly) Know What They Know. arXiv:2207.05221. Found that large LMs can estimate their own accuracy reasonably well via self-evaluation, and that token probabilities are surprisingly calibrated on factual questions.

• OpenAI. (2023). GPT-4 Technical Report. arXiv:2303.08774. Documents post-training calibration degradation and the temperature adjustments applied to partially compensate.

• Tian, K., et al. (2023). Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. EMNLP 2023. Shows that RLHF-tuned models can be prompted to produce better-calibrated verbalized confidences via specific phrasing strategies.

• Geifman, Y. & El-Yaniv, R. (2017). Selective Classification for Deep Neural Networks. NeurIPS 2017. Formalizes the selective prediction framework, coverage-vs-accuracy trade-off, and the Pareto curve as an evaluation tool.

• Zhao, S., et al. (2021). Calibrate Before Use: Improving Few-Shot Performance of Language Models. ICML 2021. Demonstrated that few-shot prompting introduces systematic calibration biases, and proposed a label-prior correction to compensate.

• Wang, X., et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023. Demonstrated that sampling multiple reasoning chains and taking the majority answer significantly improves accuracy and implicitly provides an uncertainty estimate.

• Farquhar, S., et al. (2024). Detecting Hallucinations in Large Language Models Using Semantic Entropy. Nature, 2024. Introduced semantic entropy as a calibrated uncertainty measure that clusters semantically equivalent answers, avoiding the brittleness of literal-string agreement.

• Anthropic. (2024). Claude 3 Model Card. Notes calibration as an ongoing research priority and describes behavioral abstention patterns in Claude's instruction-following training.