When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors

This paper introduces and systematically studies 'data referencing errors' (DREs) — cases where LLMs correctly parse table structure but still misquote or omit specific values during reasoning — and shows that a lightweight trained critic model can detect these errors and boost downstream accuracy via filtering and rejection sampling.

Problem Statement

LLMs are widely deployed for table-based QA and reasoning, but subtle citation errors in intermediate steps can silently corrupt correct-looking answers even when the model demonstrably understands the table's structure. Prior work has only examined this failure mode in small, ad hoc studies, leaving practitioners without a reliable way to measure or mitigate it at scale. This paper closes that gap with a systematic evaluation and a practical detection tool.

Key Novelty

First large-scale, systematic evaluation of tabular data referencing errors (DREs) spanning multiple models (1.7B–20B parameters) and multiple table tasks
Demonstration that explicit data-referencing verification can serve as an effective critic signal, improving accuracy through critic-based filtering and rejection sampling
A lightweight 4B-parameter critic model that generalizes to out-of-distribution DREs and can assist inference for much larger models

Evaluation Highlights

DREs are shown to occur pervasively across all tested models regardless of scale (1.7B to 20B parameters)
Critic-based filtering and rejection sampling improves final answer accuracy by up to 12.0%
The 4B critic model achieves an average 78.2% F1 score detecting DREs in both in-distribution and out-of-distribution settings

Signal Assessment

5/10 The paper offers a solid, well-executed empirical contribution — a new systematic framing and measurement of an underexplored failure mode plus a practical, lightweight mitigation — but it builds on established critic/rejection-sampling paradigms rather than introducing a fundamentally new method.

Methodology

Formally define and categorize data referencing errors (incorrect citation or omission of table values) distinct from structural misunderstanding
Systematically evaluate DRE frequency across models ranging from 1.7B to 20B parameters and across diverse table reasoning tasks
Use data-referencing correctness as a critic signal to filter flawed reasoning chains and drive rejection sampling for improved final answers
Train a lightweight 4B-parameter critic model specifically to detect DREs and validate its generalization on out-of-distribution data
Apply the trained critic to assist inference in larger models, verifying it improves their accuracy without requiring retraining of the larger model

System Components

DRE Taxonomy & Evaluation Framework

A systematic scheme for defining and measuring data referencing errors (incorrect/omitted value citations) independent of table structure comprehension, applied across models and tasks

Critic-based Filtering

Uses DRE detection to identify and discard reasoning chains that misreference table data before finalizing an answer

Rejection Sampling Pipeline

Regenerates model outputs when the critic flags data referencing errors, selecting cleaner reasoning traces to boost final accuracy

Lightweight 4B Critic Model

A dedicated, efficiently-sized model trained to detect DREs with strong F1 performance, generalizing to unseen (OOD) error patterns and usable to assist larger LLMs at inference time

Results

Metric/Benchmark	Baseline	This Paper	Delta
Answer accuracy with critic-based filtering/rejection sampling	Standard LLM inference (no critic)	Critic-filtered + rejection sampling	up to +12.0%
DRE detection F1 (in-distribution + out-of-distribution avg)	N/A (no prior systematic detector)	78.2% F1 (4B critic model)	New capability
Model scale where DREs observed	N/A	Present across 1.7B–20B parameter models	Confirms DREs are scale-invariant

Key Takeaways

Correct table structure comprehension does not guarantee correct value citation — practitioners should treat data referencing as a distinct failure mode requiring its own evaluation and mitigation
A small, dedicated critic model (4B params) can detect data referencing errors accurately enough (78.2% F1) to meaningfully improve larger models' outputs without retraining them, offering a cost-effective deployment pattern
Adding critic-based filtering and rejection sampling to table-reasoning pipelines is a practical, near-term way to gain up to 12% accuracy improvements
DREs persist across the entire tested model scale range (1.7B–20B), so scaling alone is unlikely to solve this issue — targeted verification mechanisms are needed

Abstract

While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.