How AI Understands Tables and Structured Data: Headers, Units, and Auditable Answers

The question that exposes weak table understanding

A finance analyst needs one number: gross margin by product line for the third quarter, excluding intercompany transfers. The answer sits in a table on page 14 of a board pack — merged header cells spanning two rows, a footnote that silently changes the unit from thousands to millions, a subtotal row that looks like a data row, and a revised version of the same table two pages later. A system that reads the document as a stream of text will return a confident number that is wrong, and it will sound exactly as confident as when it is right.

This is the central difficulty with tables, and it is not a language problem. Prose carries meaning through grammar and word order; a table carries meaning through geometry. The position of a value under a column header, the indentation that marks a subtotal, the alignment that separates a label from a figure, the blank cell that means zero in one report and not-reported in another — all of that is structure, and all of it disappears the moment a table is flattened into a sentence. Understanding tables is therefore an ingestion and modeling problem before it is ever a prompting problem.

That distinction matters commercially because tables are where the decisions live. Revenue breakdowns, pricing tiers, inventory levels, SLA matrices, headcount grids, compliance thresholds and contract rate cards are almost never written as paragraphs. If an enterprise RAG system handles narrative documents well but degrades on tables, it will look impressive in a demo and be quietly abandoned by the finance and operations teams who actually needed it.

From page layout to records: what the ingestion layer must do

Before any model reasons about a table, a deterministic pipeline has to reconstruct it. That pipeline detects the table region, recovers the cell grid including row and column spans, identifies which rows are headers and how they nest, normalizes types and units, and then emits each row as a structured record with explicit field names — for example, product_line, region, period, revenue_millions, margin_pct, is_subtotal. Only at that point does the content become something a retrieval system or a query engine can work with, because now a value is bound to a label instead of floating near one.

Header inference is where most real-world pipelines fail. Multi-tier headers (“2025” spanning four quarterly columns, with “Actual” and “Budget” underneath) must be concatenated into unambiguous field names such as q3_2025_budget. Merged cells must be forward-filled only when the merge genuinely implies repetition, not when it signals a group boundary. Footnote markers attached to a header, such as an asterisk that changes the currency, must be resolved before the numbers are indexed. Subtotal and total rows must be flagged and usually excluded from aggregation, otherwise a query that sums a region will double-count every subtotal it retrieves.

Type and unit normalization is equally consequential. Parentheses meaning negative, thin spaces as thousands separators, percentages stored as 0.14 versus 14, dates in day-first and month-first formats inside the same workbook — each of these produces plausible-looking wrong answers rather than errors. A well-built pipeline records the original string alongside the normalized value, so the answer can be traced back to what the document actually said.

The common failure modes are worth naming explicitly, because they are what you should test against:

None of this is glamorous, and that is the point. The reliability of every downstream answer is bounded by the quality of this normalization step.

• Multi-row headers collapsed into a single ambiguous column label
• Merged cells forward-filled across group boundaries, duplicating values
• Blank cells interpreted as zero when they mean not reported
• Subtotal rows counted as ordinary data, inflating every aggregate
• Footnote-driven unit changes (thousands to millions) applied to the wrong table
• Sign conventions ignored, so a negative variance is read as positive

Three levels of table understanding, and which one your question needs

Not every question requires the same depth of understanding, and choosing the wrong level is a common source of both over-engineering and silent error. At the first level, the system extracts table text and retrieves the relevant chunk. This is enough to answer “what does the pricing table say about the enterprise tier?” because the answer is a lookup and the model only needs to read adjacent cells correctly. It is not enough to answer anything that requires arithmetic across rows or a comparison between two tables.

At the second level, the system treats each row as a typed record with a stable schema, so retrieval happens over fields rather than over text. Now “which suppliers have a lead time above 30 days?” becomes a filter on a numeric field instead of a semantic similarity search, and the answer set is exact. At the third level, the system translates the question into an executable query against a governed data source — a database view, a warehouse table, or a semantic layer with defined metrics — and runs it. This is the only level that reliably handles aggregation, ranking, period comparison and joins, because the computation happens in the database rather than in the model's head.

Most enterprise failures come from asking level one to do level three work. A model that can retrieve the right table and read it fluently still cannot be trusted to sum forty rows, exclude three subtotals, and apply a footnote about currency without making an arithmetic or bookkeeping mistake somewhere in the chain.

A practical mapping looks like this:

The honest design decision is to route each question to the level it needs, rather than forcing every question through a single retrieval path.

• Text extraction — best for definitions, rate cards and single-cell lookups where the value is read, not calculated
• Record-level structure — best for filtering, matching and shortlisting across a known set of columns
• Query generation over a governed source — best for totals, averages, rankings, period comparisons and anything spanning multiple tables

Retrieval over rows versus generating a query

Once tables are records, two architectures compete. The first indexes rows as retrievable units and lets the model assemble an answer from the rows it retrieved, which works well when the corpus is document-shaped, the schema varies from file to file, and the questions are exploratory. Its weakness is arithmetic and completeness: retrieval returns a top-k subset, and a subset cannot answer “total revenue across all regions” without an aggregation step the model must perform itself.

The second generates a query — SQL against a warehouse, an API call against a live system — and returns the computed result. This is more precise for numbers but depends on a stable, documented schema and on metric definitions that do not drift between teams. If finance defines gross margin one way and sales defines it another, the model will faithfully execute whichever definition it was given and produce two correct answers that contradict each other. That is a governance problem, and it must be solved in a semantic layer, not in a prompt.

In practice, the strongest setups are hybrid. Structured questions go to query generation against live systems; document-bound tables that exist only inside a PDF or a scanned report go through record extraction and retrieval; and the router decides which path applies based on whether a governed source exists for the entity being asked about. This is the split Pilardin is built around — document understanding through RAG, plus connectors to live systems through the pipeline module, in one workspace rather than two disconnected tools.

The comparison in practice:

Whichever path is chosen, the system should state which one it used. An answer that says “computed from the Q3 sales view, filtered to region = EMEA” is auditable; an answer that says “based on the documents” is not.

• Row retrieval — best for heterogeneous document tables, fuzzy questions and small result sets; weakest on aggregation and completeness
• Query generation — best for large, well-modeled datasets and numeric questions; weakest when schemas are undocumented or metric definitions conflict
• Hybrid routing — best for organizations where the same question may touch a board pack and a live warehouse, provided the routing rule is visible to the user

Provenance, permissions, and freshness: the three things that decide adoption

A table answer is only useful if someone can check it. That means the response must carry provenance down to the cell: which document or table, which rows were included, which filters were applied, which version of the file, and when it was ingested. Without that, a wrong answer cannot be debugged and a right answer cannot be defended in a meeting. Provenance is also what makes correction possible — when a source table is fixed, the system should be able to identify which prior answers were derived from it.

Permissions are harder with tables than with prose, because tables mix sensitivity levels inside a single grid. One column may contain salary bands while the rest is job architecture; one region's rows may be restricted while the file is broadly accessible. Row- and column-level access control must therefore be enforced at retrieval or query time, against the identity of the person asking, and not applied as a post-processing filter on a generated answer. A model that has already seen restricted cells can leak them in a paraphrase.

Freshness is the third constraint. A table inside a PDF is a snapshot with a date; a table in a connected ERP is live but may lag by a batch cycle. Users need to know which they are looking at, because “revenue is 12.4 million” means something different when the number is three months old. For organizations with data residency requirements, the entire chain — parsing, indexing, retrieval and inference — has to run where the data is allowed to live, which is why on-premise deployment is a structural choice rather than a checkbox.

These three properties rarely appear in product demos, but they are the ones that determine whether a table-aware AI system survives its first quarter in production.

A readiness checklist before you trust table answers

Before evaluating any platform, assemble a test set from your own documents rather than from a vendor's sample data. Ten to twenty questions is enough if they are chosen to be adversarial: one that requires excluding a subtotal, one that depends on a footnote changing units, one that spans two documents with conflicting figures, one that requires a join across a document table and a live system, and one that a specific user should not be allowed to answer because of row-level restrictions. Run the same set against every candidate and compare not just accuracy but whether the reasoning path is visible.

Then walk through the operational questions:

Working through this list takes a few days and saves months. It also produces something valuable in its own right — a documented inventory of which tables are machine-readable today and which need their exports fixed at the source. That inventory is often the most reusable output of the whole exercise.

• Can the system show the exact rows and filters behind a numeric answer?
• Does it flag subtotal and total rows, or count them as data?
• Are units, currencies and sign conventions normalized and recorded?
• Is access control enforced before generation, at row and column level?
• Does the answer state whether the source is a snapshot document or a live connected system?
• Can it route a question to a query engine when a governed source exists, instead of guessing from text?
• Where does parsing, indexing and inference run, and does that match your data residency rules?

The real test is reproducibility, not fluency

A model that describes a table beautifully has demonstrated that it can read. A model that returns the same correct number tomorrow, for a different user with different permissions, and can show which cells it used, has demonstrated that it understands structured data in a way the business can rely on. The gap between those two capabilities is filled almost entirely by engineering that happens before generation: grid recovery, header inference, unit normalization, typed records, governed metric definitions, and enforced access control.

This is also why table understanding should not be evaluated as a model feature. It is a pipeline property. The same underlying language model can produce an auditable answer or a plausible hallucination depending on whether the rows it received were correctly bound to their headers, whether the aggregation ran in a database or in the model's imagination, and whether the retrieval layer respected the asker's permissions.

If you are scoping this now, start with the tables that already cause arguments — the ones where two teams quote different numbers from the same file. Fixing structure there produces immediate, visible value, and it gives you a realistic benchmark for everything that follows. You can see how Pilardin handles document tables and live structured sources in the Matrix modules at /matrix, or review deployment and licensing options at /pricing before committing to a pilot.

Turning the audit questions into a pilot design

The audit questions are most useful when they become acceptance criteria for a narrowly scoped pilot. Choose one or two recurring reporting workflows, such as month-end variance review or pipeline reconciliation, and define the exact questions the system must answer with row-level citations. Keep the pilot narrow enough that subject-matter experts can verify every answer during the first few weeks.

For each workflow, separate questions that should be answered from governed sources from questions that require parsing a document table. That split lets you test both paths without letting a document fallback hide gaps in the governed data layer. It also makes performance and cost visible per path rather than as a single blended average.

Record the baseline before the pilot: who currently produces the number, how long it takes, which file version they use, and how often two teams disagree. Reproducibility only becomes meaningful when it is compared with the status quo, and the baseline gives you a defensible way to decide whether to expand, redesign, or stop. Without that baseline, a pilot can look successful simply because no one measured the old process.

• Pilot exit criterion: every numeric answer includes source, filters, and the cells or rows used.
• Pilot exit criterion: repeated identical questions return identical results for the same permissions and data version.
• Pilot exit criterion: subtotal, total, and derived rows are excluded from base data unless explicitly requested.
• Pilot exit criterion: access checks are enforced before generation and logged for audit review.

Operating the pipeline after launch

Once a pilot proves the answer contract, the operational focus shifts from model selection to keeping the structure trustworthy. Source exports change, headers drift, units get relabeled, and new subtotal conventions appear, so the parsing and indexing layer needs monitoring just like any other production data pipeline. Treat schema drift as an incident category, not as a quiet degradation in answer quality.

Answers should carry a data-version identifier and a rendered query or extraction trace that can be replayed. When a source is a snapshot document, say so explicitly; when it is a live connected system, state the query time and time zone. This metadata is what lets an auditor distinguish a stale but correctly labeled answer from a confidently wrong one.

Feedback from users should be captured at the level of the answer, not just as a thumbs-up or thumbs-down. A useful correction points to the row, header, unit, or permission that was mishandled, which turns user review into targeted training and validation data. Over time, this loop is what keeps the system from silently regressing as the underlying tables evolve.

• Monitor grid recovery failures, header confidence, unit-normalization conflicts, and unresolved totals.
• Track answer reproducibility by workflow, user role, and source type rather than only overall accuracy.
• Review access-control logs for attempts to query columns or rows outside a user's entitlement.
• Re-run a fixed benchmark set whenever a source export format, metric definition, or model version changes.

Closing

AI understands tables best when the system treats structure as something to recover, verify, and govern before generation begins. The model is only one component; the durable value comes from correct header binding, normalized units, typed records, explicit source metadata, and access control that is enforced upstream. Teams that invest in those foundations get answers they can audit, reproduce, and defend, which is a far more useful outcome than fluent table summaries.

FAQ

Can AI answer questions about tables without sending the full table to the model?

Yes. A common design parses and indexes the table outside the model, then sends only the relevant rows, column definitions, and permitted context for the specific question. For governed sources, the aggregation should run in a query engine and return a compact result with a trace rather than asking the model to do arithmetic over raw text.

How should subtotal and total rows be handled?

They should be detected and marked as derived rows, not treated as ordinary data. The system should exclude them from base aggregations unless the user explicitly asks for reported totals, and it should state which convention it used. This prevents double counting and makes discrepancies between calculated and reported figures explainable.

What does an auditable answer need to include?

At minimum, it should show the source table or document, the filters and time window, the columns and rows used, and any unit or currency normalization applied. It should also record the data version or query time and indicate whether the source was a snapshot or a live system. Those details let a reviewer replay the answer instead of trusting the prose.

Is a larger language model enough to improve table understanding?

No. A larger model can improve interpretation and explanation, but it cannot compensate for broken grid recovery, ambiguous headers, unnormalized units, or missing access controls. Table understanding is a pipeline property, so evaluation should include reproducibility, citation quality, permission enforcement, and handling of derived rows alongside answer accuracy.

Where should a team start?

Start with tables that already cause disagreements or repeated manual reconciliation, because they provide a clear baseline and visible value. Build the answer contract and audit trail for that narrow workflow first, then expand only after the pipeline can reproduce results across users and data versions.