A Schema-aware Harness for Tabular Reasoning with Language Models

Abstract

Language Models (LMs) are increasingly deployed as interactive data exploration assistants, enabling users to pose natural language questions over structured datasets without requiring query language expertise. Yet as these systems grow more capable, a critical gap emerges: while LMs can generate fluent, confident responses, their reasoning over structured tabular data remains brittle when schema-level signals (field semantics, aggregation rules, relational structure) are absent or inconsistently surfaced. This brittleness is especially consequential in multi-step reasoning pipelines, where a schema error at one stage can propagate silently into downstream outputs. We address this challenge in the context of Recursive Language Model (RLM) systems, where a root-model decomposes complex queries and delegates linguistically irreducible sub-tasks to sub-models. Using IPEDS, a large-scale higher education dataset that exceeds context window limits and requires programmatic data access, we investigate whether schema-aware harnesses improve reasoning reliability. Specifically, we introduce structured schema signals at the point of use and propagate them into sub-model calls, targeting failure modes where models default to assumption over verification. We systematically evaluate three levels of schema richness across three tiers of query complexity. Our results show that incorporating schema-aware signals significantly improves reasoning reliability, with schema propagation emerging as a stronger driver of performance than increased recursion depth. These findings highlight the importance of explicitly surfacing and maintaining structured context in LM-based reasoning systems over tabular data.