Databricks officially announced the release of OfficeQA Pro V2 on August 6, 2026 through its official blog, marking a significant milestone in measuring how well AI agents reason over real enterprise documents. The announcement was made by the Databricks AI Research Team, positioning the new benchmark as a test of whether AI systems can generalize grounded reasoning to unfamiliar, enterprise-style document collections. The release matters because it addresses a fundamental open question in the industry: whether recent progress in document retrieval, parsing, and analytical reasoning reflects genuine advances in grounded reasoning or merely memorization of a single corpus. The benchmark also debuts as the basis for the inaugural Databricks Grounded Reasoning Cup, a live competition backed by OpenAI, Anthropic, and Google DeepMind.
The core advancement of OfficeQA Pro V2 lies in its entirely new corpus and its synthetic data-driven construction, a clear departure from the highly manual process behind the original OfficeQA benchmark. Instead of human annotators spending countless hours hand-crafting questions, Databricks built the new benchmark with asynth, its internal synthetic data-generation library, enabling rapid and scalable creation of diverse, verified questions. The resulting dataset contains 90 questions grounded in roughly 120,000 pages from the U.S. Federal Accounts of Receipts and Expenditures, a dataset the U.S. Treasury released as a cohesive collection for the first time alongside the nation’s 250th anniversary. The purpose is to preserve the same enterprise-relevant skills the original benchmark measured: parsing complex documents, retrieving the right evidence, and performing analytical reasoning grounded in that information.
Official details reveal a benchmark that remains genuinely difficult for today’s most advanced systems. Out-of-the-box frontier agents using Claude Code or Codex with the latest frontier models averaged only 26.0 percent accuracy, while agents built for the Grounded Reasoning Cup averaged 41.1 percent, with the winning team reaching 63.3 percent. Databricks’ own Genie agent, which uses its ai_parse tool to pre-parse the corpus, improved accuracy by an average of 24.0 percentage points across matched models, reaching up to 60 percent, demonstrating that a well-optimized harness can unlock substantial gains from existing models. Each question passes through multiple quality gates, including independent solver agents and manual final review, ensuring answers are verifiably correct. Persistent failure modes, such as parsing fidelity errors and misinterpretation of historical accounting conventions, show that grounded reasoning is far from solved.
The release carries meaningful implications for both practitioners and the wider AI industry. Enterprise users gain a rigorous, freely available tool for evaluating whether their agents can handle unfamiliar, messy document collections, while developers obtain clear evidence that agent harness design can deliver relative accuracy improvements of up to 92 percent without sacrificing efficiency. For the industry, the benchmark establishes that document-grounded reasoning remains an open frontier, likely steering further investment toward better parsing, retrieval, and reasoning pipelines rather than raw model scale alone. With the benchmark now publicly released on Hugging Face, Databricks expects broader adoption in agent development workflows, and future iterations may expand to additional enterprise corpora beyond government financial records.
Source: Databricks