Publications

2025

1. EMNLP 2025

   

   TABARD: A Novel Benchmark for Tabular Anomaly Analysis, Reasoning and Detection

   Manan Roy Choudhury, Anirudh Iyengar Kaniyar Narayana Iyengar, Shikhhar Siingh, and 2 more authors

   In EMNLP, ACL ARR (May Cycle), 2025

   Abs Bib PDF Code Website

   We study the capabilities of large language models (LLMs) in detecting fine-grained anomalies in tabular data. Specifically, we examine: (1) how well LLMs can identify diverse anomaly types—including factual, logical, temporal, and value-based errors; (2) the impact of prompt design and prompting strategies; and (3) the effect of table structure and anomaly type on detection accuracy. To this end, we introduce TABARD, a new benchmark constructed by perturbing tables from WikiTQ, FeTaQA, Spider, and BEAVER. The dataset spans multiple domains and eight anomaly categories, including paired clean and corrupted tables. We evaluate LLMs using direct, indirect, and Chain-of-Thought (CoT) prompting. Our results reveal notable limitations in standard prompting, especially for complex reasoning tasks and longer tables. To overcome these issues, we propose a unified framework combining multi-step prompting, self-verification, and constraint-based rule execution.

   @inproceedings{tabard2025,
     title = {TABARD: A Novel Benchmark for Tabular Anomaly Analysis, Reasoning and Detection},
     author = {Choudhury, Manan Roy and Kaniyar Narayana Iyengar, Anirudh Iyengar and Siingh, Shikhhar and Sugeeth, Raghavendra and Gupta, Vivek},
     booktitle = {EMNLP, ACL ARR (May Cycle)},
     year = {2025},
     url = {https://openreview.net/forum?id=8Y1G4XD3vR},
     data = {https://github.com/TABARD-emnlp-2025/TABARD-dataset},
   }