TableMoE

TableMoE is a powerful multimodal large language model built upon a novel Mixture-of-Connector-Experts (MoCE) architecture. It is designed for robust table-centric understanding and visual reasoning across diverse formats, including text, tables, charts, and images. Trained on extensive tabular corpora, TableMoE supports advanced tasks such as table editing, highlighting, replotting, transformation, and symbolic CoT-driven or PoT-driven reasoning. Its neuro-symbolic routing and expert composition enable strong conversational capabilities grounded in structured data.

Visit the project: https://ai-agi.github.io/TableMoE/

Keywords: TableMoE, Neuro-Symbolic Routing, Table Reasoning, MoE, Multimodal, Table-Centric QA, Symbolic Graph Planning, Role Classification, Structural Alignment, LLMs, LMMs, LVLMs, Vision-Language Models, Structured Reasoning

News

• 2025/06/28: 🚀🎉 Our paper TableMoE: Neuro-Symbolic Routing for Structured Expert Reasoning in Multimodal Table Understanding is now officially released on arXiv! 📄 Dive into it here: https://arxiv.org/abs/2506.21393.
• 2025/06/13: 📌 We release WildStrcut Series WMMTabDialog benchmark, see https://github.com/ai-agi/WMMTabDialog or https://huggingface.co/datasets/darkme-ai/WMMTabDialog. WMMTabDialog is a high-quality multimodal benchmark for Chinese tax and finance, featuring multi-turn dialogues and multi-table images with WildStruct traits, e.g., incomplete structures, complex nested cells, symbolic expressions, and visual noise. It targets visually structured reasoning in Multimodal Large Language Models (MLLMs). WMMTabDialog is a held-out zero-shot benchmark, excluded from all training to ensure contamination-free, unbiased evaluation.
• 2025/06/02: 🌍 We release WMMFinanceMath, see here, a visually grounded version of FinanceMath with markdown tables rendered into WildStruct-style images featuring real-world noise like blur, skew, and watermarks. 🎉
• 2025/05/05: 🎉🎉🎉 TableMoE extends the LLaVA-NeXT framework (https://github.com/LLaVA-VL/LLaVA-NeXT) by implementing new functionalities that enable encoding multiple images within multi-round conversations in a single sample per batch—capabilities not supported in the original implementation!
• 2025/05/01: 🚀 We release TableMoE!

🔥 Key Features

• 💡 Neuro-Symbolic Routing with Role and Structural Graph Alignment
• 📊 Support for WMMFinanceMath, ChartQA, WMMTATQA, WMMTabDialog, WMMFinQA datasets
• 🧠 Compatibility with Llama 3.1, Qwen2.5-VL Transformer-based LLM backbones
• 📎 Built-in evaluation and visualization scripts

🚀 Quick Start

git clone https://github.com/ai-agi/TableMoE.git
cd TableMoE
bash scripts/alignment_table2code.sh