Academic

Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution

arXiv:2603.11445v1 Announce Type: new Abstract: We present Verified Multi-Agent Orchestration (VMAO), a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop. Given a complex query, our system decomposes it into a directed acyclic graph (DAG) of sub-questions, executes them through domain-specific agents in parallel, verifies result completeness via LLM-based evaluation, and adaptively replans to address gaps. The key contributions are: (1) dependency-aware parallel execution over a DAG of sub-questions with automatic context propagation, (2) verification-driven adaptive replanning that uses an LLM-based verifier as an orchestration-level coordination signal, and (3) configurable stop conditions that balance answer quality against resource usage. On 25 expert-curated market research queries, VMAO improves answer completeness from 3.1 to 4.2 and source quality from 2.6 to 4.1 (1-5 scale) compared to a single-agent baseline, demonstrating t

X
Xing Zhang, Yanwei Cui, Guanghui Wang, Qucy Wei Qiu, Ziyuan Li, Fangwei Han, Yajing Huang, Hengzhi Qiu, Bin Zhu, Peiyang He · · 1 min read · 14 views
#cs.AI #cs.MA

arXiv:2603.11445v1 Announce Type: new Abstract: We present Verified Multi-Agent Orchestration (VMAO), a framework that coordinates specialized LLM-based agents through a verification-driven iterative loop. Given a complex query, our system decomposes it into a directed acyclic graph (DAG) of sub-questions, executes them through domain-specific agents in parallel, verifies result completeness via LLM-based evaluation, and adaptively replans to address gaps. The key contributions are: (1) dependency-aware parallel execution over a DAG of sub-questions with automatic context propagation, (2) verification-driven adaptive replanning that uses an LLM-based verifier as an orchestration-level coordination signal, and (3) configurable stop conditions that balance answer quality against resource usage. On 25 expert-curated market research queries, VMAO improves answer completeness from 3.1 to 4.2 and source quality from 2.6 to 4.1 (1-5 scale) compared to a single-agent baseline, demonstrating that orchestration-level verification is an effective mechanism for multi-agent quality assurance.

Executive Summary

This article introduces Verified Multi-Agent Orchestration (VMAO), a novel framework for complex query resolution that leverages specialized LLM-based agents and a verification-driven iterative loop. By decomposing queries into a DAG of sub-questions and executing them in parallel, VMAO improves answer completeness and source quality compared to a single-agent baseline. The framework's key features, including dependency-aware parallel execution and verification-driven adaptive replanning, demonstrate the effectiveness of orchestration-level verification for multi-agent quality assurance. The article's empirical evaluation on 25 expert-curated market research queries showcases VMAO's potential in real-world applications.

Key Points

  • ▸ VMAO decomposes complex queries into a DAG of sub-questions for parallel execution
  • ▸ The framework employs verification-driven adaptive replanning for quality assurance
  • ▸ VMAO demonstrates improved answer completeness and source quality compared to a single-agent baseline

Merits

Scalability

VMAO's parallel execution and adaptive replanning features enable efficient handling of complex queries, making it a scalable solution for large-scale applications.

Flexibility

The framework's configurable stop conditions allow users to balance answer quality against resource usage, making it adaptable to diverse real-world scenarios.

Demerits

Complexity

VMAO's reliance on LLM-based agents and verification-driven adaptive replanning may introduce additional complexity, requiring significant computational resources and expertise.

Dependence on LLMs

The framework's effectiveness is contingent upon the quality and availability of LLMs, which may be subject to limitations and biases.

Expert Commentary

While VMAO represents a significant advancement in complex query resolution, its practical implementation and adoption will depend on addressing the complexity and dependence on LLMs associated with the framework. Furthermore, as VMAO has the potential to impact various industries and policy domains, its development and deployment should be accompanied by careful consideration of its societal implications and potential biases.

Recommendations

  • ✓ Future research should focus on optimizing VMAO's computational efficiency and scalability to make it more suitable for large-scale applications.
  • ✓ The development of more robust and transparent LLMs is essential for ensuring the reliability and fairness of VMAO's results.

Sources

Related Articles