AGRI-Fidelity: Evaluating the Reliability of Listenable Explanations for Poultry Disease Detection
arXiv:2603.18247v1 Announce Type: new Abstract: Existing XAI metrics measure faithfulness for a single model, ignoring model multiplicity where near-optimal classifiers rely on different or spurious acoustic cues. In noisy farm environments, stationary artifacts such as ventilation noise can produce explanations that are faithful yet unreliable, as masking-based metrics fail to penalize redundant shortcuts. We propose AGRI-Fidelity, a reliability-oriented evaluation framework for listenable explanations in poultry disease detection without spatial ground truth. The method combines cross-model consensus with cyclic temporal permutation to construct null distributions and compute a False Discovery Rate (FDR), suppressing stationary artifacts while preserving time-localized bioacoustic markers. Across real and controlled datasets, AGRI-Fidelity effectively provides reliability-aware discrimination for all data points versus masking-based metrics.
arXiv:2603.18247v1 Announce Type: new Abstract: Existing XAI metrics measure faithfulness for a single model, ignoring model multiplicity where near-optimal classifiers rely on different or spurious acoustic cues. In noisy farm environments, stationary artifacts such as ventilation noise can produce explanations that are faithful yet unreliable, as masking-based metrics fail to penalize redundant shortcuts. We propose AGRI-Fidelity, a reliability-oriented evaluation framework for listenable explanations in poultry disease detection without spatial ground truth. The method combines cross-model consensus with cyclic temporal permutation to construct null distributions and compute a False Discovery Rate (FDR), suppressing stationary artifacts while preserving time-localized bioacoustic markers. Across real and controlled datasets, AGRI-Fidelity effectively provides reliability-aware discrimination for all data points versus masking-based metrics.
Executive Summary
This article introduces AGRI-Fidelity, a novel evaluation framework for evaluating the reliability of listenable explanations in poultry disease detection. AGRI-Fidelity addresses the limitations of existing XAI metrics by incorporating cross-model consensus and cyclic temporal permutation to suppress stationary artifacts. The proposed method computes a False Discovery Rate (FDR) to provide reliability-aware discrimination. The authors demonstrate the effectiveness of AGRI-Fidelity on real and controlled datasets, outperforming masking-based metrics. The framework has significant implications for the reliability and trustworthiness of listenable explanations in machine learning-based disease detection systems.
Key Points
- ▸ AGRI-Fidelity addresses the limitations of existing XAI metrics by incorporating cross-model consensus and cyclic temporal permutation
- ▸ The proposed framework computes a False Discovery Rate (FDR) to provide reliability-aware discrimination
- ▸ AGRI-Fidelity effectively suppresses stationary artifacts and preserves time-localized bioacoustic markers
Merits
Strength in Addressing Limitations of Existing XAI Metrics
The authors effectively identify the limitations of existing XAI metrics and propose a novel framework that addresses these limitations, providing a more reliable evaluation of listenable explanations.
Improved Reliability and Trustworthiness
AGRI-Fidelity provides reliability-aware discrimination, suppressing stationary artifacts and preserving time-localized bioacoustic markers, which improves the reliability and trustworthiness of listenable explanations.
Demerits
Assumes Availability of Multiple Models
The proposed framework assumes the availability of multiple models, which may not be feasible in all scenarios, limiting its applicability.
Limited Generalizability to Other Domains
AGRI-Fidelity is specifically designed for poultry disease detection and may not be directly applicable to other domains, requiring additional modifications or adaptations.
Expert Commentary
The article makes a significant contribution to the field of XAI by proposing a novel framework for evaluating the reliability of listenable explanations. AGRI-Fidelity addresses a critical limitation of existing XAI metrics and provides a more reliable evaluation of listenable explanations. The framework has significant implications for the reliability and trustworthiness of machine learning-based disease detection systems, particularly in agricultural and veterinary applications. However, the proposed framework assumes the availability of multiple models, which may not be feasible in all scenarios, and has limited generalizability to other domains. Nevertheless, the article provides a valuable insight into the limitations of existing XAI metrics and proposes a novel framework that addresses these limitations.
Recommendations
- ✓ Future research should focus on adapting the proposed framework to other domains and addressing the assumptions of multiple models.
- ✓ The proposed framework should be further evaluated and validated in real-world scenarios to demonstrate its effectiveness and reliability.
