CORE: Robust Out-of-Distribution Detection via Confidence and Orthogonal Residual Scoring
arXiv:2603.18290v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitation: logit-based methods see only the classifier's confidence signal, while feature-based methods attempt to measure membership in the training distribution but do so in the full feature space where confidence and membership are entangled, inheriting architecture-sensitive failure modes. We observe that penultimate features naturally decompose into two orthogonal subspaces: a classifier-aligned component encoding confidence, and a residual the classifier discards. We discover that this residual carries a class-specific directional signature for in-distribution data -- a membership signal invisible to logit-based methods and entangled with noise
arXiv:2603.18290v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection is essential for deploying deep learning models reliably, yet no single method performs consistently across architectures and datasets -- a scorer that leads on one benchmark often falters on another. We attribute this inconsistency to a shared structural limitation: logit-based methods see only the classifier's confidence signal, while feature-based methods attempt to measure membership in the training distribution but do so in the full feature space where confidence and membership are entangled, inheriting architecture-sensitive failure modes. We observe that penultimate features naturally decompose into two orthogonal subspaces: a classifier-aligned component encoding confidence, and a residual the classifier discards. We discover that this residual carries a class-specific directional signature for in-distribution data -- a membership signal invisible to logit-based methods and entangled with noise in feature-based methods. We propose CORE (COnfidence + REsidual), which disentangles the two signals by scoring each subspace independently and combines them via normalized summation. Because the two signals are orthogonal by construction, their failure modes are approximately independent, producing robust detection where either view alone is unreliable. CORE achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations, ranking first in three of five settings and achieving the highest grand average AUROC with negligible computational overhead.
Executive Summary
This article proposes a novel approach to out-of-distribution (OOD) detection in deep learning models, known as CORE (COnfidence + REsidual). CORE disentangles confidence and membership signals in the feature space by scoring two orthogonal subspaces independently and combining them via normalized summation. The approach achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations, with negligible computational overhead. The authors address a long-standing issue in OOD detection, where existing methods often fail to generalize across architectures and datasets. CORE's robustness stems from its ability to capture class-specific directional signatures in the residual subspace, which are invisible to logit-based methods and entangled with noise in feature-based methods.
Key Points
- ▸ CORE disentangles confidence and membership signals in the feature space
- ▸ The approach scores two orthogonal subspaces independently and combines them via normalized summation
- ▸ CORE achieves competitive or state-of-the-art performance across five architectures and five benchmark configurations
Merits
Robust OOD detection
CORE's ability to capture class-specific directional signatures in the residual subspace leads to robust OOD detection, which is not achieved by existing methods.
Generalizability
CORE's performance is not architecture-sensitive, unlike existing methods, making it a more generalizable approach to OOD detection.
Computational efficiency
CORE has negligible computational overhead, making it a practical solution for real-world applications.
Demerits
Complexity
CORE's approach requires disentangling confidence and membership signals in the feature space, which may add complexity to the model and require additional computational resources.
Dependence on data quality
CORE's performance may be sensitive to the quality of the training data, which can affect the effectiveness of the class-specific directional signatures in the residual subspace.
Expert Commentary
The article presents a novel and innovative approach to OOD detection, which addresses a long-standing issue in the field. The authors' ability to disentangle confidence and membership signals in the feature space is a significant contribution to the research. However, the complexity and dependence on data quality of the CORE approach may limit its practical adoption. Nevertheless, the implications of CORE are far-reaching, and its potential to improve the reliability and robustness of deep learning models makes it an exciting area of research.
Recommendations
- ✓ Future research should focus on extending CORE to other anomaly detection problems and exploring its application in real-world domains.
- ✓ The authors should investigate the sensitivity of CORE to the quality of the training data and develop strategies to mitigate its dependence on data quality.
