DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning
arXiv:2604.01740v1 Announce Type: new Abstract: A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment. Most architectures invoke an external clustering step, typically k-means, to produce pseudo-labels that guide training, preventing the backbone from directly optimising for cluster quality. This paper introduces Deep Dual Competitive Learning (DDCL), the first fully differentiable end-to-end framework for unsupervised prototype-based representation learning. The core contribution is architectural: the external k-means is replaced by an internal Dual Competitive Layer (DCL) that generates prototypes as native differentiable outputs of the network. This single inversion makes the complete pipeline, from backbone feature extraction through prototype generation to soft cluster assignment, trainable by backpropagation through a single unified loss, with no Lloyd iterations, no pseudo-label discretisation, and no external clus
arXiv:2604.01740v1 Announce Type: new Abstract: A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment. Most architectures invoke an external clustering step, typically k-means, to produce pseudo-labels that guide training, preventing the backbone from directly optimising for cluster quality. This paper introduces Deep Dual Competitive Learning (DDCL), the first fully differentiable end-to-end framework for unsupervised prototype-based representation learning. The core contribution is architectural: the external k-means is replaced by an internal Dual Competitive Layer (DCL) that generates prototypes as native differentiable outputs of the network. This single inversion makes the complete pipeline, from backbone feature extraction through prototype generation to soft cluster assignment, trainable by backpropagation through a single unified loss, with no Lloyd iterations, no pseudo-label discretisation, and no external clustering step. To ground the framework theoretically, the paper derives an exact algebraic decomposition of the soft quantisation loss into a simplex-constrained reconstruction error and a non-negative weighted prototype variance term. This identity reveals a self-regulating mechanism built into the loss geometry: the gradient of the variance term acts as an implicit separation force that resists prototype collapse without any auxiliary objective, and leads to a global Lyapunov stability theorem for the reduced frozen-encoder system. Six blocks of controlled experiments validate each structural prediction. The decomposition identity holds with zero violations across more than one hundred thousand training epochs; the negative feedback cycle is confirmed with Pearson -0.98; with a jointly trained backbone, DDCL outperforms its non-differentiable ablation by 65% in clustering accuracy and DeepCluster end-to-end by 122%.
Executive Summary
This article introduces Deep Dual Competitive Learning (DDCL), a novel, fully differentiable end-to-end framework for unsupervised prototype-based representation learning. DDCL addresses a persistent structural weakness in deep clustering by integrating feature learning and cluster assignment. The framework replaces the external k-means clustering step with an internal Dual Competitive Layer (DCL) that generates prototypes as native differentiable outputs. This innovation enables the complete pipeline to be trainable by backpropagation through a single unified loss. Theoretical grounding is provided through an exact algebraic decomposition of the soft quantisation loss, revealing a self-regulating mechanism and global Lyapunov stability. Experimental results demonstrate significant improvements in clustering accuracy over existing approaches.
Key Points
- ▸ DDCL is a fully differentiable end-to-end framework for unsupervised prototype-based representation learning.
- ▸ The framework integrates feature learning and cluster assignment, addressing a persistent structural weakness in deep clustering.
- ▸ DDCL replaces the external k-means clustering step with an internal Dual Competitive Layer (DCL).
Merits
Strength in Architecture
The integration of feature learning and cluster assignment in a single framework enables direct optimisation for cluster quality, eliminating the need for external clustering steps.
Theoretical Grounding
The exact algebraic decomposition of the soft quantisation loss provides a sound theoretical foundation for the framework, revealing a self-regulating mechanism and global Lyapunov stability.
Demerits
Limited Experimental Scope
The experimental results are limited to six blocks of controlled experiments, which may not fully capture the framework's performance in diverse real-world scenarios.
Potential Overfitting
The framework's success relies heavily on the backbone feature extractor, which may lead to overfitting if not carefully regularised.
Expert Commentary
The introduction of DDCL marks a significant advancement in the field of deep clustering and unsupervised representation learning. By integrating feature learning and cluster assignment in a single framework, DDCL addresses a long-standing challenge in the field. The theoretical grounding provided through the exact algebraic decomposition of the soft quantisation loss adds credibility to the framework's architecture. While the experimental results are promising, further research is needed to fully explore the framework's potential and limitations. As a novel contribution, DDCL has the potential to impact various applications in computer vision and natural language processing.
Recommendations
- ✓ Further experimental evaluation of DDCL in diverse real-world scenarios is necessary to fully assess its performance and potential.
- ✓ Investigating the framework's sensitivity to hyperparameters and potential overfitting through careful regularisation is crucial for its practical adoption.
Sources
Original: arXiv - cs.LG
