K-Means as a Radial Basis function Network: a Variational and Gradient-based Equivalence
arXiv:2603.04625v1 Announce Type: new Abstract: This work establishes a rigorous variational and gradient-based equivalence between the classical K-Means algorithm and differentiable Radial Basis Function (RBF) neural networks with smooth responsibilities. By reparameterizing the K-Means objective and embedding its distortion functional into a smooth weighted loss, we prove that the RBF objective $\Gamma$-converges to the K-Means solution as the temperature parameter $\sigma$ vanishes. We further demonstrate that the gradient-based updates of the RBF centers recover the exact K-Means centroid update rule and induce identical training trajectories in the limit. To address the numerical instability of the Softmax transformation in the low-temperature regime, we propose the integration of Entmax-1.5, which ensures stable polynomial convergence while preserving the underlying Voronoi partition structure. These results bridge the conceptual gap between discrete partitioning and continuous
arXiv:2603.04625v1 Announce Type: new Abstract: This work establishes a rigorous variational and gradient-based equivalence between the classical K-Means algorithm and differentiable Radial Basis Function (RBF) neural networks with smooth responsibilities. By reparameterizing the K-Means objective and embedding its distortion functional into a smooth weighted loss, we prove that the RBF objective $\Gamma$-converges to the K-Means solution as the temperature parameter $\sigma$ vanishes. We further demonstrate that the gradient-based updates of the RBF centers recover the exact K-Means centroid update rule and induce identical training trajectories in the limit. To address the numerical instability of the Softmax transformation in the low-temperature regime, we propose the integration of Entmax-1.5, which ensures stable polynomial convergence while preserving the underlying Voronoi partition structure. These results bridge the conceptual gap between discrete partitioning and continuous optimization, enabling K-Means to be embedded directly into deep learning architectures for the joint optimization of representations and clusters. Empirical validation across diverse synthetic geometries confirms a monotone collapse of soft RBF centroids toward K-Means fixed points, providing a unified framework for end-to-end differentiable clustering.
Executive Summary
The article establishes a rigorous variational and gradient-based equivalence between the classical K-Means algorithm and differentiable Radial Basis Function (RBF) neural networks. The authors reparameterize the K-Means objective and embed its distortion functional into a smooth weighted loss, demonstrating that the RBF objective converges to the K-Means solution as the temperature parameter vanishes. This connection enables K-Means to be embedded directly into deep learning architectures for joint optimization of representations and clusters. Empirical validation across synthetic geometries confirms a monotone collapse of soft RBF centroids toward K-Means fixed points, providing a unified framework for end-to-end differentiable clustering.
Key Points
- ▸ Establishes a variational and gradient-based equivalence between K-Means and RBF neural networks
- ▸ Reparameterizes K-Means objective and embeds distortion functional into smooth weighted loss
- ▸ Demonstrates convergence of RBF objective to K-Means solution as temperature parameter vanishes
Merits
Theoretical Contribution
The article provides a rigorous theoretical framework for understanding the connection between K-Means and RBF neural networks, bridging the gap between discrete partitioning and continuous optimization.
Empirical Validation
The authors provide empirical validation of their results across diverse synthetic geometries, demonstrating a monotone collapse of soft RBF centroids toward K-Means fixed points.
Demerits
Limited Generalizability
The results may not generalize to more complex or high-dimensional data sets, and further research is needed to explore the limitations of this framework in real-world applications.
Technical Complexity
The article assumes a high level of technical expertise in both machine learning and mathematical optimization, which may limit its accessibility to a broader audience.
Expert Commentary
The article provides a significant contribution to the field of machine learning and mathematical optimization, establishing a rigorous connection between K-Means and RBF neural networks. While the results are primarily theoretical, the empirical validation provided in the article demonstrates the practical relevance of this connection. However, further research is needed to explore the limitations of this framework in real-world applications and to develop more efficient and effective clustering algorithms. Additionally, the article's results have implications for the broader field of mathematical optimization, particularly in the context of clustering and partitioning problems.
Recommendations
- ✓ Future research should focus on exploring the limitations of this framework in real-world applications and developing more efficient and effective clustering algorithms.
- ✓ The article's results should be extended to more complex and high-dimensional data sets to demonstrate the generalizability of this framework.
