通过几何约束提升不平衡回归的表示能力

Improve Representation for Imbalanced Regression through Geometric Constraints

March 2, 2025

作者: Zijian Dong, Yilei Wu, Chongyao Chen, Yingtian Zou, Yichi Zhang, Juan Helen Zhou

cs.AI

摘要

在表征学习中，均匀性指的是潜在空间（即单位超球面）内特征的均匀分布。先前的研究表明，提升均匀性有助于学习那些代表性不足的类别。然而，大多数研究集中于分类问题；对于不平衡回归的表征空间仍待探索。基于分类的方法并不适用于回归任务，因为它们将特征聚类为离散的组别，而忽视了回归所必需的连续性和有序性。从几何视角出发，我们独辟蹊径，通过两种关键损失函数——包络损失和同质性损失，确保不平衡回归在潜在空间中的均匀性。包络损失促使诱导轨迹均匀覆盖超球面，而同质性损失则保证平滑性，使表征以一致的间隔均匀分布。我们的方法通过一个代理驱动的表征学习（SRL）框架，将这些几何原理融入数据表征之中。针对现实世界回归及算子学习任务的实验，凸显了均匀性在不平衡回归中的重要性，并验证了我们基于几何的损失函数的有效性。

English

In representation learning, uniformity refers to the uniform feature distribution in the latent space (i.e., unit hypersphere). Previous work has shown that improving uniformity contributes to the learning of under-represented classes. However, most of the previous work focused on classification; the representation space of imbalanced regression remains unexplored. Classification-based methods are not suitable for regression tasks because they cluster features into distinct groups without considering the continuous and ordered nature essential for regression. In a geometric aspect, we uniquely focus on ensuring uniformity in the latent space for imbalanced regression through two key losses: enveloping and homogeneity. The enveloping loss encourages the induced trace to uniformly occupy the surface of a hypersphere, while the homogeneity loss ensures smoothness, with representations evenly spaced at consistent intervals. Our method integrates these geometric principles into the data representations via a Surrogate-driven Representation Learning (SRL) framework. Experiments with real-world regression and operator learning tasks highlight the importance of uniformity in imbalanced regression and validate the efficacy of our geometry-based loss functions.