Open Internet by MindsNet

Evaluating Non-Monotonic Loss in Self-Supervised Representation Learning

The author struggles to evaluate the effectiveness of self-supervised representation learning methods with non-monotonic loss functions. They question how to select hyperparameters and architectures for methods like BYOL, JEPA, and data2vec when the loss is not monotonically decreasing. The author notes that existing methods like RankMe may not be effective in such cases.

Computing & Technology, Computer Science, Machine Learning