Open Internet by MindsNet

Overcoming Uniform and Trivial Collapse in DINOv3-style SSL for 3D Volumes

The author is struggling with porting DINOv3 to 3D volumes, encountering a bind between uniform collapse with centering and trivial collapse without. This issue arises at small-batch and low-diversity-data regimes, causing the model to fail in training.

Computing & Technology, Computer Science, Machine Learning