1: \begin{abstract}

2: Recent work has observed an intriguing ``Neural Collapse'' phenomenon in well-trained neural networks, where the last-layer representations of training samples with the same label collapse into each other. This appears to suggest that the last-layer representations are completely determined by the labels, and do not depend on the intrinsic structure of input distribution. We provide evidence that this is not a complete description, and that the apparent collapse hides important fine-grained structure in the representations.

3: Specifically, even when representations apparently collapse, the small amount of remaining variation can still faithfully and accurately captures the intrinsic structure of input distribution.

4: % within that label and is enough there is a mismatch between the inputs and the labels (e.g. when a coarser or finer labeling is provided), the learned last-layer representations often naturally exhibit a clustered structure according to the input distribution. %, and such structure can even persist all the way through the end of training when Neural Collapse has happened as driven by the labels. 

5: As an example, if we train on CIFAR-10 using only 5 coarse-grained labels (by combining two classes into one super-class) until convergence, we can reconstruct the original 10-class labels from the learned representations via unsupervised clustering. The reconstructed labels achieve $93\%$ accuracy on the CIFAR-10 test set, nearly matching the normal CIFAR-10 accuracy for the same architecture.

6: We also provide an initial theoretical result showing the fine-grained representation structure in a simplified synthetic setting.

7: Our results show concretely how the structure of input data can play a significant role in determining the fine-grained structure of neural representations, going beyond what Neural Collapse predicts.

8: \end{abstract}

9: