Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

===== The paper’s impossibility result is not about number of classes or supervision; it’s about the space of encoders that fit the ID data and how unconstrained they are OOD. =====

Roughly, they show (Theorem 3.2) that when dx≫dzd_x \gg d_zdx≫dz, for almost any Jacobian/Hessian you want for an encoder ggg at a point on the manifold, there exists some generator f∈Fintf \in F_{\text{int}}f∈Fint such that ggg is a valid inverse of fff on the data manifold – but its behavior off the manifold (i.e., in unseen combinations) can be arbitrarily bad. The structure that characterizes the “good” inverses GintG_{\text{int}}Gint only shows up restricted to the data manifold geometry, which you don’t know OOD. So you can’t encode the right inductive bias with architecture/regularization alone. 2512.08854v1

Your pseudo-label idea still only constrains ggg on the training points (ID images), not on the whole manifold or its OOD regions. So there are many encoders that:
* give perfect pseudo-label accuracy on all training images,
* but implement completely different—and arbitrarily bad—extensions to any new combination (OOD).

Nothing in the pseudo-label objective forces you towards the particular inverse g∈Gintg \in G_{\text{int}}g∈Gint that corresponds to the “true slots”.