Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

==== The second strategy is to fix the runtime cost by doing the hard work offline. ====

Idea:
# Your decoder f^\hat ff^ identifies the ground-truth generator fff up to slot-wise transforms.
# That means OOD images are generated by re-combining slots from ID in new ways.

So they say: “If I can sample slot combinations z^\hat zz^ from a distribution pz^p_{\hat z}pz^ whose marginals are independent per slot, and decode them via f^\hat ff^, I can generate synthetic OOD images f^(z^)\hat f(\hat z)f^(z^).”

Concretely:
* Choose pz^p_{\hat z}pz^ such that pz^(z)=∏k=1Kpk(zk)p_{\hat z}(z) = \prod_{k=1}^K p_k(z_k)pz^(z)=k=1∏Kpk(zk) where each pkp_kpk is estimated from ID slot distributions, but they’re independent across slots.
* Sample z^∼pz^\hat z \sim p_{\hat z}z^∼pz^, decode: x^=f^(z^).\hat x = \hat f(\hat z).x^=f^(z^).
* Because slots are sampled independently, many of these (z^,x^)(\hat z, \hat x)(z^,x^) pairs correspond to unseen combinations ⇒ they lie in XOODX_{\text{OOD}}XOOD.

Now you have unlimited synthetic OOD data with ground-truth latents!

Then you can train a new encoder g^\hat gg^ to invert f^\hat ff^ on this synthetic data:

min⁡g^  Ez^∼pz^ ∥z^−g^(f^(z^))∥2(4.4)\min_{\hat g} \;\mathbb E_{\hat z \sim p_{\hat z}} \,\big\|\hat z - \hat g(\hat f(\hat z))\big\|^2 \tag{4.4}g^minEz^∼pz^z^−g^(f^(z^))2(4.4)
So we’re training on pairs:
* input image: f^(z^)\hat f(\hat z)f^(z^) (possibly OOD combination),
* target latent: z^\hat zz^.

This is just a self-supervised autoencoder on synthetic data, but crucially:
* The images explore combinatorial variations (thanks to independent slot sampling).
* So g^\hat gg^ is trained to correctly invert f^\hat ff^ across a much larger region of XXX, including OOD.

At test time:

: 

That’s what Fig. 4 (right) illustrates:
decode synthetic OOD images with f^\hat ff^ → train g^\hat gg^ on them → deploy g^\hat gg^ at test time.

Pros:
* After offline training, inference is fast (single forward pass).
* The encoder has seen many OOD combinations via generated images, so it can generalize better.

Cons:
* Depends on how well f^\hat ff^ approximates the true generative process.
* If f^\hat ff^ can’t generate realistic or diverse OOD combinations, replay is limited.