Editing Openai/693c0f4f-255c-8008-92e9-0cd44c6d6226 (section)

==== : ====

You’re exactly right that they’re assuming some things already trained:
* f^\hat ff^ (decoder/generator) is learned first, with the inductive biases that make it belong to FintF_{\text{int}}Fint.
* g^\hat gg^ is initially trained on ID via an autoencoding/VAE objective (Eq. 4.2).

Section 4 then does not focus on how to train f^\hat ff^; it assumes it already exists and is good, and studies:
# How to invert f^\hat ff^ on OOD images by: - Search: per-image gradient optimization (Eq. 4.3) initialized with g^(x)\hat g(x)g^(x). - Replay: offline training of g^\hat gg^ on synthetic OOD images generated by f^\hat ff^ (Eq. 4.4).

So conceptually:
* Non-generative baseline: just use g^(x)\hat g(x)g^(x) trained on ID and hope it generalizes.
* Generative approach (Sec. 4): - either refine g^(x)\hat g(x)g^(x) via optimization (search), - or retrain g^\hat gg^ on synthetic OOD from f^\hat ff^ (replay), - or both.

If you’d like, next we can:
* Draw pseudo-code for the search and replay procedures as if you were implementing them.
* Or unpack the assumptions on pz^p_{\hat z}pz^ and what “independent slot marginals” really mean in practice.