Editing Openai/69191dd5-6360-800d-b47b-7ebcc8b69dc8 (section)

===== 3.1 Direct ratio =====

Using:
* 400B run cost: $92.5–$185M (GPU only)
* 8B hybrid cost: $4.25–$5.1k

Two useful edges:
# Optimistic for the 400B (cheap H100) vs expensive hybrids - Numerator: $92.5M - Denominator: $5.1k → ≈18,000 8B hybrids\approx 18{,}000\ \text{8B hybrids}≈18,000 8B hybrids
# Pessimistic for the 400B (expensive H100) vs cheap hybrids - Numerator: $185M - Denominator: $4.25k → ≈43,000 8B hybrids\approx 43{,}000\ \text{8B hybrids}≈43,000 8B hybrids

So the cost of ''one'' Llama-3-class 400B pre-train roughly buys you:

: ~18k–40k post-trained 8B Zebra-Llama-style models.

Even if my hybrid GPU-hrs estimate is off by a big factor (say 3×), you’re still in the many thousands of hybrids per 400B run.