Editing Openai/694ab010-3e7c-8000-b0e8-b98c22b7a063 (section)

=== ## ===

What you used before:

<syntaxhighlight>One Docker container
 ├── Streamlit UI
 ├── FastAPI backend
 └── OpenAI API key inside the container

</syntaxhighlight>

This works. Many people start here.
But it creates hidden problems as soon as you move from “demo” to “real system”.

AWS best practices are about reducing blast radius, separating concerns, and using the right AWS service for the right traffic.

Let’s go point by point.

==== ### ====

A secret is something that must never reach:
* the browser
* JavaScript
* frontend logs
* UI containers that users indirectly interact with

===== If Streamlit and FastAPI are in the same container: =====
* The OpenAI key is available to both
* The UI process can read it (even if you don’t intend it)
* A bug, debug print, exception, or misconfiguration can leak it

Even worse:
* Streamlit executes Python code dynamically
* Any vulnerability in UI code could expose environment variables

===== <syntaxhighlight>Frontend (UI) =====
  ❌ No secrets
  ❌ No credentials
  ❌ No cloud permissions

Backend (API)
  ✅ Has secrets
  ✅ Talks to OpenAI
  ✅ Protected behind API Gateway

</syntaxhighlight>

By physically separating containers, AWS guarantees:
* The UI container literally does not have access to the secret
* Even if the UI is compromised, the attacker still cannot call OpenAI

This is called least privilege and blast-radius reduction.

: 

==== ### ====

API Gateway is optimized for:
* Short-lived HTTP API calls
* JSON request/response
* Authentication
* Rate limiting
* Logging
* WAF integration

This is perfect for:

<syntaxhighlight>POST /api/chat
GET  /api/health

</syntaxhighlight>

===== Streamlit is not just an API. =====

Streamlit:
* Opens multiple internal endpoints
* Uses background refreshes
* Sends many small requests
* Sometimes holds connections open
* Assumes stable headers and paths

API Gateway:
* Rewrites paths
* Enforces timeouts
* Is optimized for stateless request/response

This mismatch causes:
* Broken UI updates
* Random freezes
* Partial page loads
* Hard-to-debug behavior

===== <syntaxhighlight>API Gateway → FastAPI =====
ALB → Streamlit

</syntaxhighlight>

Each service gets traffic it is designed to handle.

: 

==== ### ====

Application Load Balancer:
* For websites
* Handles large responses
* Stable HTTP behavior
* Works perfectly with browsers

===== - Supports all Streamlit internal routes =====
* No path rewriting surprises
* No websocket/streaming confusion
* Predictable browser behavior

So:

<syntaxhighlight>Browser → ALB → Streamlit

</syntaxhighlight>

This is exactly what ALB was built for.

==== ### ====

If UI + API are together:
* 1 user opens the page → UI load
* 1 user chats → API load
* You must scale both together

Example:
* 100 users browsing UI
* Only 10 users chatting

You still pay for 100 backend LLM-capable containers

===== <syntaxhighlight>UI service: 2–3 tasks =====
API service: autoscale on CPU / requests

</syntaxhighlight>

Result:
* UI stays cheap and stable
* API scales only when chat traffic increases
* Much lower cost
* Much better reliability

This is why every serious SaaS separates frontend and backend.

==== No. It was correct for a prototype. ====

But the moment you add:
* API Gateway
* Secrets Manager
* WAF
* Scaling
* Cost control
* Security

You must split them.

: