07 — API key security and going further

Duration: 10 min Prerequisites: chapter 02 (you have a key in .env).

The threat model

A leaked OpenAI API key gives an attacker:

• Full access to your account (all models, no per-key restriction unless you set one).
• The ability to burn your billing balance — bots scan public repos within seconds for new sk-... strings.
• The ability to read your usage history.

It does not give them access to:

• Your bank CSV (it lives on your machine).
• Your SQLite database (also local).
• Past chat history (OpenAI doesn’t expose this via the API).

So the primary asset to protect is the key itself — and your bill.

The defense, in 5 layers

Layer 1 — .env + .gitignore

The repo ships with:

csv-llm-openai/.gitignore

.env
.cache/
__pycache__/

And csv-llm-openai/.env.example (committed) is the template:

OPENAI_API_KEY=

You copy it to .env (not committed) and fill in the real key.

Verification:

cd csv-llm-openai
git status
# .env must NOT appear in the output

Layer 2 — separate keys per environment

Don’t reuse one key everywhere. Create:

• csv-llm-openai-dev (your laptop, low monthly limit).
• csv-llm-openai-prod (if you deploy somewhere, higher limit).
• csv-llm-openai-ci (for GitHub Actions, separate budget).

If one leaks, you only revoke that one. The rest keep working.

Layer 3 — set a monthly usage limit

In platform.openai.com → Settings → Limits, set:

• Hard limit: $10/month (refuses requests past this).
• Soft limit: $5/month (email you a warning).

For a course like this, $10 is way more than you’ll ever need. But it caps the blast radius if a key leaks.

Layer 4 — restrict the key’s permissions

OpenAI lets you create restricted keys that can only call specific endpoints. For this app you only need:

• model.read (list models).
• model.write (call chat completions).

Other scopes (file uploads, fine-tuning, batch jobs) can be disabled on the key. If a bot grabs your key and tries to fine-tune a model, the call fails.

Layer 5 — rotate proactively

Every 90 days, regenerate your key:

1. Create a new key.
2. Update your .env (and any deployment).
3. Verify everything still works.
4. Delete the old key in the dashboard.

Even if you can’t see a leak, rotation closes the window of any breach you haven’t noticed yet.

What to do if a key leaks

1. Immediately revoke the leaked key at platform.openai.com.
2. Check your usage page for anomalous activity in the last 24h.
3. If you see fraud, contact OpenAI billing — they often refund.
4. If the leak was on GitHub, scan with trufflehog or GitHub’s secret scanning to ensure nothing else leaked.
5. Investigate: how did it leak? .env committed? Screenshot with key visible? Logged to a public file?

Beyond the key: 3 more hygiene items

Don’t print() your messages in production

The messages list contains your transaction descriptions and amounts. If your app logs print(messages) and that ends up in a public log aggregator (CloudWatch, Datadog, GCP Logs), your data is now there for the duration of the log retention.

Use a structured logger with PII filters, or redact merchant names before logging.

Don’t expose Streamlit publicly

streamlit run app.py --server.port 8505 binds to 0.0.0.0 by default in some setups. Always pass --server.address 127.0.0.1 (or use the start.ps1 helper which already does this) unless you’ve fronted Streamlit with auth.

Don’t ship the .cache/ SQLite

If you containerize the app and accidentally include .cache/csv-llm-openai.sqlite in the image, you’ve leaked all 734 transactions to anyone who pulls the image. .dockerignore should have .cache/. The repo’s .gitignore already does.

What to build next

A. Multi-profile support

Right now the app loads one CSV at a time. Extend:

• A profiles table in SQLite: (profile_id, name, owner, created_at).
• A sidebar dropdown to pick a profile.
• Filter all dashboards and chat by the selected profile.

Use case: a household tracking 2 cards + 1 joint account.

B. Forecasting tool

Add a second tool to the agent:

FORECAST_TOOL = {
    "type": "function",
    "function": {
        "name": "forecast_next_month",
        "description": "Predict next-month spending in a given category using a moving average of the last 3 months.",
        "parameters": {
            "type": "object",
            "properties": {"category": {"type": "string"}},
            "required": ["category"],
        },
    },
}

The LLM now has 2 tools. “What will I likely spend on groceries next month?” triggers forecast_next_month("Groceries"), not query_sql. The LLM picks.

C. Streamed responses

Replace client.chat.completions.create(...) with the streaming variant:

stream = client.chat.completions.create(model=..., messages=..., tools=[...],
                                        stream=True)
for chunk in stream:
    if chunk.choices[0].delta.content:
        st.write(chunk.choices[0].delta.content)

The answer appears word-by-word, much nicer UX. Caveat: handling tool_calls in a stream is trickier (you have to buffer the function name and arguments across chunks).

D. Hybrid Ollama + OpenAI router

Use Ollama for cheap, short, common queries; OpenAI for complex multi-join ones. A trivial heuristic (“does the question contain ‘compare’ or ‘forecast’? → OpenAI; otherwise → Ollama”) gets you 80% of the cost-savings.

Where to learn more

• OpenAI function calling docs — the canonical reference for tools=.
• Project 3 — csv-llm-shared — deep-dive on the shared library both apps use.
• Project 1 — csv-llm-ollama — the local sibling, useful contrast.

Takeaways

• .env + .gitignore + separate keys + monthly limit + rotation = 5 layers, almost no work.
• Don’t log raw messages, don’t bind to 0.0.0.0, don’t ship the SQLite cache.
• Build next: multi-profile, forecasting tool, streaming UI, hybrid router.
• 100% of the agent loop logic transfers to those extensions — you’ve already learned the hard part.

← Back to the Project 2 overview