01 — Overview and goals
Duration: 8 min Prerequisites: none, but having read Project 1 — chapter 01 helps.
When OpenAI beats Ollama
Section titled “When OpenAI beats Ollama”| You… | OpenAI is the right pick when… |
|---|---|
| Have < 8 GB of RAM | Local model would swap to disk and crawl |
| Want fast answers | gpt-4o-mini ≈ 2 s/question vs ~10 s on CPU |
| Are willing to pay ~$0.001 per question | < $0.50 total to finish the project |
| Accept your data going to the cloud | Otherwise → stay on Ollama |
If you already did Project 1, you’ll notice: nothing changes UI-wise. Only llm.py differs. That’s the whole point of the “2 apps + 1 shared module” design.
What stays identical
Section titled “What stays identical”| Aspect | Identical to Project 1? |
|---|---|
| Streamlit UI (3 tabs, sidebar) | Yes |
csv-llm-shared/ingest.py | Yes (same module, same import) |
csv-llm-shared/normalize.py | Yes |
csv-llm-shared/db.py + safe_query | Yes |
csv-llm-shared/dashboards.py | Yes |
| The 4 Plotly figures | Yes (identical KPIs and shapes) |
SQL_TOOL spec (the JSON schema) | Yes, word for word |
SYSTEM_PROMPT | Yes, word for word |
| 4 canonical questions | Yes |
What changes
Section titled “What changes”| Aspect | Ollama (csv-llm-ollama/llm.py) | OpenAI (csv-llm-openai/llm.py) |
|---|---|---|
| Client | ollama.Client(host="…") | openai.OpenAI(api_key="…") |
| Call | client.chat(model, messages, tools=…) | client.chat.completions.create(model, messages, tools=…) |
| Response shape | response["message"]["tool_calls"] | completion.choices[0].message.tool_calls |
| Tool message format | {"role":"tool","name":"query_sql","content":...} | {"role":"tool","tool_call_id":id,"content":...} |
| Default model | llama3.1:8b | gpt-4o-mini |
| Cost per question | $0 | ~$0.0003 |
| Latency on the 4 canonical questions | ~10 s avg | ~2 s avg |
| Port | 8504 | 8505 |
That’s about 40 lines of code changing in llm.py. Everything else is the same.
The privacy trade-off
Section titled “The privacy trade-off”Every time you ask a question in this app:
- Your question text is sent to OpenAI.
- The system prompt (which describes your schema, including category names) is sent to OpenAI.
- After the SQL runs locally, the result rows are sent to OpenAI so it can phrase the answer.
That last point is the important one. If you ask “List my top 5 merchants”, the merchant names and amounts leave your machine.
If that’s a deal-breaker for you:
- Pick Project 1 — Ollama instead.
- Or, weaken what you send: don’t include the result, ask the model to compose a generic answer template (rarely satisfying).
- Or, use a “BYO key + private endpoint” deployment of OpenAI (Azure OpenAI with a no-log contract). Out of scope here.
A representative cost calculation
Section titled “A representative cost calculation”gpt-4o-mini pricing (as of 2026-05):
- Input: $0.15 / 1M tokens.
- Output: $0.60 / 1M tokens.
A typical question in this app:
- System prompt + question = ~600 input tokens.
- Generated SQL + result + answer = ~400 output tokens.
So:
- Input cost: 600 × 0.15 / 1,000,000 = $0.00009.
- Output cost: 400 × 0.60 / 1,000,000 = $0.00024.
- Total: ~$0.00033 per question.
For 1,000 questions, that’s 33 cents. Building the whole course took us ~50 questions — under one cent.
What you won’t have to do
Section titled “What you won’t have to do”- Implement a UI (Streamlit handles it — same as Project 1).
- Re-write ingestion, normalization, DB, dashboards (all from
csv-llm-shared/). - Pay more than a few cents.
- Manage your own SSL or auth (we stay on
127.0.0.1).
Takeaways
Section titled “Takeaways”- OpenAI = same app, faster, paid, less private.
- Only
llm.pydiffers from Project 1 — and only ~40 lines of it. - Cost: well under $0.001 per question with
gpt-4o-mini. - Privacy: question + schema + result rows are sent to OpenAI. Trade-off is yours.