08 — LLM agent with SQL tool

Duration: 25 min Prerequisites: chapters 03 (Ollama installed) and 05 (DB loaded).

The tool-calling pattern

sequenceDiagram
participant U as You
participant S as Streamlit
participant L as Ollama (llama3.1:8b)
participant D as SQLite

U->>S: "What is my biggest spending category?"
S->>L: chat(messages, tools=[SQL_TOOL])
L->>S: tool_call: query_sql(SELECT ...)
S->>D: safe_query(SELECT ...)
D->>S: DataFrame [(Housing, 30895.29)]
S->>L: chat(messages + tool_result)
L->>S: "Your biggest category is Housing with $30,895.29."
S->>U: rendered answer + SQL expander

Two LLM round-trips per question. One for “decide what to do”, one for “phrase the answer”.

The tool spec

Ollama’s tools= parameter takes a JSON-schema-ish dict. The model sees this and learns it can call query_sql:

SQL_TOOL = {
    "type": "function",
    "function": {
        "name": "query_sql",
        "description": (
            "Run a read-only SELECT query on a SQLite table called `transactions`. "
            "Columns: profile (text), date (text YYYY-MM-DD), card (text), "
            "description (text, the merchant), category (text), amount (real, "
            "positive = expense, negative = refund). "
            "Always filter by `profile = 'data1'`."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "sql": {
                    "type": "string",
                    "description": "A SELECT statement. No INSERT/UPDATE/DELETE."
                }
            },
            "required": ["sql"],
        },
    },
}

The description is the contract. The model uses it to:

know there’s one table called transactions,
know amount > 0 is an expense,
know to filter by profile.

The system prompt

SYSTEM_PROMPT = """You are a personal-finance analyst. The user asks questions about their credit-card transactions. You have one tool: `query_sql(sql)` — SELECT-only on a SQLite table called `transactions`.

Schema:
- profile (text) — always filter `profile = 'data1'`
- date (text YYYY-MM-DD)
- card (text, masked)
- description (text) — the merchant
- category (text) — one of: Housing, Groceries, Dining, Transport, Travel, Business, Utilities, Health, Entertainment, Education, Fees, Payments, Refunds
- amount (real) — positive = expense, negative = refund/credit

Rules:
1. Always call `query_sql` to fetch numbers. NEVER compute or guess.
2. For "spending" questions, filter `amount > 0`.
3. Use exact category names from the list (case-sensitive).
4. After the tool returns, phrase a one-sentence answer with the actual number.
5. If the result is empty, say so explicitly.
"""

Notice rule 1: NEVER compute or guess. This is the lock that prevents hallucinated numbers.

The agent loop in `csv-llm-ollama/llm.py`

import ollama
import json
import sys
sys.path.insert(0, "../csv-llm-shared")
import db


def ask(question: str,
        conn,
        model: str = "llama3.1:8b",
        host: str = "http://127.0.0.1:11434") -> dict:
    client = ollama.Client(host=host)
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": question},
    ]
    sql_calls = []

    # 1. First LLM call: may include tool_calls
    response = client.chat(model=model, messages=messages, tools=[SQL_TOOL])
    msg = response["message"]
    messages.append(msg)

    # 2. If the model called the tool, execute and append the result
    for tc in (msg.get("tool_calls") or []):
        args = tc["function"]["arguments"]
        if isinstance(args, str):
            args = json.loads(args)
        sql = args["sql"]
        try:
            result_df = db.safe_query(conn, sql)
            tool_content = result_df.to_csv(index=False)
        except ValueError as e:
            tool_content = f"ERROR: {e}"
        sql_calls.append({"sql": sql, "result": tool_content})
        messages.append({
            "role": "tool",
            "name": "query_sql",
            "content": tool_content,
        })

    # 3. Second LLM call: phrase the answer from the result
    if sql_calls:
        response = client.chat(model=model, messages=messages, tools=[SQL_TOOL])

    return {
        "answer":    response["message"]["content"],
        "sql_calls": sql_calls,
    }

About 40 lines of glue. That’s it.

A full example, end to end

import sys
sys.path.insert(0, "../csv-llm-shared")
import db
from llm import ask  # csv-llm-ollama/llm.py

conn = db.open_db("csv-llm-ollama/.cache/csv-llm-ollama.sqlite")
# (data already loaded — see chapter 05)

out = ask("What is my biggest spending category?", conn)
print(out["answer"])
for i, c in enumerate(out["sql_calls"], 1):
    print(f"\n--- SQL #{i} ---")
    print(c["sql"])
    print(c["result"])

Real output:

Your biggest spending category is Housing with a total of $30,895.29.

--- SQL #1 ---
SELECT category, SUM(amount) AS total
FROM transactions
WHERE profile = 'data1' AND amount > 0
GROUP BY category
ORDER BY total DESC
LIMIT 1
category,total
Housing,30895.29

The SQL is generated by the model. The number comes from SQLite.

What can go wrong

Issue	Symptom	Fix
Model not tool-capable	Answer like “I can’t query a database”	Use `llama3.1:8b`, not `llama3.2:3b` or `gemma3:270m`
Model invents a column	`no such column: tx_date`	Improve the description in `SQL_TOOL`, list columns explicitly
Model omits `profile = '...'`	Counts rows from other profiles	Add it as rule 1 in the system prompt
Empty result	”No rows returned” but user expected data	Phrase the answer to say so (rule 5)
Tool returns ERROR	LLM panics	Catch in step 2; the LLM sees `ERROR: ...` and can adapt

Takeaways

The agent loop is just two LLM calls with a tool execution in between.
The tool description is the contract — invest time writing it.
The system prompt enforces “never compute, always query”.
All numbers come from SQLite. The LLM only translates question ↔ SQL ↔ English.

Next: Running the app step-by-step →