Demo 01 — CSV / SQL RAG with Ollama

Duration: ~25 min Prerequisites: Ollama installed (ollama --version) and Python 3.10+.

Objective

In this demo you will:

1. Check that Ollama is installed and running.
2. Download and run a small local LLM.
3. Download a CSV dataset.
4. Build a simple CSV RAG / SQL RAG system.
5. Ask natural-language questions about the CSV file.

Key idea — for CSV files, numbers, transactions and analytics, do not rely on a vector RAG. Use a SQL RAG approach:

CSV file → database → SQL query → LLM explanation

The database calculates. The LLM explains.

How it works

flowchart LR
  Q["User question<br/>(plain English)"] --> L1["LLM writes SQL"]
  L1 --> D["DuckDB runs SQL<br/>on the CSV"]
  D --> R["Result rows"]
  R --> L2["LLM explains<br/>the result"]
  L2 --> A["Final answer"]

The retrieval step is a SQL query — that's why we call it SQL RAG.

---

Part 1 — Check Ollama

Open a terminal (PowerShell on Windows).

Check the version:

ollama --version

You should see something like ollama version is 0.x.x.

List the installed models:

ollama list

If the list is empty, no model has been downloaded yet.

Part 2 — Download and run a model

Pull a small model (good for a weak or standard laptop):

ollama pull phi3

Run it:

ollama run phi3

Try a couple of questions:

• Explain what a CSV file is in 3 sentences.
• Explain what RAG (Retrieval-Augmented Generation) is in 3 sentences.

To exit the model, type:

/bye

Optional — pull a stronger model

If your machine has enough RAM:

ollama pull mistral:7b
ollama pull llama3.1:8b
ollama pull qwen2.5-coder:7b

Rule of thumb:

Machine	Recommended model
Weak laptop	phi3
Standard laptop	mistral:7b or llama3.1:8b
Coding project	qwen2.5-coder:7b

Part 3 — Test Ollama from the API

Ollama usually runs in the background. If needed, start it:

ollama serve

Test the local API:

curl http://localhost:11434/api/generate -d "{\"model\":\"phi3\",\"prompt\":\"Say hello in one sentence.\",\"stream\":false}"

Ollama returns a JSON response containing the generated answer. You can paste it into jsonlint.com/json-tree to inspect it.

Part 4 — Create the project

Create a project folder:

mkdir ollama-csv-rag
cd ollama-csv-rag

Create and activate a virtual environment:

py -3.12 -m venv .venv
.venv\Scripts\activate

On macOS / Linux:

python3 -m venv .venv
source .venv/bin/activate

Install the libraries:

pip install pandas duckdb requests tabulate

Library	Why
pandas	read and inspect the CSV file
duckdb	local SQL database
requests	call the Ollama API
tabulate	display query results clearly

Part 5 — Download the CSV file

PowerShell:

Invoke-WebRequest -Uri "https://raw.githubusercontent.com/inskillflow/csv-llm-openai/refs/heads/main/data/data1-anonymized.csv" -OutFile "data1-anonymized.csv"

Alternative with curl:

curl -L "https://raw.githubusercontent.com/inskillflow/csv-llm-openai/refs/heads/main/data/data1-anonymized.csv" -o data1-anonymized.csv

Verify the file is there:

dir

You should see data1-anonymized.csv.

Part 6 — The Python script

Create a file named csv_rag_ollama.py and paste this code:

import pandas as pd
import duckdb
import requests
from tabulate import tabulate

CSV_FILE = "data1-anonymized.csv"
MODEL_NAME = "phi3"
OLLAMA_URL = "http://localhost:11434/api/generate"


def ask_ollama(prompt, model=MODEL_NAME):
    payload = {"model": model, "prompt": prompt, "stream": False}
    response = requests.post(OLLAMA_URL, json=payload, timeout=120)
    response.raise_for_status()
    return response.json()["response"].strip()


def clean_sql(sql_text):
    sql_text = sql_text.strip()
    sql_text = sql_text.replace("```sql", "").replace("```", "").strip()
    # Keep only the first SQL statement
    if ";" in sql_text:
        sql_text = sql_text.split(";")[0]
    return sql_text.strip()


def is_safe_select_query(sql):
    forbidden = ["insert", "update", "delete", "drop", "alter",
                 "create", "truncate", "replace"]
    sql_lower = sql.lower()
    if not sql_lower.startswith("select"):
        return False
    return not any(word in sql_lower for word in forbidden)


def main():
    print("-" * 60)
    print("CSV RAG WITH OLLAMA")
    print("-" * 60)

    print("\nLoading CSV file...")
    df = pd.read_csv(CSV_FILE)

    print("\nCSV columns:")
    for col in df.columns:
        print("-", col)

    print("\nFirst rows:")
    print(df.head())

    con = duckdb.connect()
    con.register("transactions", df)

    schema_description = ""
    for col in df.columns:
        schema_description += f"- {col}: {df[col].dtype}\n"

    while True:
        print("\n" + "-" * 60)
        question = input("Ask a question about the CSV file, or type 'exit': ")

        if question.lower() == "exit":
            print("Goodbye.")
            break

        sql_prompt = f"""
You are a data analyst.

Your task is to write a DuckDB SQL query.

The table name is:

transactions

The available columns are:

{schema_description}

User question:

{question}

Rules:
- Return only SQL.
- Return only one SELECT query.
- Do not use INSERT, UPDATE, DELETE, DROP, ALTER, CREATE, or TRUNCATE.
- Do not explain the SQL.
"""

        print("\nGenerating SQL with Ollama...")
        sql_query = clean_sql(ask_ollama(sql_prompt))

        print("\nGenerated SQL:")
        print(sql_query)

        if not is_safe_select_query(sql_query):
            print("\nThe generated SQL query was rejected for safety reasons.")
            continue

        try:
            result_df = con.execute(sql_query).df()
        except Exception as e:
            print("\nSQL execution error:")
            print(e)
            continue

        print("\nSQL result:")
        if result_df.empty:
            print("No result found.")
        else:
            print(tabulate(result_df.head(20), headers="keys", tablefmt="grid"))

        answer_prompt = f"""
You are a helpful data assistant.

The user asked:

{question}

The SQL query used was:

{sql_query}

The SQL result is:

{result_df.head(20).to_string(index=False)}

Write a clear answer for a non-technical user.

Important:
- Do not invent numbers.
- Use only the SQL result.
- If the result is limited, say that it is based on the available CSV data.
"""

        print("\nGenerating final answer with Ollama...")
        print("\nFinal answer:")
        print(ask_ollama(answer_prompt))


if __name__ == "__main__":
    main()

Part 7 — Run it

python csv_rag_ollama.py

You should see the CSV columns, the first rows, and a prompt asking you to ask a question.

Try questions such as:

• What is the total amount in the dataset?
• Which category has the highest total amount?
• What are the top 5 merchants by total amount?
• How many transactions are in the file?
• What is the average transaction amount?
• Show me the monthly spending trend.

Part 8 — What just happened?

This is not a normal chatbot. It is a simple CSV RAG system:

1. The user asks a question in English.
2. The LLM converts the question into a SQL query.
3. DuckDB executes the SQL query on the CSV data.
4. The result is returned to the LLM.
5. The LLM explains the result in natural language.

RAG = Retrieval-Augmented Generation: the model does not answer only from memory, it first retrieves information (here via SQL), then generates an answer. Because the retrieval step is a SQL query, we call this SQL RAG (or CSV RAG, or Database RAG).

Important limitation

The LLM should never invent answers.

Bad design	Good design
User: How much did I spend last month? → LLM answers from imagination.	User: How much did I spend last month? → system runs SELECT SUM(amount) ... → LLM explains the real result.

The two safety nets in the code matter:

• is_safe_select_query() rejects anything that is not a single SELECT (no INSERT, DELETE, DROP, …).
• The answer prompt explicitly says “do not invent numbers, use only the SQL result”.

When to use which RAG

Data type	Use
CSV, tables, numbers, money, statistics	SQL RAG (this demo)
PDFs, text documents, course notes, manuals	Document RAG with embeddings (e.g. ollama pull nomic-embed-text)
Products, recommendations	Metadata RAG
Healthcare / sensitive info	Controlled RAG with trusted sources

Main idea: the database calculates, the LLM explains.