01 — Why a shared library

Duration: 8 min Prerequisites: none.

The problem

When we built the Ollama app and the OpenAI app side by side, we discovered they share an enormous amount of code:

Shared between Ollama and OpenAI	Lines of code (~)
CSV ingestion (header aliases, delimiter detection)	150
Normalization (dates, signed amount, trimming)	80
SQLite schema, load, safe_query	200
Plotly figures (4 functions)	180
Category rules engine	100
Total shared	~710

What’s app-specific:

App-specific code	Lines (~)
csv-llm-ollama/llm.py (Ollama agent loop)	60
csv-llm-openai/llm.py (OpenAI agent loop)	65
csv-llm-ollama/app.py (Streamlit UI, identical to OpenAI 95%)	230
csv-llm-openai/app.py (Streamlit UI)	230

So roughly 710 lines are shared, ~600 lines are app-specific. Extracting the shared portion saves 710 lines of duplication.

Three options for sharing

Option 1 — Copy-paste

Each app keeps its own copy of ingest.py, normalize.py, etc.

Pros	Cons
Apps are fully self-contained	Drift: a bug fixed in Ollama goes unfixed in OpenAI
No dependency to manage	Every improvement has to be replicated by hand
Easy to “git clone and run”	710 lines of duplicate maintenance

We rejected this.

Option 2 — One app imports from the other

csv-llm-openai does from csv-llm-ollama.ingest import read_csv.

Pros	Cons
Avoids copy	Inappropriate coupling: OpenAI depends on Ollama for non-LLM logic
	Renaming Ollama renames a dep of OpenAI
	Confusing for newcomers

Also rejected.

Option 3 — Extract a shared library

Both apps from csv_llm_shared import ingest, normalize, db, dashboards, categorize.

Pros	Cons
Clean dependency graph	One more repo to clone
Each repo has one job	Versioning across 3 repos
Tested independently
Reusable for other apps (chapter 08)

We picked this. The downside (3 repos to clone instead of 2) is mitigated by sibling-directory checkouts.

The dependency graph

flowchart LR
Shared[csv-llm-shared]
Ollama[csv-llm-ollama]
OpenAI[csv-llm-openai]
Other[your-other-app]
Ollama --> Shared
OpenAI --> Shared
Other -.optional.-> Shared

csv-llm-shared knows nothing about Streamlit, Ollama, or OpenAI. It’s a pure data-engineering library.

Design rules csv-llm-shared follows

These are the rules we imposed on the library to make it reusable:

1. Pure functions over classes

Every function takes inputs and returns outputs. Almost no state. Easy to test.

# Good
def normalize(df, profile="default") -> DataFrame:
    ...

# Bad (we don't do this)
class Normalizer:
    def __init__(self, ...): ...
    def normalize(self, df): ...

2. DataFrames in, DataFrames out

The currency of the library is pandas.DataFrame. Every public function either consumes or produces one. No custom classes.

3. No LLM, no UI, no secrets

The library has zero imports from streamlit, ollama, openai. The closest thing to an LLM is categorize_with_llm() in categorize.py — and even that takes an injected client (not a hard-coded import), so it can be skipped entirely.

4. Fail loudly, fail safely

db.safe_query() raises ValueError on anything non-SELECT. Better to crash than silently mutate.

5. One source of truth for the schema

The SQLite schema is defined once in db.py as a SCHEMA constant. Both apps reference the same one.

6. Ships with realistic demo data

csv-llm-shared/data/data1-anonymized.csv (734 rows). Students can clone && python smoke_pipeline.py and see real numbers.

What this enables

Once the library is in place, the two apps become almost trivial:

# csv-llm-ollama/app.py (excerpt)
import sys; sys.path.insert(0, "../csv-llm-shared")
import ingest, normalize, db, dashboards, categorize

df = ingest.read_csv(csv_path)
nf = normalize.normalize(df, profile=profile_name)
conn = db.open_db(".cache/csv-llm-ollama.sqlite")
db.load_dataframe(conn, nf, profile=profile_name)
st.plotly_chart(dashboards.fig_monthly(nf))
# ... then 60 lines of Ollama-specific LLM glue

The app is now ~290 lines instead of 1000. Onboarding a new developer takes minutes, not days.

What csv-llm-shared is NOT

To set expectations:

• ❌ Not a general-purpose ETL framework. It’s tailored to credit-card-like CSVs.
• ❌ Not an OR/M. We use sqlite3 directly because the schema is tiny and stable.
• ❌ Not a feature-engineering pipeline. No model training, no embeddings.
• ❌ Not a multi-database adapter. It’s SQLite, period.

If you outgrow these, the library is small enough to fork and rewrite. That’s intentional.

Takeaways

• Two apps that share ~710 lines is enough to justify a shared library.
• Rules: pure functions, DataFrame currency, no LLM/UI/secrets, fail loudly, one source of truth.
• The library shrinks each app from ~1000 to ~300 lines.
• In the next chapters, we dissect each module of the library, one per chapter.

Next: ingest.py — multi-bank reader →