From rules to data

Three styles of building software

Before talking about Machine Learning we need a shared vocabulary. Almost every piece of software you have ever used falls into one of three styles:

flowchart TB
  A["Traditional programming<br/>(if/else, loops)"] --> B["Libraries &amp; frameworks<br/>(re-use blocks)"]
  B --> C["Machine learning<br/>(learn from data)"]
  style A stroke-dasharray: 0
  style B stroke-dasharray: 0
  style C stroke-dasharray: 0

Each style is built on top of the previous one.

1. Traditional programming

A human writes every rule. Inputs go in, the code applies the logic, outputs come out.

def is_adult(age):
    if age >= 18:
        return True
    return False

The world is full of these systems: payroll, calculators, login flows, sorting algorithms. You wrote the rules, the computer just executes them.

2. Functions, libraries, frameworks, and features

Nobody writes everything from scratch any more. A modern app stitches together reusable blocks:

• Functions — small reusable units (calc_tax).
• Libraries — bags of functions you import (numpy, requests).
• Frameworks — opinionated scaffolds that call your code (FastAPI, Next.js).
• Features — packaged user-facing capabilities (login, search, payments).

This is still traditional programming — the rules are explicit, just borrowed from someone else’s package.

3. Rule-based systems

For decades, the way to build “AI” was to write a lot of rules. A rule-based system is essentially a giant decision tree maintained by domain experts.

Example — a tax software contains thousands of rules like “if the user is self-employed and earned more than X, then apply form 1099.”

The hard limits:

• Brittle: a missing rule = a wrong answer with no warning.
• Expensive: rules must be written, audited, and updated by experts.
• Domain-locked: a rule-based system for tax cannot help with medical diagnosis without being rewritten from scratch.

The Machine Learning shift

Around 1990–2010 a different idea took over: don’t write the rules — let the machine derive them from examples.

flowchart LR
  subgraph Traditional["Traditional programming"]
    direction LR
    I1["Input"] --> R1["Rules<br/>(human-written)"] --> O1["Output"]
  end
  subgraph ML["Machine Learning"]
    direction LR
    I2["Input<br/>+<br/>Labelled examples"] --> R2["Model<br/>(rules learned automatically)"] --> O2["Output"]
  end

The same problem, two opposite philosophies. ML moves the hard work from coding to data.

In practice, instead of writing “if email contains ‘free Viagra’ then it’s spam”, we show the algorithm 10,000 labelled emails and let it figure out the patterns by itself.

The trade-off is real and worth memorising:

Approach	You write…	You need…	Fails when…
Traditional / Rule-based	The rules	Domain experts	The world changes
Machine learning	The training loop	Lots of labelled data	Data is biased or scarce

Why this matters for the rest of the course

Once you accept that software can be derived from data, everything else falls into place:

• LLMs are just gigantic models derived from gigantic text corpora.
• “Training a model” = the cost paid once.
• “Inference” = using the model afterwards = what you pay every time a user queries it.

Key takeaways

• Traditional code = rules written by humans.
• Libraries and frameworks don’t change the philosophy — they just reuse other people’s rules.
• Rule-based systems hit a ceiling: too many rules, too rigid.
• Machine learning flips the problem: you supply data, the model derives the rules.

---

Next: Types of machine learning — supervised, unsupervised, and reinforcement learning.