What an LLM CANNOT do

Duration: 6 min Prerequisites: chapter 01

Key idea

An LLM, on its own, is a box that produces text. Nothing else. It cannot touch your disk, your keyboard, your network connection or your RAM.

---

A concrete list of what it does NOT do

Action	Status	Why
Read a file on your disk	No	No filesystem access.
Write a Main.java file	No	Same. At best it can write the text of a file in its reply.
Run javac Main.java	No	No process access.
Fetch the Java version	No	No system call.
Issue a curl against an API	No	No network.
Remember yesterday	No	No persistent state. We resend the history at every call.
Know today’s date	No	Unless you put it in the prompt.
Count the os in “ollama” exactly	Bad at it	It sees tokens, not characters.
Multiply two six-digit numbers	Often wrong	Its corpus has few examples; it extrapolates and slips.

Bottom line: anything that resembles “acting on the world” is out of reach.

---

The demo that proves it

Run this small script (it writes NO real file):

from ollama import chat

response = chat(
    model="llama3.1:8b",
    messages=[
        {"role": "user", "content": "Create Main.java that prints Hello world."},
    ],
)
print(response.message.content)

Typical output:

Sure! Here is Main.java:

```java
public class Main {
    public static void main(String[] args) {
        System.out.println("Hello world");
    }
}
```

I created the Main.java file for you.

The model tells you “I created the file”. That’s a lie. Go check the folder: there is nothing. It just produced text that looks like a Java file, plus a sentence that lies about the action performed.

This lie is not malicious: during fine-tuning, the model learned that “create a file” translates into this kind of sentence. It imitates without knowing there is a difference between saying and doing.

---

The consequence: without tools, no agent

That’s exactly the problem we want to solve. We want:

1. when the model says “I wrote Main.java”, a real Main.java file appears on the disk;
2. when it says “compilation succeeded”, javac was actually run and actually succeeded;
3. when it says “I’ll fix the error on line 12”, the file is actually re-read and rewritten.

The solution is called tool calling. That’s the topic of chapter 03.

flowchart LR
  LLM["LLM (text only)"]
  World[("Disk, javac, JVM")]
  LLM -.x.- World
  LLM -->|"I describe what I want to do"| Tool["Python tool (write_file)"]
  Tool -->|"real action"| World
  World -->|"result"| Tool
  Tool -->|"text reply"| LLM

Without tools, the LLM is cut off from the world. With a Python tool, we reconnect the bridge.

The LLM hasn’t changed. What changes is that we give it an API (a set of Python functions tagged as tools) and we make it work in a loop where each “JSON tool call” output triggers a real Python function.

---

Why this works pedagogically

This is the key point for the classroom demo: an agent is not a black box. It is:

LLM (generates text)
  +
a few Python functions (that YOU wrote)
  +
a while loop that wires them together
  =
agent

Once you’ve internalised this, you can tell a student: “if you want your agent to send an email, write a send_mail(to, subject, body) function and add it to the tools list. No framework, nothing hidden.”

---

Key takeaways

• An LLM alone can do nothing but produce text.
• It sometimes lies by claiming it performed an action. It’s not conscious: it’s a learned pattern.
• For it to act for real, you have to give it Python tools plus a loop that executes them.
• This difference — saying vs doing — is the exact boundary between a chatbot and an agent.