---
title: "Ollama Repl"
---

# Project: Ollama REPL

**Source:** `5-ollama-repl/`  
**Model:** `gemma4:26b`  
**Type:** TypeScript (Bun runtime, Ollama chat API)

## What It Does

A fully interactive chat agent that talks to your local Ollama instance. It supports:

- **Turn-by-turn conversation** — type messages, get streaming responses, maintain context
- **Single-shot mode** — pipe a prompt in, get the response, exit
- **Code extraction** — automatically pulls code blocks from responses and saves them to `repl-code-output`
- **Streaming** — responses appear in real-time, token by token

This is the most sophisticated project — a real chat application with conversation history, streaming, and code extraction, all built by the AI.

## The Prompt

````
use typescript and bun

output only the code with no commentary and a proper shebang

write a chat agent that talks to the local ollama instance using the
model named "gemma4:26b"

keep track of messages and responses so that we can have a conversation

look for code blocks inside of the response, starting with ```, and
write them to repl-code-output

stream the responses

i want to either cat a message in and wait for the response, or I
want to have turn by turn

if the user echos a prompt into it, it should generate the response
and close the sessions

when implementing the interactive prompt, ensure you use
`process.stdin` and `process.stdout` for the readline interface to
avoid import errors from the node:readline module

output only code
````

## The Model's Reasoning

The model's thinking followed a clear chain:

- **Ollama API:** Use `POST http://localhost:11434/api/chat` with `stream: true` — this is the chat endpoint (different from `/api/generate` which is for single completions)
- **History Management:** Maintain `{ role, content }[]` array — push user messages before sending, push assistant response after receiving
- **Streaming:** Read response body as chunks, decode JSON lines, print content as it arrives
- **Code Extraction:** After streaming completes, regex-search the full response for code blocks and append to file
- **Dual Mode:** Check `process.argv` for a piped message; if present, single-shot and exit; otherwise, interactive loop

## The Code

> **Note:** The code below is from the model's `response.md` output. The extractor only captured a regex fragment to `code/` (first code block behavior), but the full working script is preserved here.

````typescript
#!/usr/bin/env bun

import readline from "node:readline";
import { appendFileSync } from "node:fs";

const MODEL = "gemma4:26b";
const CODE_FILE = "repl-code-output";
const history: { role: "user" | "assistant"; content: string }[] = [];

async function executeChat(prompt: string, isSingleMode: boolean) {
  history.push({ role: "user", content: prompt });

  try {
    const response = await fetch("http://localhost:11434/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        model: MODEL,
        messages: history,
        stream: true,
      }),
    });

    if (!response.body) {
      throw new Error("No response body from Ollama");
    }

    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let assistantContent = "";

    process.stdout.write("\nAssistant: ");

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value, { stream: true });
      const lines = chunk.split("\n");

      for (const line of lines) {
        if (!line.trim()) continue;
        try {
          const json = JSON.parse(line);
          const content = json.message?.content || "";
          assistantContent += content;
          process.stdout.write(content);
          if (json.done) break;
        } catch (e) {
          // Handle partial JSON chunks
        }
      }
    }

    process.stdout.write("\n");
    history.push({ role: "assistant", content: assistantContent });

    // Extract code blocks
    const codeBlockRegex = /```(?:[a-z]*\n)?([\s\S]*?)```/g;
    let match;
    while ((match = codeBlockRegex.exec(assistantContent)) !== null) {
      appendFileSync(CODE_FILE, match[1].trim() + "\n\n");
    }

    if (isSingleMode) {
      process.exit(0);
    }
    return true;
  } catch (error) {
    console.error("\nError:", error instanceof Error ? error.message : error);
    if (isSingleMode) process.exit(1);
    return false;
  }
}

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
  terminal: true,
});

const args = process.argv.slice(2);

if (args.length > 0) {
  executeChat(args[0], true).catch(() => process.exit(1));
} else {
  rl.setPrompt("User > ");
  rl.prompt();

  rl.on("line", async (line) => {
    const input = line.trim();
    if (!input) {
      rl.prompt();
      return;
    }
    if (input.toLowerCase() === "exit" || input.toLowerCase() === "quit") {
      rl.close();
      process.exit(0);
    }

    const success = await executeChat(input, false);
    if (success) {
      rl.prompt();
    } else {
      rl.prompt();
    }
  });
}
````

## How to Use

```bash
# Install Bun if you haven't
curl -fsSL https://bun.sh/install | bash

# Make it executable
chmod +x 5-ollama-repl/code

# Interactive mode
./5-ollama-repl/code
User > What's the capital of France?
Assistant: The capital of France is Paris.
User > What's its population?
Assistant: Paris has a population of approximately 2.1 million...

# Single-shot mode (pipe a question in)
echo "Write a haiku about coding" | ./5-ollama-repl/code

# Code blocks are automatically saved
cat repl-code-output
```

## The Test

```bash
#!/bin/bash
echo "write a funny poem about a puppy" | ./5-ollama-repl/code
```

## Architecture Notes

### Ollama Chat API vs Generate API

The REPL uses `/api/chat` (not `/api/generate`). The difference:

| Feature      | `/api/generate`              | `/api/chat`                           |
| ------------ | ---------------------------- | ------------------------------------- |
| Input        | Single prompt string         | Array of `{ role, content }` messages |
| Conversation | Must manually manage context | API handles multi-turn                |
| Streaming    | `"stream": true`             | `"stream": true`                      |
| Use case     | One-shot completions         | Conversations                         |

The chat API is the right choice for an interactive REPL because it natively handles conversation history.

### Streaming Implementation

The model reads the response body as a stream of NDJSON (newline-delimited JSON) chunks:

```
{"message":{"content":"The "}}
{"message":{"content":"capital "}}
{"message":{"content":"is "}}
{"message":{"content":"Paris."}}
{"done":true}
```

Each chunk contains a fragment of the response. The script accumulates them into `assistantContent` while printing to stdout in real time.

### Code Extraction Pattern

After streaming completes, the script runs a regex over the full response:

````typescript
const codeBlockRegex = /```(?:[a-z]*\n)?([\s\S]*?)```/g;
````

This matches fenced code blocks (with optional language tag) and captures their content. Each match is appended to `repl-code-output`.

## Key Lessons

- **Two Ollama APIs** — `/api/generate` for completions, `/api/chat` for conversations
- **Streaming is straightforward** — NDJSON chunks, parse and print
- **History management** — push user → send → accumulate assistant → push assistant
- **Bun is fast** — no compilation step, instant startup, native fetch
- **The model built a real chat app** — with two input modes, streaming, and code extraction, all from a single prompt

---

**Back to [Projects Index →](/run-ai-locally/projects/)** or **[Course Home →](/run-ai-locally/)**