Project: Ollama REPL

Source: 5-ollama-repl/
Model: gemma4:26b
Type: TypeScript (Bun runtime, Ollama chat API)

What It Does

A fully interactive chat agent that talks to your local Ollama instance. It supports:

• Turn-by-turn conversation — type messages, get streaming responses, maintain context
• Single-shot mode — pipe a prompt in, get the response, exit
• Code extraction — automatically pulls code blocks from responses and saves them to repl-code-output
• Streaming — responses appear in real-time, token by token

This is the most sophisticated project — a real chat application with conversation history, streaming, and code extraction, all built by the AI.

The Prompt

use typescript and bun

output only the code with no commentary and a proper shebang

write a chat agent that talks to the local ollama instance using the
model named "gemma4:26b"

keep track of messages and responses so that we can have a conversation

look for code blocks inside of the response, starting with ```, and
write them to repl-code-output

stream the responses

i want to either cat a message in and wait for the response, or I
want to have turn by turn

if the user echos a prompt into it, it should generate the response
and close the sessions

when implementing the interactive prompt, ensure you use
`process.stdin` and `process.stdout` for the readline interface to
avoid import errors from the node:readline module

output only code

The Model’s Reasoning

The model’s thinking followed a clear chain:

• Ollama API: Use POST http://localhost:11434/api/chat with stream: true — this is the chat endpoint (different from /api/generate which is for single completions)
• History Management: Maintain { role, content }[] array — push user messages before sending, push assistant response after receiving
• Streaming: Read response body as chunks, decode JSON lines, print content as it arrives
• Code Extraction: After streaming completes, regex-search the full response for code blocks and append to file
• Dual Mode: Check process.argv for a piped message; if present, single-shot and exit; otherwise, interactive loop

The Code

Note: The code below is from the model’s response.md output. The extractor only captured a regex fragment to code/ (first code block behavior), but the full working script is preserved here.

#!/usr/bin/env bun

import readline from "node:readline";
import { appendFileSync } from "node:fs";

const MODEL = "gemma4:26b";
const CODE_FILE = "repl-code-output";
const history: { role: "user" | "assistant"; content: string }[] = [];

async function executeChat(prompt: string, isSingleMode: boolean) {
  history.push({ role: "user", content: prompt });

  try {
    const response = await fetch("http://localhost:11434/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        model: MODEL,
        messages: history,
        stream: true,
      }),
    });

    if (!response.body) {
      throw new Error("No response body from Ollama");
    }

    const reader = response.body.getReader();
    const decoder = new TextDecoder();
    let assistantContent = "";

    process.stdout.write("\nAssistant: ");

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value, { stream: true });
      const lines = chunk.split("\n");

      for (const line of lines) {
        if (!line.trim()) continue;
        try {
          const json = JSON.parse(line);
          const content = json.message?.content || "";
          assistantContent += content;
          process.stdout.write(content);
          if (json.done) break;
        } catch (e) {
          // Handle partial JSON chunks
        }
      }
    }

    process.stdout.write("\n");
    history.push({ role: "assistant", content: assistantContent });

    // Extract code blocks
    const codeBlockRegex = /```(?:[a-z]*\n)?([\s\S]*?)```/g;
    let match;
    while ((match = codeBlockRegex.exec(assistantContent)) !== null) {
      appendFileSync(CODE_FILE, match[1].trim() + "\n\n");
    }

    if (isSingleMode) {
      process.exit(0);
    }
    return true;
  } catch (error) {
    console.error("\nError:", error instanceof Error ? error.message : error);
    if (isSingleMode) process.exit(1);
    return false;
  }
}

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
  terminal: true,
});

const args = process.argv.slice(2);

if (args.length > 0) {
  executeChat(args[0], true).catch(() => process.exit(1));
} else {
  rl.setPrompt("User > ");
  rl.prompt();

  rl.on("line", async (line) => {
    const input = line.trim();
    if (!input) {
      rl.prompt();
      return;
    }
    if (input.toLowerCase() === "exit" || input.toLowerCase() === "quit") {
      rl.close();
      process.exit(0);
    }

    const success = await executeChat(input, false);
    if (success) {
      rl.prompt();
    } else {
      rl.prompt();
    }
  });
}

How to Use

# Install Bun if you haven't
curl -fsSL https://bun.sh/install | bash

# Make it executable
chmod +x 5-ollama-repl/code

# Interactive mode
./5-ollama-repl/code
User > What's the capital of France?
Assistant: The capital of France is Paris.
User > What's its population?
Assistant: Paris has a population of approximately 2.1 million...

# Single-shot mode (pipe a question in)
echo "Write a haiku about coding" | ./5-ollama-repl/code

# Code blocks are automatically saved
cat repl-code-output

The Test

#!/bin/bash
echo "write a funny poem about a puppy" | ./5-ollama-repl/code

Architecture Notes

Ollama Chat API vs Generate API

The REPL uses /api/chat (not /api/generate). The difference:

The chat API is the right choice for an interactive REPL because it natively handles conversation history.

Streaming Implementation

The model reads the response body as a stream of NDJSON (newline-delimited JSON) chunks:

{"message":{"content":"The "}}
{"message":{"content":"capital "}}
{"message":{"content":"is "}}
{"message":{"content":"Paris."}}
{"done":true}

Each chunk contains a fragment of the response. The script accumulates them into assistantContent while printing to stdout in real time.

Code Extraction Pattern

After streaming completes, the script runs a regex over the full response:

const codeBlockRegex = /```(?:[a-z]*\n)?([\s\S]*?)```/g;

This matches fenced code blocks (with optional language tag) and captures their content. Each match is appended to repl-code-output.

Key Lessons

• Two Ollama APIs — /api/generate for completions, /api/chat for conversations
• Streaming is straightforward — NDJSON chunks, parse and print
• History management — push user → send → accumulate assistant → push assistant
• Bun is fast — no compilation step, instant startup, native fetch
• The model built a real chat app — with two input modes, streaming, and code extraction, all from a single prompt

Back to Projects Index → or Course Home →