Skip to main content

Your first LLM stage

Intermediate ⏱ 12 min llm · ai · anthropic · ollama

You will build: a pipeline where cheap deterministic stages filter the stream and a model reasons only over what remains — the cost-sane way to put an LLM on events.

webhook → rule-based:triage → llm:reason → (topic out)

Prerequisites​

  • A model provider — pick one: an ANTHROPIC_API_KEY in your environment, or a local Ollama for fully on-prem inference.
Honest constraint

Without a provider the llm stage reports itself unhealthy — visibly, by design. A broken stage is never invisible.

1 · Scaffold rule → llm​

pulse new oncall --source webhook --stage rule-based:triage --stage llm:reason

2 · Gate the stream before the model​

stages:
  - name: triage
    engine: rule-based
    rules:
      - { condition: "severity == 'error'", action: flag }

This is the pattern that keeps LLM pipelines affordable: the model only ever sees events the rules couldn't settle. Info-level noise never costs a token.

3 · Write the prompt​

  - name: reason
    engine: llm
    systemPrompt: |
      You receive one production error event as JSON.
      Classify its likely cause in one word
      (code | infra | config | unknown) and draft a
      one-line summary for the on-call engineer.

4 · Give it a provider and deploy​

export ANTHROPIC_API_KEY=sk-ant-…   # or configure Ollama
pulse secret set PULSE_WEBHOOK_SECRET dev-secret   # the scaffolded webhook source requires it
pulse deploy .

5 · Send an error and read the reasoning​

pulse events publish --topic oncall.in --value '{"severity":"error","msg":"OOMKilled in payments pod"}'
pulse events tail --topic oncall.reason.out

The model's structured answer arrives as an event — classification + summary — ready for the next stage or a sink.

What just happened​

The LLM is a stage, not the architecture. It consumes a topic, emits to a topic, and everything around it — filtering before, routing after, retries, observability — is the same machinery as every other stage. Swapping Anthropic for a local Ollama is a config change, not a rewrite; with Ollama the event data never leaves your machine.

Troubleshooting​

The stage shows unhealthy

No provider is reachable. Check the key is exported in the environment where Pulse runs, or that Ollama is up. The stage stays visibly unhealthy rather than silently passing events through — that's intentional.

Nothing reaches the llm stage

Your triage rule filtered everything. Send an event that matches (severity == 'error') or loosen the rule.

Next​