Skip to main content

Multi-topic union — merge the flows

Intermediate ⏱ 10 min streaming · union · fan-in

You will build: one stage consuming three regional order topics as a single merged flow — fan-in without glue code.

orders-us + orders-eu + orders-apac → union stage → orders-merged

Prerequisites​

1 · Scaffold​

pulse new global-orders --source webhook --stage rule-based:merge

2 · Declare the union​

stages:
  - name: merge
    engine: rule-based
    inputTopics: [ orders-eu, orders-apac ]   # in addition to the primary input
    rules:
      - { condition: "true", action: "emit:orders-merged" }

inputTopics subscribes the stage to extra topics on top of its primary input (global-orders.in). The rule with condition: "true" passes everything through, re-emitting onto one merged topic via action: "emit:orders-merged".

3 · Deploy and feed all three regions​

pulse secret set PULSE_WEBHOOK_SECRET dev-secret   # the scaffolded webhook source requires it
pulse deploy .
pulse events publish --topic global-orders.in --value '{"region":"us","amount":10}'
pulse events publish --topic orders-eu --value '{"region":"eu","amount":20}'
pulse events publish --topic orders-apac --value '{"region":"apac","amount":30}'

4 · One merged flow​

pulse events tail --topic orders-merged

All three events arrive on the single merged topic — the stage consumed the union of its inputs.

What just happened​

Fan-in is declarative: inputTopics on the stage config, nothing else. Combined with fan-out (several stages subscribing to one topic) and the stream-stream join (which auto-subscribes its rightTopic), you have every flow-composition primitive without writing a consumer.

Troubleshooting​

Events from the extra topics don't arrive

inputTopics is a list on the stage config, sibling of rules/operators. Blank entries and the primary topic itself are ignored — check spelling against pulse events tail on the source topic.

Next​