When to split a Kafka topic

"Market data" is a label, not a topic design. A tick is a small message at a high rate. A headline is rare and wide. A fundamentals snapshot changes on a filing, not on a trade. Put them on one partition and the consumer that only wants candles still deserializes headlines. A slow parse then stalls the price path.

Split on rate, size, and reader

• market.ticks for last trade, bid, ask, and the exchange timestamp
• market.candles for a completed bar, published when the bar closes
• market.news for a headline, its source, and the symbols it mentions
• orders for something a strategy wants done, which is not a price

A service subscribes to the topic it can act on. A moving-average cross reads candles. A sentiment job reads news. An order does not go on market.ticks. Mixing those makes replay meaningless, because a "tick" sometimes meant "we decided to buy."

Key by the thing that must stay ordered

await producer.send({
  topic: "market.candles",
  messages: [{ key: symbol, value: JSON.stringify(bar) }],
});

The same symbol stays on the same partition, so one name is ordered without a lock across the whole stream. Do not promise order across symbols. Most readers do not need it.

The value is the fact a consumer is allowed to see. It is not the producer's private state. State that cannot be rebuilt from the log does not belong in the log.

The browser is not a Kafka client

A web app should read a snapshot over HTTP and hear "something changed" over a socket. The socket service is a consumer with a short buffer, not a second source of truth. If it restarts, the client refetches. That shape is the same one in what a WebSocket event should carry.

A practical test

If two messages differ by an order of magnitude in size or rate, or if one consumer must ignore the other kind, give them different topics. One topic is simpler only until the first slow consumer.