Deduplication

The most subtle aspect of the stream-json protocol. If you're building a consumer that processes assistant events, you must understand this.

The Problem

When Claude Code runs with --verbose, every assistant event contains the complete message so far — not an incremental delta. Each event repeats all previous content blocks plus any new ones.

Example: Three Consecutive Events

Event 1: Claude starts thinking

{
  "type": "assistant",
  "message": {
    "content": [
      { "type": "thinking", "thinking": "Let me look at the code..." }
    ]
  }
}

Event 2: Claude adds text response

{
  "type": "assistant",
  "message": {
    "content": [
      { "type": "thinking", "thinking": "Let me look at the code..." },
      { "type": "text", "text": "I found the issue." }
    ]
  }
}

Event 3: Claude adds a tool call

{
  "type": "assistant",
  "message": {
    "content": [
      { "type": "thinking", "thinking": "Let me look at the code..." },
      { "type": "text", "text": "I found the issue." },
      { "type": "tool_use", "id": "toolu_1", "name": "Edit", "input": {...} }
    ]
  }
}

Without Dedup

A naive consumer that processes every content block in every event would see:

Block	Times Processed
thinking	3 times
text	2 times
tool_use	1 time

This means duplicated text in your UI, duplicated tool calls, or duplicated costs.

The Solution: Content Index Tracking

Track the index of the last processed content block. On each assistant event, only process blocks from lastContentIndex onward:

Event 1: content.length=1 → process blocks[0..0] → lastContentIndex=1
Event 2: content.length=2 → process blocks[1..1] → lastContentIndex=2
Event 3: content.length=3 → process blocks[2..2] → lastContentIndex=3

Each block emitted exactly once.

Turn Boundaries

Reset lastContentIndex to 0 when a result or system/result event arrives. This marks the end of one turn and the start of the next.

Content Array Resets

After a tool_use → tool_result cycle, Claude starts a new API call with a fresh content array. The content length drops below lastContentIndex. The translator detects this and auto-resets.

How the Translator Handles It

The Translator class does all of this automatically:

import { parseLine, Translator } from 'claude-code-parser'

const translator = new Translator()

// Feed events in order — dedup is automatic
for (const line of ndjsonLines) {
  const event = parseLine(line)
  if (!event) continue

  const relayEvents = translator.translate(event)
  // relayEvents contains only NEW content — no duplicates
}

You don't need to implement dedup yourself. Just use the Translator.

Edge Cases

Thinking Blocks May Grow

The thinking block text itself can grow between events (Claude streams thinking). The index-based approach treats each snapshot as final — the last thinking block at a given index is the complete one.

Text Blocks May Grow In-Place

Similarly, text content can grow within a single block between events. The index-based approach sends the full block content at first sight. For true character-level streaming, you would need character-level diffing on top of the translator output.

Non-Verbose Mode

Without --verbose, events may be true deltas (not cumulative). The translator handles both modes — if content doesn't grow cumulatively, the index tracking is a no-op.