Skip to main content

Building a Custom Blockchain Indexer: What It Actually Takes

· 3 min read
OverBlock Team
OverBlock Team
OverBlock Engineering

Custom blockchain indexers are one of those projects that look like a week of work and turn into a quarter of work. Not because the core loop is hard, but because every edge case compounds.

The components​

A production indexer has more moving parts than just the core loop:

Block reader - connects to the node, requests blocks in order, handles disconnections and reconnections without data loss.

Reorg handler - detects when a block you have already processed is no longer in the canonical chain. Must roll back all state changes from that block before processing the replacement chain. This is the hardest part to get right - getting it wrong produces silently corrupted data.

State machine - your business logic. For most use cases, this is the part developers want to spend time on. The rest is infrastructure.

Checkpoint persistence - stores the last successfully processed block height. Without this, any crash requires reprocessing from the start.

API layer - serves your downstream consumers. Often a separate service with its own caching and auth.

Monitoring - node health, indexer lag (how far behind tip), error rates, and storage usage.

Realistic timeline​

For a team of two developers building from scratch:

  • Basic block loop with storage: 1 week
  • Reorg handling done correctly: 2-4 weeks
  • API layer and integration tests: 1-2 weeks
  • First production deploy and monitoring: 1 week
  • Stabilization and edge cases: ongoing

Six to eight weeks before the first reliable API response is a realistic estimate.

When to build vs buy​

Build when:

  • Your data model is unique and no existing app covers it
  • You have dedicated infrastructure capacity (5+ person engineering team)
  • Regulatory or data sovereignty requirements mandate on-premises

Use managed when:

  • Your team is small (1-3 developers) and cannot afford weeks on infrastructure
  • Your use case will eventually be covered by a ready-made app
  • Time-to-first-data is a competitive factor

OverBlock's custom indexers (coming soon) let you define the state machine in TypeScript - the specific part - while infrastructure, reorg handling, and operations are managed.

A minimal Bitcoin block reader illustrates the core loop:

// The easy part: fetch and process
async function runIndexer(startHeight: number) {
  let height = await loadCheckpoint() ?? startHeight;

  while (true) {
    const block = await node.getBlock(height);
    if (!block) { await sleep(10_000); continue; }

    // The hard part: reorg detection
    const prev = await db.getBlock(height - 1);
    if (prev && block.previousHash !== prev.hash) {
      await rollbackTo(height - 1); // your state must support this
      height--;
      continue;
    }

    await db.transaction(async (tx) => {
      await processBlock(tx, block);
      await tx.setCheckpoint(height); // must be atomic
    });
    height++;
  }
}

The rollbackTo function is where most teams underestimate complexity.

→ Custom Indexers - Coming Soon