Pp2pbuilders
learn › The P2P Engineer's Field Manual

10 min read · also at #/learn/fm-day6

Day 06: High-Performance JS Networking

The event loop, buffers, backpressure, and the allocation discipline that separates a toy from a relay.

⏱ ~90 min · theory 45 / lab 45

You can know every protocol and still write a P2P node that melts under load. Performance in Node networking is
mostly about three things: respecting the event loop, managing buffers without
drowning the garbage collector, and honoring backpressure so a fast sender can't overwhelm a slow
consumer. This chapter is where the relay-and-throughput intuition that interviewers probe actually lives.

6.1 The event loop is your whole performance model

Node runs your JavaScript on a single thread. I/O (sockets, files) is handed off to libuv and
completes asynchronously, but every callback, every data handler, every bit of parsing runs on that one
thread. The rule that follows is absolute: never block the loop. A synchronous JSON.parse of a huge
payload, a tight CPU loop, a synchronous crypto call on every packet — each one stalls all connections, not
just one. Throughput collapses not because the network is slow but because the loop is busy.

Mental model
Think of the event loop as a single cashier serving every customer. The cashier is extremely fast, but if you ask
them to do long division by hand for one customer, the entire queue waits. Your job is to keep every interaction
short and push the long division elsewhere.

6.2 Buffers and the GC tax

Every Buffer you allocate in a hot path is future work for the garbage collector. Under high packet
rates, allocation churn causes GC pauses that show up as latency spikes and jitter. The disciplines that matter:

Holepunch note
This is exactly the discipline a relay like a HiveRelay node lives or dies by. A relay forwards huge numbers of
small frames; if each forwarded frame allocates a fresh buffer and triggers a concat, GC pressure caps
your throughput long before bandwidth does. Pre-sized read buffers, subarray views for forwarding, and
bounded per-event work are what let a single Node process push serious packet rates.

6.3 Backpressure: the rule everyone forgets

socket.write(chunk) returns a boolean. When it returns false, the kernel/stream buffer is
full and you must stop writing until the 'drain' event fires. Ignore this and Node
happily queues your data in memory — unbounded — until the process balloons and dies. A fast producer feeding a slow
socket is the most common way a P2P node runs out of memory.

The clean solutions: use stream.pipe() (which handles backpressure for you), or pipeline(),
or honor the write/'drain' contract manually. Async iterators (for await...of
over a readable) also propagate backpressure naturally.

6.4 Measuring: latency vs throughput

Two different numbers people constantly conflate:

You can have great throughput and terrible tail latency (big batches, long GC pauses) or low latency and poor
throughput (tiny unbatched writes with per-message overhead). Always measure percentiles (p50, p99),
never just the mean — the mean hides exactly the stalls that matter. perf_hooks.monitorEventLoopDelay()
quantifies how much the loop itself is the bottleneck.

6.5 Lab — find and fix the throughput killers

Goal

Measure three real performance effects: (1) the Buffer.concat O(n²) trap vs a
pre-sized buffer, (2) allocUnsafe vs alloc in a hot loop, and (3) backpressure on a TCP
socket. You'll produce numbers, not just theory.

Step 1 — the accumulating-concat trap

// concat-trap.js — why naive reassembly is O(n^2)
const { performance } = require('perf_hooks');
const CHUNKS = 20000;
const chunk = Buffer.allocUnsafe(64);

// BAD: concat the whole growing buffer on every chunk
let t = performance.now();
let acc = Buffer.alloc(0);
for (let i = 0; i < CHUNKS; i++) acc = Buffer.concat([acc, chunk]);
console.log('concat-each-time :', (performance.now() - t).toFixed(1), 'ms');

// GOOD: collect references, concat ONCE at the end
t = performance.now();
const parts = [];
for (let i = 0; i < CHUNKS; i++) parts.push(chunk);
const joined = Buffer.concat(parts);
console.log('collect-then-join:', (performance.now() - t).toFixed(1), 'ms');

// BEST when total size known: one allocation, copy into it
t = performance.now();
const dst = Buffer.allocUnsafe(CHUNKS * 64);
for (let i = 0; i < CHUNKS; i++) chunk.copy(dst, i * 64);
console.log('presized-copy    :', (performance.now() - t).toFixed(1), 'ms');
node concat-trap.js
// concat-each-time : 3380.0 ms    <- quadratic; explodes with size
// collect-then-join:    2.3 ms     <- linear
// presized-copy    :    6.9 ms     <- linear, one allocation

The exact numbers vary by machine, but the shape is the lesson: the naive version is *hundreds to
thousands* of times slower and gets worse as the buffer grows. This is the framing decoder from Day 4 done wrong.

Step 2 — allocUnsafe vs alloc in a hot path

// alloc-bench.js
const { performance } = require('perf_hooks');
const N = 2_000_000, SIZE = 256;

let t = performance.now();
for (let i = 0; i < N; i++) { const b = Buffer.alloc(SIZE); b[0] = 1; }
console.log('alloc      :', (performance.now() - t).toFixed(0), 'ms');

t = performance.now();
for (let i = 0; i < N; i++) { const b = Buffer.allocUnsafe(SIZE); b[0] = 1; }
console.log('allocUnsafe:', (performance.now() - t).toFixed(0), 'ms');

// BEST: one reusable buffer, no per-iteration allocation at all
t = performance.now();
const reusable = Buffer.allocUnsafe(SIZE);
for (let i = 0; i < N; i++) { reusable[0] = 1; }
console.log('reused     :', (performance.now() - t).toFixed(0), 'ms');

Zero-filling (alloc) costs measurable time at scale, and not allocating at all wins biggest.
In a real reader you keep one read buffer and parse out of it.

Step 3 — backpressure on a real socket

// backpressure.js — a server that floods, a client that reads slowly
const net = require('net');

const server = net.createServer((socket) => {
  const payload = Buffer.allocUnsafe(64 * 1024); // 64 KB
  let sent = 0, blocked = 0;

  function pump() {
    while (sent < 2000) {
      const ok = socket.write(payload); // returns false when buffer is full
      sent++;
      if (!ok) {                       // BACKPRESSURE: stop and wait for drain
        blocked++;
        socket.once('drain', pump);     // resume only when flushed
        return;
      }
    }
    console.log(`done: sent ${sent} chunks, paused ${blocked} times for drain`);
    socket.end();
  }
  pump();
});

server.listen(8124, () => {
  const client = net.connect(8124);
  let received = 0;
  client.on('data', (d) => {
    received += d.length;
    client.pause();                       // simulate a slow consumer
    setTimeout(() => client.resume(), 1);
  });
  client.on('end', () => {
    console.log(`client received ${(received / 1024 / 1024).toFixed(1)} MB`);
    server.close();
  });
});
node backpressure.js
// done: sent 2000 chunks, paused 80+ times for drain
// client received 125.0 MB

The "paused for drain" count is the system protecting itself. Delete the if (!ok) branch and keep
writing regardless: memory usage climbs without bound because Node queues everything you hand it. That is the
bug behind most "my P2P node leaks memory under load" reports.

Experiments

6.6 Self-check & interview drill

« Day 5: Cryptographic Primitives Day 7: Profiling, Optimization & Integration »