The net Module, TCP Servers, Buffers, and Binary Data

The net module provides the low-level TCP/IPC networking API that all higher-level modules (http, https, tls) are built on. Understanding TCP sockets gives you the ability to build custom protocols, implement bidirectional communication without HTTP overhead, create inter-process communication channels, and debug network issues at the protocol level. Buffers and encodings are the memory abstraction for binary data in Node.js — understanding how Buffer works, when to use different encodings, and how to avoid common string/binary conversion mistakes is essential for any network or file system work.

Buffer vs String vs ArrayBuffer #

Complete net Module and Buffer Examples #

const net           = require('net');
const { StringDecoder } = require('string_decoder');

// ── TCP Echo Server ───────────────────────────────────────────────────────
const echoServer = net.createServer({ allowHalfOpen: false }, socket => {
    console.log(`Client connected: ${socket.remoteAddress}:${socket.remotePort}`);

    socket.setEncoding('utf8');  // decode incoming bytes as UTF-8
    socket.setTimeout(30000);    // 30-second idle timeout

    socket.on('data', data => {
        console.log('Received:', data);
        socket.write(data);       // echo back
    });

    socket.on('end', () => {
        socket.end();             // send FIN to client
    });

    socket.on('timeout', () => {
        socket.destroy();         // close idle connections
    });

    socket.on('error', err => {
        console.error('Socket error:', err.message);
    });
});

echoServer.listen(9000, '127.0.0.1', () => {
    console.log('TCP echo server on port 9000');
});

// ── Length-prefix message framing ────────────────────────────────────────
// Protocol: [4 bytes: message length as uint32 BE][N bytes: message body]

class FramedSocket {
    constructor(socket) {
        this.socket    = socket;
        this._buffer   = Buffer.alloc(0);
        this.onMessage = null;

        socket.on('data', chunk => this._onData(chunk));
    }

    _onData(chunk) {
        this._buffer = Buffer.concat([this._buffer, chunk]);

        while (this._buffer.length >= 4) {
            const msgLen = this._buffer.readUInt32BE(0);  // read length prefix

            if (this._buffer.length < 4 + msgLen) break;  // incomplete message

            const msg = this._buffer.slice(4, 4 + msgLen);
            this._buffer = this._buffer.slice(4 + msgLen);  // consume message

            if (this.onMessage) this.onMessage(msg);
        }
    }

    send(data) {
        const body   = Buffer.isBuffer(data) ? data : Buffer.from(data);
        const header = Buffer.alloc(4);
        header.writeUInt32BE(body.length, 0);  // 4-byte length prefix
        this.socket.write(Buffer.concat([header, body]));
    }
}

// ── Buffer operations ─────────────────────────────────────────────────────

// Create buffers
const zero    = Buffer.alloc(16);                          // 16 zero bytes
const unsafe  = Buffer.allocUnsafe(16);                    // 16 uninit bytes — overwrite immediately
const fromHex = Buffer.from('deadbeef', 'hex');            // from hex string
const fromStr = Buffer.from('Hello, Node.js', 'utf8');     // from string
const fromArr = Buffer.from([0x48, 0x65, 0x6c, 0x6c, 0x6f]); // from array

// Reading/writing numeric values
const buf = Buffer.alloc(16);
buf.writeUInt32BE(0xdeadbeef, 0);     // big-endian uint32 at offset 0
buf.writeFloatLE(3.14159, 4);         // little-endian float at offset 4
buf.writeBigInt64LE(9007199254740991n, 8); // little-endian int64 at offset 8

console.log(buf.readUInt32BE(0).toString(16));  // 'deadbeef'
console.log(buf.readFloatLE(4).toFixed(5));     // '3.14159'

// Encoding conversions
const original = 'Hello, 世界';
const utf8Buf  = Buffer.from(original, 'utf8');
const base64   = utf8Buf.toString('base64');
const restored = Buffer.from(base64, 'base64').toString('utf8');
console.log(original === restored);  // true

// ── StringDecoder for UTF-8 safe chunk decoding ───────────────────────────
const decoder = new StringDecoder('utf8');

// Simulated chunked UTF-8 where a multi-byte char is split across chunks
const full    = Buffer.from('Hello 世界', 'utf8');
const chunk1  = full.slice(0, 8);   // may split in middle of '世' (3-byte char)
const chunk2  = full.slice(8);

console.log(decoder.write(chunk1) + decoder.write(chunk2));  // 'Hello 世界' — correct
// vs Buffer.toString(): chunk1.toString() + chunk2.toString() = 'Hello ??界' — garbled

// ── IPC via Unix domain socket ────────────────────────────────────────────
// Faster than TCP for same-machine process communication
const ipcServer = net.createServer(socket => {
    socket.on('data', data => {
        const msg = JSON.parse(data.toString());
        // handle IPC message
        socket.write(JSON.stringify({ ok: true, received: msg }));
    });
});

ipcServer.listen('/tmp/taskmanager.sock', () => {
    console.log('IPC server listening on Unix socket');
});

How It Works #

Step 1 — TCP Is a Stream Protocol — Framing Is Your Responsibility #

TCP delivers a continuous stream of bytes with no message boundaries. If you call socket.write('hello') twice, the receiver might get 'hellohe' and 'llo' in two reads — or 'hellohello' in one. A framing protocol (length prefix, delimiter-based, or fixed-size) is required to identify where one message ends and the next begins. The length-prefix pattern is the most efficient for binary protocols.

Step 2 — Buffer.concat() Reassembles Fragmented Messages #

The FramedSocket accumulates incoming chunks in this._buffer using Buffer.concat(). It only emits a complete message when the buffer contains at least the 4-byte header plus the declared message length. If not enough data has arrived, it returns and waits for the next data event. This is the standard accumulation pattern for any streaming binary protocol.

Step 3 — Numeric Encoding Requires Explicit Byte Order #

When writing numbers into a Buffer for network transmission, you must choose between big-endian (BE) and little-endian (LE) byte order. Network protocols typically use big-endian (network byte order). Mismatched endianness is a common bug where values are silently read as wrong numbers. The readUInt32BE/writeUInt32BE methods enforce explicit byte order — never assume platform endianness.

Step 4 — StringDecoder Handles Multi-Byte Character Boundaries #

UTF-8 encodes some characters using 2–4 bytes. If a TCP chunk boundary falls inside a multi-byte character, converting the chunk to a string with .toString('utf8') produces the replacement character (U+FFFD) for the partial sequence. StringDecoder buffers the partial bytes internally and includes them in the next write() call, producing correct output across chunk boundaries.

Step 5 — Unix Domain Sockets Are Faster Than TCP for Local IPC #

Unix domain sockets use the filesystem as an address rather than an IP:port. They bypass the full TCP/IP stack — no IP routing, no port binding, no TCP handshake — making them 2–4x faster for same-machine process communication. Node.js’s Cluster and Worker Threads IPC both use Unix domain sockets internally. For custom IPC (microservices on the same host, sidecar processes), Unix sockets are the performant choice.

Quick Reference #