child_process: spawn Streaming vs exec Buffer vs fork IPC
- Last updated
- Prerequisites:
- Node.js streams and events
- Understanding of OS processes and IPC
- nodejs
- child-process
- spawn
- exec
- fork
- ipc
Read at your depth
The practical view
child_process runs OS processes: const { spawn, exec, execFile, fork } = require('node:child_process'). spawn(command, args) streams stdout/stderr (returning ChildProcess with .stdout/.stderr streams) — best for large output, no shell by default. exec(command, cb) runs through a shell and buffers all output (maxBuffer default 1MB, can OOM on large output) — best for simple commands with small output. execFile(file, args, cb) runs a binary directly (no shell). fork(modulePath) spawns a Node child with an IPC channel: child.send(msg) / process.on('message') for structured messaging. Use spawn for streams/large data, exec for quick shell commands, fork for Node-to-Node worker processes.
The same idea in other frameworks
N/A — Node.js runtime concern.N/A — Node.js concern.N/A — Node.js concern.N/A — Node.js concern.Legacy vs modern
exec for large output vs spawn streaming
exec buffers everything and can blow maxBuffer; spawn streams, so output size doesn't matter.
exec('cat hugefile.log', (err, stdout) => {
console.log(stdout.length); // ❌ OLD: buffers the whole fileconst child = spawn('cat', ['hugefile.log']); // ✅ NEW: streams output
child.stdout.on('data', (chunk) => processLine(chunk)); // ✅ NEW: incrementalInterview gotchas
Context
This tests the shell-interpretation boundary — interviewers want the sh -c semantics and the injection vector that exec introduces when command strings include user input.
The mechanical answer
exec runs the command through a shell (/bin/sh -c on POSIX): the string is interpreted by the shell, so pipes, globs, and environment expansions work, and you can pass a single command string. execFile runs the binary DIRECTLY with an argument array, no shell — arguments are passed literally, so shell metacharacters have no special meaning. The security consequence: with exec, if the command string includes user input (exec('rm -rf ' + userInput)), the user can inject shell syntax ('; rm -rf /' or '$(...)') and execute arbitrary commands. execFile passes the input as a literal argument, so injection is impossible (the binary receives it verbatim). The guidance: use execFile (or spawn with an args array) for anything involving untrusted input; use exec only for fixed, trusted command strings.
Trap
The naive answer is 'exec is fine if I quote the input'. Quoting is fragile and easily bypassed; the correct fix is no shell at all (execFile/spawn with an args array). Another trap: thinking execFile still interprets shell syntax — it doesn't, which is the point. Candidates who describe the sh -c boundary and the args-array pattern show real security awareness.
Context
This probes the Node-to-Node communication channel — interviewers want the message-passing model (send/on('message')), the serialization, and the process-model implications.
The mechanical answer
fork is spawn specialized for Node children: it launches a new Node process running the given module and automatically creates an IPC channel (a pair of connected pipes/sockets). The parent calls child.send(message) and the child receives it via process.on('message'); the child sends back with process.send(message) and the parent listens via child.on('message'). Messages are serialized with the V8 serializer (structured-clone-like: supports most JS values, but not functions or class instances with methods). The child is a completely separate process with its own V8 heap and event loop — it does not share memory (unlike worker_threads). fork is the classic way to split CPU work into a separate process or run a long-lived helper that communicates incrementally.
Trap
The naive answer is 'fork is a lighter worker'. It's a full separate OS process (like spawn) plus an IPC channel — heavier than a worker_thread. Another trap: sending non-serializable values (functions, circular structures beyond the serializer's support) — they fail or get mangled. Candidates who describe the IPC channel and the serialization boundary show real understanding.