feat: add security load test and regression tooling#462
feat: add security load test and regression tooling#462YashIIT0909 wants to merge 4 commits intocameri:mainfrom
Conversation
cameri
left a comment
There was a problem hiding this comment.
Please add documentation on how to use npm run test:load
There was a problem hiding this comment.
Pull request overview
Adds a new manual security/load-testing utility to reproduce and guard against WebSocket “zombie connection” heartbeat regressions, and wires it into the repo’s npm scripts.
Changes:
- Added
scripts/security-load-test.jsto generate zombie WebSocket connections and optionally flood the relay with valid signed Nostr events. - Added
npm run test:loadentry inpackage.jsonto run the new tooling.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| scripts/security-load-test.js | New load/security test script to simulate zombie clients + optional event spammer. |
| package.json | Adds test:load script to execute the new tool. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
scripts/security-load-test.js
Outdated
| const args = process.argv.slice(2).reduce((acc, arg, i, arr) => { | ||
| if (arg.startsWith('--')) acc[arg.slice(2)] = arr[i + 1]; | ||
| return acc; | ||
| }, {}); | ||
|
|
||
| const RELAY_URL = args.url || 'ws://localhost:8008'; | ||
| const TOTAL_ZOMBIES = parseInt(args.zombies || '5000', 10); | ||
| const SPAM_RATE = parseInt(args['spam-rate'] || '0', 10); |
There was a problem hiding this comment.
The CLI parsing logic treats every --flag as having a value at arr[i + 1] (even if the next token is another flag), which can lead to TOTAL_ZOMBIES / SPAM_RATE becoming NaN (e.g., --zombies --spam-rate 100). Consider using a more robust parser (or at least validating parseInt(...) results and exiting with a helpful message when values are missing/invalid).
| const args = process.argv.slice(2).reduce((acc, arg, i, arr) => { | |
| if (arg.startsWith('--')) acc[arg.slice(2)] = arr[i + 1]; | |
| return acc; | |
| }, {}); | |
| const RELAY_URL = args.url || 'ws://localhost:8008'; | |
| const TOTAL_ZOMBIES = parseInt(args.zombies || '5000', 10); | |
| const SPAM_RATE = parseInt(args['spam-rate'] || '0', 10); | |
| function parseCliArgs(argv) { | |
| const acc = {}; | |
| for (let i = 0; i < argv.length; i++) { | |
| const arg = argv[i]; | |
| if (!arg.startsWith('--')) continue; | |
| const key = arg.slice(2); | |
| const value = argv[i + 1]; | |
| if (value === undefined || value.startsWith('--')) { | |
| console.error(`Missing value for --${key}`); | |
| process.exit(1); | |
| } | |
| acc[key] = value; | |
| i++; | |
| } | |
| return acc; | |
| } | |
| function parseIntegerArg(value, defaultValue, flagName) { | |
| if (value === undefined) return defaultValue; | |
| const parsed = Number.parseInt(value, 10); | |
| if (Number.isNaN(parsed)) { | |
| console.error(`Invalid value for --${flagName}: ${value}. Expected an integer.`); | |
| process.exit(1); | |
| } | |
| return parsed; | |
| } | |
| const args = parseCliArgs(process.argv.slice(2)); | |
| const RELAY_URL = args.url || 'ws://localhost:8008'; | |
| const TOTAL_ZOMBIES = parseIntegerArg(args.zombies, 5000, 'zombies'); | |
| const SPAM_RATE = parseIntegerArg(args['spam-rate'], 0, 'spam-rate'); |
scripts/security-load-test.js
Outdated
|
|
||
| ws.on('open', () => { | ||
| console.log(`\n[SPAMMER] Connected. Flooding ${SPAM_RATE} events/sec...`); | ||
| setInterval(async () => { | ||
| const event = await createValidEvent(spammerPrivKey); | ||
| ws.send(JSON.stringify(['EVENT', event])); | ||
| spamSent++; | ||
| }, intervalMs); | ||
| }); | ||
|
|
||
| ws.on('close', () => { | ||
| console.log('[SPAMMER] Disconnected. Reconnecting...'); | ||
| setTimeout(startSpammer, 1000); | ||
| }); | ||
|
|
||
| ws.on('error', () => { }); |
There was a problem hiding this comment.
startSpammer() creates a setInterval that is never cleared. If the socket closes and startSpammer() is called again, the previous interval keeps firing and will attempt ws.send(...) on a closed socket, potentially spamming errors / leaking timers. Store the interval handle and clear it on close/error, and guard sends with ws.readyState === WebSocket.OPEN.
| ws.on('open', () => { | |
| console.log(`\n[SPAMMER] Connected. Flooding ${SPAM_RATE} events/sec...`); | |
| setInterval(async () => { | |
| const event = await createValidEvent(spammerPrivKey); | |
| ws.send(JSON.stringify(['EVENT', event])); | |
| spamSent++; | |
| }, intervalMs); | |
| }); | |
| ws.on('close', () => { | |
| console.log('[SPAMMER] Disconnected. Reconnecting...'); | |
| setTimeout(startSpammer, 1000); | |
| }); | |
| ws.on('error', () => { }); | |
| let spammerInterval = null; | |
| function clearSpammerInterval() { | |
| if (spammerInterval !== null) { | |
| clearInterval(spammerInterval); | |
| spammerInterval = null; | |
| } | |
| } | |
| ws.on('open', () => { | |
| console.log(`\n[SPAMMER] Connected. Flooding ${SPAM_RATE} events/sec...`); | |
| clearSpammerInterval(); | |
| spammerInterval = setInterval(async () => { | |
| if (ws.readyState !== WebSocket.OPEN) { | |
| return; | |
| } | |
| const event = await createValidEvent(spammerPrivKey); | |
| if (ws.readyState === WebSocket.OPEN) { | |
| ws.send(JSON.stringify(['EVENT', event])); | |
| spamSent++; | |
| } | |
| }, intervalMs); | |
| }); | |
| ws.on('close', () => { | |
| clearSpammerInterval(); | |
| console.log('[SPAMMER] Disconnected. Reconnecting...'); | |
| setTimeout(startSpammer, 1000); | |
| }); | |
| ws.on('error', () => { | |
| clearSpammerInterval(); | |
| }); |
| // Suppress the automatic internal pong handling | ||
| if (ws._receiver) { | ||
| ws._receiver.removeAllListeners('ping'); | ||
| ws._receiver.on('ping', () => { }); | ||
| } | ||
| ws.pong = function () { }; | ||
|
|
There was a problem hiding this comment.
This script reaches into ws private internals (ws._receiver) and monkey-patches ws.pong. Those are not part of the public ws API and can break across ws updates, making the load test flaky. Prefer a supported approach (e.g., using documented options/events to disable auto-pong, or at minimum feature-detect and fail fast with a clear error if the expected internals aren’t present).
| ws.on('error', (err) => { | ||
| errors++; | ||
| resolve(null); | ||
| }); | ||
|
|
||
| ws.on('message', () => { }); // Discard broadcast data | ||
| }); |
There was a problem hiding this comment.
On connection errors, the promise resolves null but the WebSocket instance isn't explicitly cleaned up (listeners removed / socket terminated). Under high connection counts this can leave resources around longer than necessary. Consider calling ws.terminate() (or ws.close() if applicable) and removing listeners in the error path before resolving.
scripts/security-load-test.js
Outdated
| * Simulates a combined Slowloris (Zombie) attack and an Event Flood attack. | ||
| * | ||
| * Features: | ||
| * 1. Zombie Connections: Opens connections, subscibes, and silences pongs. |
There was a problem hiding this comment.
Typo in the header comment: "subscibes" → "subscribes".
| * 1. Zombie Connections: Opens connections, subscibes, and silences pongs. | |
| * 1. Zombie Connections: Opens connections, subscribes, and silences pongs. |
| "cover:unit": "nyc --report-dir .coverage/unit npm run test:unit", | ||
| "docker:build": "docker build -t nostream .", | ||
| "pretest:integration": "mkdir -p .test-reports/integration", | ||
| "test:load": "node ./scripts/security-load-test.js", | ||
| "test:integration": "cucumber-js", |
There was a problem hiding this comment.
The PR description/checklist mentions documentation updates and added tests, but this change set only adds a new script and wires it into package.json. If docs/tests are intended, please include them (e.g., README/SECURITY.md mentioning npm run test:load, and/or automated coverage for the new regression tooling) or adjust the PR description/checklist to match the actual changes.
Description
This PR introduces a specialized security and load-testing tool, scripts/security-load-test.js. These assets are designed to reproduce, verify, and permanently safeguard the relay against memory leaks in the WebSocket heartbeat mechanism (specifically the "zombie connection" issue).
The tool has been integrated into package.json via the
npm run test:loadcommand to allow for easy execution during manual security audits.Related Issue
Fixes #429
Motivation and Context
While the primary heartbeat bug (eviction blocked by active subscriptions) has been addressed, the relay previously lacked a reliable way to verify that dead connections are actually being evicted from both the
wsclient set and the internalWeakMapadapters.This change is required to:
How Has This Been Tested?
The tooling was tested in a Linux environment using a 1-worker cluster configuration:
Screenshots
WeakMapwhile the socket remains in the server's active client set.Types of changes
Checklist: