Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e244cfd
Remove test skip
dwoz Feb 3, 2023
ce8ff86
Add changelog for re-enabled test_interrupt test
Jun 9, 2026
194f930
Use KillMode=mixed in salt-minion.service unit
Jun 7, 2026
42a0aba
Fix process and file descriptor leaks in Salt Master
May 26, 2026
33ad623
Fix resource leaks in master, minion, and API
Jun 3, 2026
4691af9
Add nightly stress test workflow and monitoring tools
Jun 4, 2026
23bb5bc
Fix lint errors in monitoring scripts
Jun 4, 2026
4017532
Capture Prometheus metrics as build artifacts
Jun 4, 2026
0c3f53d
Remove __del__ methods from leak fixes
Jun 4, 2026
6f1fe5b
Fix file descriptor leak in ZeroMQSocketMonitor
Jun 5, 2026
633ea83
Fix CI failures from diff: Login.api, MasterMinion ctx, monitoring sc…
dwoz Jun 6, 2026
0ac93d9
Remove MWorker handler recycling in _handle_payload
dwoz Jun 6, 2026
5c17988
Restore worker_resource_backcount and fix test_publisher_mem flakiness
dwoz Jun 6, 2026
90dc6a3
Fix missing explicit teardown for LazyLoaders and bound methods
Jun 7, 2026
42cf499
Diagnose PAM eauth failure when master runs as non-root user
dwoz Jun 18, 2026
e977559
Fix minion SIGTERM blocked by resolve_dns retry loop
dwoz Jun 18, 2026
28d34f9
Make FakeRunner test doubles context managers
dwoz Jun 18, 2026
356218b
fix(logging): tolerate unset options dict in worker bootstrap
TeddyAndrieux May 7, 2026
d4e2e07
Fix multiple memory leaks across master, minion and IPC transport
dwoz Jun 20, 2026
1c4e53a
Stress-test infrastructure: ipc_write_buffer caps, debug-symbol Pytho…
dwoz Jun 20, 2026
7c5b94b
Merge remote-tracking branch 'origin/3006.x' into 3006leak
dwoz Jun 20, 2026
63e21ea
Fix test_master MaintenanceTestCase for _cached_mminion attribute
dwoz Jun 21, 2026
03c7d71
Restore IPCMessageSubscriber._read callback loop
dwoz Jun 21, 2026
baf789a
Fix two regressions from the leak-fix commit
dwoz Jun 22, 2026
c554f13
Merge pull request #69081 from TeddyAndrieux/bugfix/none-opts-when-se…
dwoz Jun 22, 2026
2404e38
Merge pull request #69473 from dwoz/fix/issue-64275
dwoz Jun 22, 2026
2d383b2
Merge pull request #69480 from dwoz/fix/issue-69466
dwoz Jun 22, 2026
7537c1c
Merge pull request #63627 from dwoz/test_inturrupt
dwoz Jun 22, 2026
d128283
Merge pull request #69384 from dwoz/fix/issue-68406
dwoz Jun 22, 2026
569db36
Fix MWorkerQueue ZMQ leak under sustained CLI churn
dwoz Jun 23, 2026
87b7a5b
Skip ZMQ identity scoping on minion daemon REQ sockets
dwoz Jun 23, 2026
f9f272c
Restrict stable ZMQ identity to salt CLI tools only
dwoz Jun 23, 2026
fa7e51f
Merge pull request #69227 from dwoz/3006leak
dwoz Jun 24, 2026
f2c62b1
Release v3006.26
saltbot-open Jun 24, 2026
7579b96
Fix PublishChannel leak in Minion.eval_master single-master path
dwoz Jun 6, 2026
90645bc
Merge pull request #69341 from dwoz/fix/issue-68901
dwoz Jun 25, 2026
0f74d15
Fix metadata grain KeyError on http.query error responses
dwoz Jun 18, 2026
5ff02d6
Merge branch '3006.x' into merge/3006.x/3007.x-26-06-25
dwoz Jun 26, 2026
4b3b864
Decode 4-byte length prefix in TCPPuller.handle_stream
dwoz Jun 26, 2026
9afaa38
Fix three unit / integration class regressions on the 3006.x → 3007.x…
dwoz Jun 27, 2026
d88c148
Clean up minion-2 and non-root-minion keys after factory teardown
dwoz Jun 28, 2026
3ae4154
Make RunnerClient/WheelClient event lazy to fix rest_tornado leaks
dwoz Jun 28, 2026
b8eb1b9
Destroy LocalClient/RunnerClient in BaseSaltAPIHandler.on_finish
dwoz Jun 28, 2026
28da4d8
Destroy rest_tornado EventListener in TestsTornadoHttpServer teardown
dwoz Jun 28, 2026
a1fd694
Revert ineffective rest_tornado leak fixes - wrong direction
dwoz Jun 28, 2026
6645017
Fix salt-key cleanup and broaden SyntaxWarning filter
dwoz Jun 28, 2026
2d119bb
Drop clean_modules() from LazyLoader.destroy() to fix rest_tornado ti…
dwoz Jun 28, 2026
b4451ec
Make LazyLoader.destroy() a no-op to restore 3007.x per-request latency
dwoz Jun 28, 2026
96f9d45
Fix test_performance container Python selection for salt:3005
dwoz Jun 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions .github/workflows/nightly-stress-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: Nightly Stress Test

on:
schedule:
- cron: '0 2 * * *' # 2 AM UTC
workflow_dispatch:
inputs:
duration:
description: 'Duration of the stress test (e.g., 30m, 1h)'
required: true
default: '30m'

jobs:
stress-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-

- name: Build and Start Environment
run: |
cd tests/monitoring
docker compose build
docker compose up -d
sleep 30 # Wait for initialization

- name: Verify Connections
run: |
docker exec salt-master salt '*' test.ping

- name: Run Aggressive Stress Test
run: |
cd tests/monitoring
chmod +x stress_test.sh stress_api.sh
# Run in background and wait for defined duration
./stress_test.sh &
STRESS_PID=$!

# Default to 30m if not workflow_dispatch
DURATION="${{ github.event.inputs.duration || '30m' }}"
echo "Running stress test for $DURATION..."

# Use sleep with suffix support (m, h)
sleep $DURATION

echo "Stopping stress test..."
pkill -P $STRESS_PID || true
kill $STRESS_PID || true

- name: Analyze Results
run: |
cd tests/monitoring
# Give Prometheus a moment to finish scraping the final points
sleep 30
python3 analyze_stats.py

- name: Snapshot Metrics
if: always()
run: |
# Stop containers to ensure data is flushed to disk
cd tests/monitoring
docker compose stop prometheus
sudo tar -czf ../../prometheus-data.tar.gz ./prometheus_data

- name: Collect Logs on Failure
if: failure()
run: |
mkdir -p artifacts
docker logs salt-master > artifacts/salt-master.log
docker logs salt-minion-1 > artifacts/salt-minion-1.log
cp monitoring/event_log.txt artifacts/ || true

- name: Upload Artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: stress-test-results
path: |
artifacts/
prometheus-data.tar.gz
319 changes: 319 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion changelog/30690.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/30971.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/38551.fixed.md

This file was deleted.

9 changes: 0 additions & 9 deletions changelog/41347.added.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/52605.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/52793.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/53190.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/55561.fixed.md

This file was deleted.

4 changes: 0 additions & 4 deletions changelog/57377.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/57754.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/57847.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/57848.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/57951.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/60276.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/60877.fixed.md

This file was deleted.

4 changes: 0 additions & 4 deletions changelog/61974.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/61983.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/62061.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/62732.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/63700.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/64505.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/64915.fixed.md

This file was deleted.

1 change: 1 addition & 0 deletions changelog/65184.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed the EC2/cloud metadata grain crashing with ``KeyError: 'headers'`` when ``salt.utils.http.query`` returns an error response (4xx/5xx with a body, e.g. when the IMDS rejects a recursive sub-path lookup). Since 3006.3 the tornado backend has populated ``body`` on HTTPError without also populating ``headers``; the grain now treats the missing ``headers`` key as "no Content-Type information" instead of letting the lookup blow up the whole grain load.
1 change: 0 additions & 1 deletion changelog/65301.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/65317.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/65360.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/65516.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/65531.added.md

This file was deleted.

8 changes: 0 additions & 8 deletions changelog/65709.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/65870.fixed.md

This file was deleted.

9 changes: 0 additions & 9 deletions changelog/66148.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/67061.fixed.md

This file was deleted.

7 changes: 0 additions & 7 deletions changelog/67716.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68103.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68105.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68110.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68115.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68129.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68137.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68181.fixed.md

This file was deleted.

9 changes: 0 additions & 9 deletions changelog/68208.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68210.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68214.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68227.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68248.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68269.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68273.fixed.md

This file was deleted.

5 changes: 0 additions & 5 deletions changelog/68293.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68326.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68351.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68353.fixed.md

This file was deleted.

1 change: 0 additions & 1 deletion changelog/68419.fixed.md

This file was deleted.

Loading
Loading