Skip to content

Commit 1293b3b

Browse files
jdaltonjdalton
committed
docs: add external tools documentation
1 parent 0631001 commit 1293b3b

1 file changed

Lines changed: 192 additions & 0 deletions

File tree

‎docs/external-tools.md‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# External Tools
2+
3+
Socket CLI integrates with external security tools for scanning, analysis, and vulnerability detection. This document explains how tools are bundled and executed in different deployment modes.
4+
5+
## Deployment Modes
6+
7+
| Mode | Description | Tool Source |
8+
|------|-------------|-------------|
9+
| **SEA** | Standalone executable with bundled VFS | Tools pre-bundled at build time |
10+
| **npm CLI** | Installed via npm/pnpm/yarn | Tools downloaded at runtime |
11+
12+
## Tool Matrix
13+
14+
| Tool | Type | SEA Mode | npm CLI Mode |
15+
|------|------|----------|--------------|
16+
| @coana-tech/cli | npm | VFS (node_modules) | dlx download |
17+
| @cyclonedx/cdxgen | npm | VFS (node_modules) | dlx download |
18+
| opengrep | github-release | VFS (/snapshot/) | GitHub download |
19+
| python | github-release | VFS (/snapshot/) | GitHub download |
20+
| socket-basics | github-source | VFS (pre-installed) | N/A (SEA only) |
21+
| socket-patch | github-release | VFS (/snapshot/) | GitHub download |
22+
| socketsecurity | pypi | VFS (pre-installed via pip) | pip install |
23+
| sfw | hybrid | VFS (GitHub binary) | dlx (npm package) |
24+
| synp | npm | VFS (node_modules) | dlx download |
25+
| trivy | github-release | VFS (/snapshot/) | GitHub download |
26+
| trufflehog | github-release | VFS (/snapshot/) | GitHub download |
27+
28+
## Configuration
29+
30+
All tools are defined in `packages/cli/external-tools.json`:
31+
32+
```json
33+
{
34+
"tool-name": {
35+
"description": "Tool description",
36+
"type": "npm | github-release | pypi | github-source",
37+
"version": "1.0.0",
38+
"checksums": { ... }
39+
}
40+
}
41+
```
42+
43+
---
44+
45+
## SEA Mode (Standalone Executable)
46+
47+
SEA binaries contain all tools pre-bundled in a Virtual File System (VFS). Tools are extracted to a temp directory on first use.
48+
49+
### VFS Structure
50+
51+
```
52+
/snapshot/
53+
├── node_modules/ # npm packages with full dependency trees
54+
│ ├── @coana-tech/cli/
55+
│ ├── @cyclonedx/cdxgen/
56+
│ ├── @socketsecurity/sfw-bin/sfw
57+
│ └── synp/
58+
├── opengrep/ # Standalone binaries
59+
├── python/ # Python runtime + pre-installed packages
60+
│ └── lib/python3.11/site-packages/
61+
│ ├── socketsecurity/
62+
│ └── socket_basics/
63+
├── socket-patch/
64+
├── trivy/
65+
└── trufflehog/
66+
```
67+
68+
### Python Package Pre-bundling
69+
70+
Python packages (`socketsecurity`, `socket_basics`) are installed at **build time** into the bundled Python:
71+
72+
1. Build downloads `python-build-standalone` runtime
73+
2. Build runs `pip install socketsecurity==X.X.X` into bundled Python
74+
3. Build copies `socket-basics` source into site-packages
75+
4. VFS contains complete Python with packages pre-installed
76+
5. Runtime skips pip install (checks `import socketsecurity` first)
77+
78+
### VFS Extraction
79+
80+
Tools are extracted on first use to `~/.socket/_vfs/`:
81+
82+
```typescript
83+
// Detection
84+
if (isSeaBinary() && areExternalToolsAvailable()) {
85+
// Use VFS-extracted tool
86+
return spawnToolVfs(args, options)
87+
}
88+
```
89+
90+
---
91+
92+
## npm CLI Mode
93+
94+
When installed via npm, tools are downloaded at runtime.
95+
96+
### Download Locations
97+
98+
| Source | Cache Location |
99+
|--------|----------------|
100+
| npm dlx | `~/.socket/_dlx/{package}@{version}/` |
101+
| GitHub releases | `~/.socket/_dlx/github/{owner}/{repo}/{version}/` |
102+
| PyPI | `~/.socket/_dlx/pypi/{package}/{version}/` |
103+
| Python runtime | `~/.socket/_dlx/python/{version}-{tag}-{platform}-{arch}/` |
104+
105+
### Download Flow
106+
107+
```
108+
1. Check local path override (SOCKET_CLI_*_LOCAL_PATH env var)
109+
└── If set, use local binary directly
110+
111+
2. Check cache
112+
└── If cached and valid, use cached binary
113+
114+
3. Download
115+
├── npm packages: dlxPackage() from npm registry
116+
├── GitHub releases: downloadGitHubReleaseBinary()
117+
└── PyPI packages: downloadPyPIWheel()
118+
119+
4. Verify integrity
120+
└── SHA-256 checksum validation (required in production)
121+
122+
5. Extract and cache
123+
└── Save to ~/.socket/_dlx/
124+
```
125+
126+
---
127+
128+
## Security
129+
130+
### Checksum Verification
131+
132+
All downloads are verified with SHA-256 checksums defined in `external-tools.json`:
133+
134+
```json
135+
{
136+
"trivy": {
137+
"checksums": {
138+
"trivy_0.69.2_macOS-ARM64.tar.gz": "320c0e6af90b5733...",
139+
"trivy_0.69.2_Linux-64bit.tar.gz": "affa59a1e37d86e4..."
140+
}
141+
}
142+
}
143+
```
144+
145+
Checksums are **required** in production builds. Dev mode allows downloads without checksums for testing.
146+
147+
### Archive Extraction Safety
148+
149+
- Path traversal validation (no `../` escapes)
150+
- Symlink target validation (no escapes via symlinks)
151+
- Lock file protection against concurrent downloads
152+
153+
### Local Path Overrides
154+
155+
Environment variables for development/testing:
156+
157+
| Variable | Tool |
158+
|----------|------|
159+
| `SOCKET_CLI_CDXGEN_LOCAL_PATH` | cdxgen |
160+
| `SOCKET_CLI_COANA_LOCAL_PATH` | coana |
161+
| `SOCKET_CLI_PYCLI_LOCAL_PATH` | socketsecurity |
162+
| `SOCKET_CLI_SFW_LOCAL_PATH` | sfw |
163+
| `SOCKET_CLI_SOCKET_PATCH_LOCAL_PATH` | socket-patch |
164+
165+
---
166+
167+
## Implementation Files
168+
169+
| File | Purpose |
170+
|------|---------|
171+
| `external-tools.json` | Tool definitions, versions, checksums |
172+
| `src/utils/dlx/resolve-binary.mts` | Binary resolution logic |
173+
| `src/utils/dlx/spawn.mts` | Tool spawning (VFS + dlx) |
174+
| `src/utils/dlx/vfs-extract.mts` | VFS extraction utilities |
175+
| `src/utils/basics/spawn.mts` | Python-based tools (basics) |
176+
| `src/utils/basics/vfs-extract.mts` | Basics tools VFS extraction |
177+
| `src/env/*-version.mts` | Version getters (esbuild inlined) |
178+
| `src/env/*-checksums.mts` | Checksum getters (esbuild inlined) |
179+
180+
---
181+
182+
## Adding a New Tool
183+
184+
1. Add entry to `external-tools.json` with version and checksums
185+
2. Create `src/env/{tool}-version.mts` version getter
186+
3. Create `src/env/{tool}-checksums.mts` checksum getter (if applicable)
187+
4. Add resolve function in `src/utils/dlx/resolve-binary.mts`
188+
5. Add spawn functions in `src/utils/dlx/spawn.mts`:
189+
- `spawn{Tool}Vfs()` - VFS extraction path
190+
- `spawn{Tool}Dlx()` - Download path
191+
- `spawn{Tool}()` - Auto-detect wrapper
192+
6. Update build scripts to bundle tool in VFS (for SEA)

0 commit comments

Comments
 (0)