LSL Update

.github/workflows/analysis.yml

@@ -0,0 +1,39 @@
+name: Analysis (ERP round-trip)
+
+# Validates that a CSV recorded by the app yields a correct ERP, end to end.
+# Runs the app's real analysis code (src/renderer/utils/webworker/utils.py)
+# against NATIVE MNE — no Pyodide boot — so it's fast and pins down the
+# data-integrity contract: event_id values must equal the numeric codes in the
+# CSV Marker column, and the sfreq heuristic must survive timestamp jitter.
+#
+# Note: the app runs this same Python under Pyodide/WASM. A slower
+# Pyodide-fidelity smoke job (TODO) will guard against native/WASM divergence.
+
+on:
+  push:
+    branches:
+      - '**'
+  pull_request:
+
+jobs:
+  analysis:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out Git repository
+        uses: actions/checkout@v4
+
+      - name: Install Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+          cache: 'pip'
+          cache-dependency-path: tests/analysis/requirements.txt
+
+      - name: Install native MNE + test deps
+        run: pip install -r tests/analysis/requirements.txt
+
+      - name: ERP round-trip validation tests
+        env:
+          MPLBACKEND: agg
+        run: python -m pytest tests/analysis -v

.github/workflows/device.yml

@@ -0,0 +1,35 @@
+name: Device & LSL Integration
+
+# Fast, focused gate for the EEG device + Lab Streaming Layer connectivity
+# tests. These mock the native liblsl/Neurosity layers with an in-memory "LSL
+# network" (see src/main/lsl/__tests__/fakeLslNetwork.ts), so the job needs no
+# liblsl on the runner and can skip the heavy Pyodide postinstall entirely via
+# --ignore-scripts. The full cross-OS suite still runs separately in test.yml.
+
+on:
+  push:
+    branches:
+      - '**'
+  pull_request:
+
+jobs:
+  integration:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out Git repository
+        uses: actions/checkout@v4
+
+      - name: Install Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: 'npm'
+
+      # --ignore-scripts skips electron-builder app-deps + Pyodide/MNE downloads;
+      # the connectivity tests mock every native module, so none are required.
+      - name: Install dependencies (no native/postinstall steps)
+        run: npm ci --ignore-scripts
+
+      - name: Device & LSL integration tests
+        run: npm run device-integration

.github/workflows/release.yml

@@ -31,7 +31,4 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           USERNAME: ${{ secrets.USERNAME }}
           PASSWORD: ${{ secrets.PASSWORD }}
-          CLIENT_ID: ${{ secrets.CLIENT_ID }}
-          CLIENT_SECRET: ${{ secrets.CLIENT_SECRET }}
-          LICENSE_ID: ${{ secrets.LICENSE_ID }}
         run: npm run package-ci

.gitignore

@@ -26,5 +26,15 @@ out
 dist
 
 .idea
-keys.js
 src/renderer/utils/webworker/src
+
+# Pyodide runtime + package wheels (downloaded or extracted locally; see docs/pyodide-in-electron-vite.md)
+src/renderer/utils/pyodide/src
+
+# Local git worktrees (not part of the repo tree)
+.worktrees/
+
+# Python venv for native analysis tests (tests/analysis)
+.venv-test/
+.venv/
+__pycache__/

.llms/learnings.md

@@ -9,6 +9,58 @@ Format: brief heading + explanation + (optional) relevant file paths.
 
 <!-- Add entries below this line -->
 
+## Markers: device-agnostic injection via the EEGDriver interface
+
+Marker injection used to be Muse-only and lived in the UI (`RunComponent.eventCallback`
+called `injectMuseMarker` directly). Neurosity recordings therefore had an all-zero
+Marker column and could not yield ERPs. Now every first-party backend implements a
+shared `EEGDriver` interface (`src/renderer/utils/eeg/types.ts`) with `injectMarker`,
+registered in `src/renderer/utils/eeg/index.ts`. `deviceEpics` resolves drivers via
+`getDriver(deviceType)` instead of branching on MUSE/NEUROSITY, and the UI calls the
+device-agnostic `injectMarker()` dispatcher (delegates to the active driver, set on
+connect via `setActiveDriver`). A new device cannot ship without a marker path — the
+interface won't compile without it. LSL inlet is intentionally NOT in the registry
+(separate external-recorder mode; `injectMarker` no-ops for it). Neurosity has no
+native marker stream, so its `injectMarker` attaches the code to the next emitted
+sample (one epoch of latency); Muse merges via muse-js eventMarkers + `synchronizeTimestamp`.
+
+## Marker codes are numeric, and the analysis event_id must match them
+
+Markers passed to `callbackForEEG` (and into the CSV) are **numeric** EVENTS codes
+(`stimulus.type`, e.g. STIMULUS_1 = 1), not strings — see the experiment files'
+`callbackForEEG(this.parameters.congruent === 'yes' ? 1 : 2)` etc. The CSV Marker
+column carries these codes; MNE `find_events` reads them off the last (`stim`) channel.
+The bug: `pyodideEpics.loadEpochsEpic` built the MNE `event_id` map as
+`{stimulus.title: arrayIndex}` (0-based), which did not match the 1-based codes in the
+data — so code-2 epochs matched no event_id and MNE raised "No matching events". Fixed
+by `buildMarkerRegistry` (`src/renderer/utils/eeg/markerRegistry.ts`), the single source
+of truth used by BOTH collection (CSV codes + `-events.json` sidecar) and analysis
+(event_id). Also a latent bug: `epochEvents` interpolated an undefined `reject` as
+`reject = undefined` (Python NameError) — now coerced to `None`.
+
+## Testing the analysis pipeline against native MNE (no Pyodide)
+
+`webworker/utils.py` is testable outside Pyodide: `load_data(csv_strings=...)` skips the
+`js.csvArray` global, and epoching is factored into `get_raw_epochs(raw, event_id,
+tmin, tmax, ...)` which `webworker/index.ts` `epochEvents` also calls — so the in-app
+analysis and the tests share one implementation (no drift). `tests/analysis/` runs the
+real `utils.py` against native MNE via pytest (`conftest.py` puts the webworker dir on
+`sys.path`; `MPLBACKEND=agg` because utils.py imports pyplot at module load). The golden
+test (`test_erp_roundtrip.py` + `synthetic.py`) plants a P300-like bump on one condition
+and asserts it's recovered after `load_data → filter(1,30,'iir') → get_raw_epochs →
+average`, plus sfreq-under-jitter and dropped-sample cases. CI: `.github/workflows/
+analysis.yml` (Python job, `pip install -r tests/analysis/requirements.txt`). The app
+runs the same Python under Pyodide/WASM — a Pyodide-fidelity smoke job is still a TODO.
+
+## Testing device + LSL connectivity without native deps
+
+Device/LSL connectivity is integration-tested with the native layers fully mocked, so the tests run anywhere (incl. CI on all OSes) with **no liblsl / koffi / SDK installed**:
+- `src/main/lsl/__tests__/fakeLslNetwork.ts` — an in-memory fake of `node-labstreaminglayer`: creating a `StreamOutlet` registers its `StreamInfo` on a shared broker; `resolveStreams()` lists live outlets; a `StreamInlet`'s `pullChunk()` reads what its outlet pushed since the last pull. This lets `outlets.ts` → `inlets.ts` be tested as a real round trip. Inject it by `vi.mock('../native')` then `vi.mocked(loadLSL).mockReturnValue(fake.module as unknown as ...)`. `outlets.ts`/`inlets.ts` also import `electron-log`, so `vi.mock('electron-log', () => ({ default: { info/warn/error: vi.fn() } }))` is required. The inlet poll loop is a `setInterval`, so use `vi.useFakeTimers()` + `vi.advanceTimersByTime()`.
+- Renderer drivers mock `window.electronAPI` (capture the `onLSLInletData`/`onLSLInletDisconnected` handlers to "push" inlet epochs) and `@neurosity/sdk`. The Neurosity mock's `Neurosity` must be `new`-able — `neurosity.ts` does `new Neurosity(...)` — so define a plain `function Neurosity(){ return client }` inside `vi.hoisted` (arrow fns aren't constructable) and `vi.mock('@neurosity/sdk', () => ({ Neurosity: h.Neurosity }))`.
+- `lslBridge.ts` probes availability at module load via a top-level `isLSLAvailable()` promise; to test the gate, set `window.electronAPI.isLSLAvailable` before `vi.resetModules()` + dynamic `await import('../lslBridge')`, then flush a microtask.
+
+**CI**: `npm run device-integration` (script targets `src/main/lsl src/renderer/utils/eeg`) runs as its own `.github/workflows/integration.yml` job with `npm ci --ignore-scripts` — since every native module is mocked, it skips the slow Pyodide/MNE postinstall and needs no liblsl. The full cross-OS suite still runs these too via `npm test` in `test.yml` (whose `Test` step runs before the lint step).
+
 ## Styling System (post Phase 4 migration)
 
 The app uses shadcn/ui + Tailwind CSS. CSS modules have been fully removed. Key conventions:
@@ -21,23 +73,23 @@ The app uses shadcn/ui + Tailwind CSS. CSS modules have been fully removed. Key
 - **Background gradient** used on all main screens: `bg-gradient-to-b from-[#f9f9f9] to-[#f0f0ff]`
 - **`@radix-ui/react-select`** is installed for the shadcn Select component
 
-## Pyodide Asset Serving — Vite SPA Fallback Problem
+## Pyodide Asset Serving — Custom `pyodide://` Protocol
 
-Vite's `historyApiFallback` returns `index.html` for **all** `fetch()` requests from web workers, including `/@fs/` and `publicDir` paths. This breaks Pyodide's package loading entirely.
+Vite's `historyApiFallback` returns `index.html` for **all** `fetch()` requests from web workers, breaking Pyodide's package loading. We solved this with a custom Electron protocol scheme registered in `src/main/index.ts` (`protocol.handle('pyodide', ...)`). The web worker uses `pyodide://host` as `PYODIDE_ASSET_BASE` and the handler resolves paths against the local filesystem — no HTTP socket required, works identically in dev and prod.
 
-**Solution (two-part):**
-1. A custom Vite middleware in `vite.config.ts` intercepts `/pyodide/` and `/packages/` requests before the SPA fallback and serves them directly from `src/renderer/utils/webworker/src/`.
-2. An Electron `http` server on **port 17173** (started in `src/main/index.ts`) serves the same directory. Web workers use `http://127.0.0.1:17173` as `PYODIDE_ASSET_BASE`. This is the authoritative path — web worker `fetch()` calls bypass Vite entirely.
+**Filesystem roots resolved by the handler:**
+- Dev: `src/renderer/utils/webworker/src/`
+- Prod: `process.resourcesPath/pyodide/` — `package.json` `extraResources` copies `webworker/src/` to a folder named `pyodide`. The protocol handler must match this destination name (mismatched once and broke prod entirely).
 
-Port 17173 is hardcoded in both `src/main/index.ts` and `src/renderer/utils/webworker/webworker.js` and in the CSP (`src/renderer/index.html`).
+**`indexURL` is required in prod, not just `packageBaseUrl`.** In dev, `pyodide.mjs` is imported via Vite's `?url` from `node_modules/pyodide/`, and the runtime files (`pyodide.asm.wasm`, `python_stdlib.zip`) load via `import.meta.url`-relative fetch — siblings live alongside it in node_modules. In prod, Vite bundles `pyodide.mjs` into `out/renderer/assets/` *without* its siblings, so `import.meta.url` resolution fails. Setting `indexURL: '${PYODIDE_ASSET_BASE}/pyodide/'` routes runtime fetches through the protocol handler. Set both `packageBaseUrl` (for `.whl` files via `loadPackage`) and `indexURL` (for the runtime).
 
 **Other Pyodide loading gotchas:**
 - `pyodide.mjs` must be loaded via dynamic `import()` (not `fetch()`), using a `?url` Vite import — `import()` bypasses the SPA fallback, `fetch()` does not
 - The lock file is embedded via `?raw` and wrapped in a `Blob` + `createObjectURL` to avoid an HTTP fetch
-- Use `packageBaseUrl` (not `indexURL`) to tell Pyodide where to find `.whl` files; `indexURL` is for WASM/stdlib
 - `checkIntegrity: false` is required — SHA256 hashes in the npm lock file don't match CDN-downloaded wheels
 - Workers must be created with `type: 'module'` (Pyodide 0.26+ ships `pyodide.mjs` as ESM)
 - `optimizeDeps.exclude: ['pyodide']` in `vite.config.ts` prevents Vite from pre-bundling it
+- `micropip.install()` only accepts `http://`, `https://`, `emfs://`, and relative paths — it rejects custom schemes like `pyodide://`. Workaround: JS-fetch each `.whl` via the protocol handler, write into Pyodide's emscripten FS at `/tmp/`, then install via `emfs:///tmp/...`.
 
 ## Pyodide Offline Package Installation (InstallMNE.mjs)
 
@@ -81,6 +133,23 @@ In dev: use `/@fs<absPath>` (Vite's `/@fs/` serving). In prod: use `file://<absP
 `prepareNested` (in `flow/util/nested.js`) sets IDs on cloned loop components via `c.id = [parent.id, i].join('_')` — this sets the **component's own property**, NOT `c.options.id`. The options proxy reads through `rawOptions`, which never has an `id` for template-cloned components (the JSON template has no explicit `id` field).
 
 Any hook function (e.g. `initResponseHandlers` in `src/renderer/utils/labjs/functions.ts`) that needs the component ID must use `this.id`, not `this.options.id`. Using `this.options.id` will always be `undefined` for loop-cloned components, causing silent early returns and broken behavior (e.g. keydown handlers never installed).
+## liblsl on Apple Silicon
+
+`node-labstreaminglayer@0.3.0` ships only an **x86_64** `liblsl.dylib` in its `prebuild/` directory — the package has no arm64 build and was last updated 2025-08. Loading it on Apple Silicon throws `Failed to load shared library: ... (mach-o file, but is an incompatible architecture)`.
+
+**Fix**: install liblsl via Homebrew (`brew install labstreaminglayer/tap/lsl`), then `internals/scripts/patchDeps.mjs` symlinks `/opt/homebrew/Cellar/lsl/<version>/Frameworks/lsl.framework/Versions/A/lsl` over the bundled x86_64 dylib on every install/dev run. The patch is a no-op on x86_64 macs and on Linux/Windows (which ship usable `.so`/`.dll` in the same prebuild dir).
+
+Alternatives evaluated and rejected: `@neurodevs/node-lsl` and `@neurodevs/ndx-native` both require the same Homebrew install (they hard-code `/opt/homebrew/Cellar/lsl/...` paths) and have a much different async/worker-thread API that would force a substantial rewrite.
+
+## LSL is optional — load the native module lazily, never statically
+
+`node-labstreaminglayer` `dlopen`s liblsl via koffi **at require time**. A static `import … from 'node-labstreaminglayer'` in the main process therefore runs that dlopen during module evaluation at startup — so a missing/incompatible liblsl (e.g. Apple Silicon without the Homebrew build) crashes the *entire app* on launch, even for Muse-only users who don't need LSL at all.
+
+LSL is an advanced, opt-in capability: Muse and Neurosity connect via Web Bluetooth and record to CSV without it. So the native module must load lazily and fail soft:
+- `src/main/lsl/native.ts` does a guarded `require('node-labstreaminglayer')` in try/catch (memoized), exposing `loadLSL()` (module | null) and `isLSLAvailable()`. `outlets.ts`/`inlets.ts` use **type-only** imports + `loadLSL()` at call time, no-opping when null.
+- Feature-detect in the renderer via the `lsl:isAvailable` IPC: `ConnectModal` hides the "External LSL stream" option and `lslBridge` no-ops `sendEpoch`/`sendMarker` when unavailable (avoids IPC spam from first-party devices).
+
+Build note: with `module: ESNext` source but CommonJS main output, a guarded `require(...)` of an externalized dep type-checks (global `require` from `@types/node`) and stays a `require` in `out/main/index.js` (electron-vite externalizes it) — confirmed lazy, not bundled. Do **not** revert to a static import.
 
 ## Pre-existing TypeScript errors (do not treat as regressions)
 

README.md

@@ -26,6 +26,16 @@
 
 > **Note:** `npm install` downloads ~300 MB of Pyodide WASM files on first run. This is expected and only happens once.
 
+### macOS (Apple Silicon) — install liblsl
+
+The `node-labstreaminglayer` npm package only ships an x86_64 `liblsl.dylib`, so arm64 Macs (M1/M2/M3/M4) need an arm64 build of liblsl from Homebrew. The dev script automatically symlinks the Homebrew binary into `node_modules/` on every install.
+
+```bash
+brew install labstreaminglayer/tap/lsl
+```
+
+If you skip this step, `npm run dev` will fail at startup with `Failed to load shared library: ... incompatible architecture`. Intel Macs, Linux, and Windows do not need this step — the bundled binaries work as-is.
+
 ## Installing from Source (for developers)
 
 1. Clone the repo: