Skip to content

Add web-client pointer-lock mouse capture and serial bridge to MacFriends Arduino#88

Open
h4rm0n1c wants to merge 12 commits intomainfrom
codex/implement-serial-communication-with-arduino
Open

Add web-client pointer-lock mouse capture and serial bridge to MacFriends Arduino#88
h4rm0n1c wants to merge 12 commits intomainfrom
codex/implement-serial-communication-with-arduino

Conversation

@h4rm0n1c
Copy link
Copy Markdown
Owner

Motivation

  • Enable the web UI to drive the MacFriends Arduino ADB core so browser pointer motion and clicks control the Macintosh via the existing Arduino firmware.
  • Keep the transport byte-for-byte compatible with the MacFriends host client so firmware expectations are preserved (magic=123, updateType/mouse layout, dx/dy clamped).

Description

  • Add AdbSerialBridge to client_web/src/ebd_ipkvm_web/app.py which discovers /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_*-if00-port0 by default (override via ADB_SERIAL_PORT), opens it at 115200, and exposes connect, send_mouse, and close helpers.
  • Wire the bridge into SessionManager so the serial device is opened on session start and closed on session stop/disconnect, and handle incoming WebSocket mouse_input messages by forwarding them to the serial bridge.
  • Implement browser pointer-lock capture in client_web/src/ebd_ipkvm_web/static/index.html to accumulate relative dx/dy and left-button state and send mouse_input over the WebSocket (safety mouse-up on unlock).
  • Document the serial packet format and usage in client_web/README.md and docs/protocol/adb.md, and add brief entries to docs/agent/log.md, docs/agent/decisions.md, and docs/agent/notes.md describing the change and pointer-lock caveat.

Testing

  • Created an editable venv and installed dependencies with pip install -e . which succeeded.
  • Compiled the web package with python -m compileall src/ebd_ipkvm_web and successfully imported create_app via python -c "from ebd_ipkvm_web.app import create_app; app=create_app()".
  • Validated the serial packet layout size via struct.calcsize('<bBbbbBBB') to confirm the runtime packet length.
  • Ran the web server (python -m ebd_ipkvm_web) and performed a browser smoke check with Playwright to render the UI (server started cleanly and screenshot captured).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant