@@ -26,11 +26,12 @@ It combines direct input execution, multiple locator strategies, CLI and MCP int
2626- MCP server for AI tool calling
2727- YAML case runner for repeatable automation flows
2828- Skill definitions and references for AI-assisted execution
29+ - Structured ` doctor ` diagnostics with remediation hints for local setup issues
2930
3031## Current Platform Status
3132
3233- Windows: primary implementation, real execution and smoke tested
33- - macOS: MVP driver implemented, requires Accessibility / Automation / Screen Recording permissions
34+ - macOS: MVP driver implemented, with structured permission diagnostics, title-plus-geometry window matching, and safer ` click_uia ` pre-checks
3435- Linux X11: MVP driver implemented, depends on ` wmctrl ` , ` xdotool ` , screenshot helpers, and optional AT-SPI tooling
3536- Linux Wayland: compatibility layer, helper-tool dependent and not yet full parity
3637
@@ -57,10 +58,18 @@ It combines direct input execution, multiple locator strategies, CLI and MCP int
5758``` powershell
5859$env:PYTHONPATH='src'
5960python -m simulateinput.cli.main doctor
61+ python -m simulateinput.cli.main doctor --compact
62+ python -m simulateinput.cli.main doctor --verbose
6063python -m simulateinput.cli.main session start
6164python -m simulateinput.cli.main mcp tools
6265```
6366
67+ ` doctor ` output modes:
68+
69+ - default: built-in profiles, MCP tool names, and driver diagnostics
70+ - ` --compact ` : reduced payload for UI surfaces that mainly need driver state and remediation
71+ - ` --verbose ` : default payload plus full MCP tool metadata
72+
6473## Typical CLI Workflow
6574
6675``` powershell
@@ -75,6 +84,8 @@ python -m simulateinput.cli.main action click-uia --session-id <session_id> --na
7584python -m simulateinput.cli.main action screenshot --session-id <session_id> --output artifacts/shot.png
7685```
7786
87+ On macOS, ` click-uia ` now validates that the chosen control is visible, enabled, and reasonably actionable before clicking its center.
88+
7889## YAML Case Runner
7990
8091``` powershell
@@ -135,6 +146,13 @@ See `docs/cross-platform-installation.md` for:
135146- CLI reference: ` skills/simulateinput/references/cli-usage.md `
136147- MCP reference: ` skills/simulateinput/references/mcp-tools.md `
137148
149+ ## macOS Notes
150+
151+ - ` doctor ` reports structured permission status for ` Accessibility ` , ` Automation ` , and ` Screen Recording `
152+ - remediation hints now include ` system_settings_path ` , ` shell_hint ` , and ` copyable_steps `
153+ - ` find_uia ` and ` focus_window ` use title-plus-geometry matching to reduce wrong-window selection when titles repeat
154+ - screenshots, OCR, and image matching still require ` Screen Recording `
155+
138156## Safety Boundary
139157
140158SimulateInput is intended for automation of your own software, test environments, and explicitly authorized systems.
@@ -169,11 +187,12 @@ SimulateInput 是一个跨平台的桌面与浏览器自动化测试平台,用
169187- MCP 服务,可供 AI 通过工具调用
170188- YAML case runner,可执行可复用的自动化测试流程
171189- 为 AI 使用准备的 skill 文档和参考资料
190+ - 结构化 ` doctor ` 诊断输出,可直接给本地环境修复提示
172191
173192## 当前平台状态
174193
175194- Windows:主实现,已完成真实执行和 smoke test
176- - macOS:已完成 MVP 驱动,实现依赖 Accessibility / Automation / Screen Recording 权限
195+ - macOS:已完成 MVP 驱动,支持结构化权限诊断、基于标题加几何信息的窗口匹配,以及更安全的 ` click_uia ` 预检查
177196- Linux X11:已完成 MVP 驱动,依赖 ` wmctrl ` 、` xdotool ` 、截图工具和可选 AT-SPI 环境
178197- Linux Wayland:当前是兼容层,依赖外部 helper,能力还未与 Windows 等价
179198
@@ -200,10 +219,18 @@ SimulateInput 是一个跨平台的桌面与浏览器自动化测试平台,用
200219``` powershell
201220$env:PYTHONPATH='src'
202221python -m simulateinput.cli.main doctor
222+ python -m simulateinput.cli.main doctor --compact
223+ python -m simulateinput.cli.main doctor --verbose
203224python -m simulateinput.cli.main session start
204225python -m simulateinput.cli.main mcp tools
205226```
206227
228+ ` doctor ` 输出模式:
229+
230+ - 默认:内置 profile、MCP 工具名和当前 driver 诊断
231+ - ` --compact ` :更适合 UI 消费的精简结果,重点保留 driver 状态和 remediation
232+ - ` --verbose ` :在默认结果上附加完整 MCP 工具元数据
233+
207234## 常见 CLI 流程
208235
209236``` powershell
@@ -218,6 +245,8 @@ python -m simulateinput.cli.main action click-uia --session-id <session_id> --na
218245python -m simulateinput.cli.main action screenshot --session-id <session_id> --output artifacts/shot.png
219246```
220247
248+ 在 macOS 上,` click-uia ` 现在会在点击中心点之前先检查目标控件是否可见、可用,并且是否具备合理的可操作性。
249+
221250## YAML 用例执行
222251
223252``` powershell
@@ -278,6 +307,13 @@ python -m simulateinput.cli.main mcp serve
278307- CLI 参考:` skills/simulateinput/references/cli-usage.md `
279308- MCP 参考:` skills/simulateinput/references/mcp-tools.md `
280309
310+ ## macOS 说明
311+
312+ - ` doctor ` 会输出 ` Accessibility ` 、` Automation ` 、` Screen Recording ` 的结构化权限状态
313+ - remediation 结果包含 ` system_settings_path ` 、` shell_hint ` 和 ` copyable_steps `
314+ - ` find_uia ` 和 ` focus_window ` 会用“标题 + 几何信息”匹配窗口,降低重名窗口误选概率
315+ - 截图、OCR 和图像匹配仍然依赖 ` Screen Recording ` 权限
316+
281317## 安全边界
282318
283319SimulateInput 只应用于:
0 commit comments