Skip to content

test(ssh-commands): lock down update timing fix and clarify skill docs#20

Merged
facundofarias merged 2 commits into
mainfrom
fix/ssh-commands-timing-tests-and-docs
Jun 8, 2026
Merged

test(ssh-commands): lock down update timing fix and clarify skill docs#20
facundofarias merged 2 commits into
mainfrom
fix/ssh-commands-timing-tests-and-docs

Conversation

@facundofarias

@facundofarias facundofarias commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

The original ssh-commands update timing bug (PR #14, shipped in v0.16.4) was code-correct but untested, and the agent-facing skill reference still advertised a --name flag that doesn't exist while saying nothing about --timing. Mixpanel still shows 140 historical 422s from a single codex agent on v0.16.3 failing with timing is not included in the list — modern CLIs are immune, but the docs that drive agent input were still steering them wrong.

  • pkg/sdk/ssh_commands_test.go (new) — asserts omitempty keeps empty Timing out of the PATCH body, plus a positive test that explicit values pass through. This is the actual regression guard for the original bug.
  • internal/commands/ssh_commands_test.go (new) — unit test for validateSSHTiming covering empty (allowed on update), the three valid values, and the historically misleading before/after strings that the old help text suggested.
  • skills/deployhq/references/configuration.md — removed the nonexistent --name flag, documented --command/--description/--timing with valid values, and added the missing update subcommand section.

Test plan

  • go test ./... — all packages green
  • golangci-lint run ./... — 0 issues
  • New TestUpdateSSHCommand_OmitsEmptyTiming passes; reverting omitempty on SSHCommandCreateRequest.Timing makes it fail (verified the test actually guards the regression)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Updated SSH commands docs: create examples now use --command (no --name), document optional --description and optional --timing (default: all), and update examples/flag table now clarify that update performs partial updates (only passed flags are sent).
  • Tests
    • Added tests covering SSH timing validation (allowed/invalid values) and SSH command update requests (omits empty timing, sends explicit timing).

The original timing bug (PR #14, v0.16.4) had no tests, leaving the
omitempty behaviour on SSHCommandCreateRequest.Timing easy to regress.
Mixpanel still shows 140 historical hits from one codex agent on
v0.16.3 attempting `ssh-commands update` with bad timing values — modern
versions are immune, but the skill reference that drives agent input
still advertised a nonexistent --name flag and never mentioned --timing
at all.

- Add pkg/sdk regression test asserting empty Timing is dropped from
  the PATCH body, and a positive test that explicit values pass through.
- Add internal/commands unit test for validateSSHTiming covering empty,
  valid, and the historically-misleading invalid values (before/after).
- Replace the stale --name flag in the skill reference with the real
  flags (--command, --description, --timing) and document the missing
  update subcommand so future agents see valid timing values.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4a570cf1-2317-4cdc-a583-04b350d5e48c

📥 Commits

Reviewing files that changed from the base of the PR and between ed89189 and 981320f.

📒 Files selected for processing (1)
  • skill-evals/deployhq/evals.json

Walkthrough

Adds tests for CLI timing validation and SDK request JSON serialization around the timing field, updates CLI documentation for dhq ssh-commands create/update to remove --name and document --timing, and updates an eval fixture to remove --name.

Changes

SSH Command Timing Tests & Documentation

Layer / File(s) Summary
CLI timing validation tests
internal/commands/ssh_commands_test.go
TestValidateSSHTiming verifies validateSSHTiming accepts empty string and all, first, after_first, and rejects invalid/case-variant values with an error containing Invalid --timing.
SDK request timing serialization tests
pkg/sdk/ssh_commands_test.go
Two tests start an httptest server and assert UpdateSSHCommand omits command.timing when timing is not provided and includes it with the explicit value when set; includes readAllJSON helper to decode request JSON.
Command documentation and eval fixture updates
skills/deployhq/references/configuration.md, skill-evals/deployhq/evals.json
Documentation for dhq ssh-commands create removes the --name flag and documents --command, optional --description, and optional --timing; update docs clarify partial-update behavior. Eval fixture removes --name from expected flags.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • deployhq/deployhq-cli#14: Implements validateSSHTiming logic and adjusts request/flag behavior around valid timing values (all, first, after_first), which this PR adds test coverage for.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly summarizes the main changes: adding regression tests for ssh-commands update timing and clarifying skill documentation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ssh-commands-timing-tests-and-docs

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ed89189d5b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

| `--timing` | no | When the command runs: `all` (default), `first`, or `after_first` |

```bash
dhq ssh-commands create -p my-app --command "sudo systemctl restart app" --description "Restart" --json

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the SSH command eval expectation

Once this reference tells agents to create SSH commands without --name, the add-ssh-command skill eval is left inconsistent: skill-evals/deployhq/evals.json still requires the generated command to contain --name, and run-evals.sh reports MISSING_FLAG:--name whenever that expected flag is absent. In contexts where these skill evals are run against the updated reference, a correct agent response following this example will fail until that eval is updated to remove --name (and include the intended timing flag if needed).

Useful? React with 👍 / 👎.

The --name flag does not exist on `dhq ssh-commands create` — the
reference docs in this PR now correctly point at --command,
--description, and --timing. Leaving the eval expectation requiring
--name would make run-evals.sh emit MISSING_FLAG:--name on a
correct agent response.

Codex flagged this on PR #20:
#20 (comment)...

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@facundofarias facundofarias merged commit ac2dbcc into main Jun 8, 2026
3 checks passed
@facundofarias facundofarias deleted the fix/ssh-commands-timing-tests-and-docs branch June 8, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant