Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ Command-only flags (like `find --first`) that do not flow to the platform layer

## React Native Verification
- After changing runtime code exercised through `bin/agent-device.mjs` or the daemon, run `pnpm build` and `pnpm clean:daemon` before manual device verification so snapshots use current `dist` output.
- For repo-owned `Agent Device Tester` verification, use `examples/test-app/README.md` as the source of truth for simulator, physical-device, Metro/dev-client, and app-surface verification steps. Do not treat an already installed `com.callstack.agentdevicelab` as sufficient unless the README's Metro/dev-build and `snapshot -i` checks prove the expected app surface is running.
- For Android RN/Expo/dev-client apps connected to any local Metro port, `adb reverse tcp:<port> tcp:<port>` is harmless and should be run before opening the app or URL on the emulator/device.
- In sandboxed agent environments, run manual `agent-device` CLI verification that starts the daemon outside the sandbox with escalation. The daemon binds localhost, and sandboxed runs can fail before any product code executes with `listen EPERM: operation not permitted 127.0.0.1` or repeated `Failed to start daemon`/metadata cleanup messages. Do not spend time debugging those as agent-device regressions; rerun the same command with escalation. Unit tests, typecheck, lint, and build can stay sandboxed unless they need platform devices or network/listener access.

Expand Down
86 changes: 81 additions & 5 deletions examples/test-app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@ The app declares `@expo/dom-webview` directly to keep Expo's development runtime
on the SDK 56 native module; Android verification failed when the dev client
resolved an older transitive copy.

From the repo root:
### iOS simulator

From the repo root, install dependencies and run the development build on the
target simulator:

```bash
pnpm test-app:install
Expand All @@ -63,13 +66,51 @@ pnpm test-app:ios -- --device "iPhone 17 Pro"
terminal running, then use a separate terminal for `agent-device` or Maestro
commands.

Or on Android:
### iOS physical device

Use the physical device name from `agent-device devices --platform ios` or
`xcrun devicectl list devices`. Keep the `expo run:ios` terminal running so
Metro stays visible to the development build:

```bash
pnpm test-app:install
pnpm test-app:ios -- --device "<physical device name>"
```

Then verify the installed development build from another terminal with the same
physical device identifier:

```bash
agent-device open com.callstack.agentdevicelab --platform ios --udid "<physical udid>" --session test-app-physical
agent-device snapshot -i --platform ios --udid "<physical udid>" --session test-app-physical
```

The snapshot should show the `Agent Device Tester` home screen, for example the
`Agent Device Tester` heading and tab bar. An already installed
`com.callstack.agentdevicelab` is not enough evidence by itself: confirm Metro
is running for the development build and verify the visible app surface before
using the session for manual logs, network, replay, or interaction checks. Close
the same session when verification is complete:

```bash
agent-device close --platform ios --udid "<physical udid>" --session test-app-physical
```

### Android emulator or device

Install dependencies and run the development build on the target Android
emulator or device:

```bash
pnpm test-app:install
pnpm test-app:android -- --device "$ANDROID_DEVICE"
```

For Android app/package launches connected to local Metro, run `adb reverse`
for the Metro port when needed before opening the app with `agent-device`.

### Running from the app folder

If you prefer to work from inside the app folder:

```bash
Expand All @@ -87,9 +128,44 @@ pnpm android
```

After the first native build is installed, use `pnpm test-app:start` when you only
need to restart Metro for JavaScript or TypeScript changes. Once the app is
running, use `agent-device` against `Agent Device Tester` like any other target
app.
need to restart Metro for JavaScript or TypeScript changes. `test-app:start`
starts Metro only; it does not build, install, or prove a physical device is
running the development build. Once the app is running and verified with
`snapshot -i`, use `agent-device` against `Agent Device Tester` like any other
target app.

### Non-default Metro ports

If the default Metro port is already in use, start Metro on another port. Do not
reinstall the native development build just to change the JavaScript server port:

```bash
pnpm test-app:start -- --port 8082
```

If you are building and installing for the first time in that terminal, Expo's
`run:ios` and `run:android` commands also accept `--port`:

```bash
pnpm test-app:ios -- --device "<device name>" --port 8082
pnpm test-app:android -- --device "$ANDROID_DEVICE" --port 8082
```

After the development build is installed, keep using the same native app. The
current `agent-device open` CLI does not accept `--metro-host` or `--metro-port`;
open the app normally, then use the Metro command surface for Metro-specific
actions:

```bash
agent-device metro prepare --project-root examples/test-app --kind expo --port 8082 --public-base-url http://127.0.0.1:8082
agent-device metro reload --metro-host 127.0.0.1 --metro-port 8082
```

Use `metro prepare` when you want `agent-device` to start or reuse Metro and
print the runtime URLs. Use `metro reload` when Metro is already running and the
installed development build is connected to that server. For Android local
device/emulator runs, also run `adb reverse tcp:8082 tcp:8082` when the device
needs host port forwarding.

## Local Agent Device suites

Expand Down
1 change: 1 addition & 0 deletions src/cli/parser/cli-help.ts
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ Logs:
Do not cat a full stale log into agent context. Open or grep only the relevant window when needed.
logs clear --restart is the compact command to clear old logs and start a fresh capture; do not split it into logs stop, logs clear, logs start.
On iOS simulators, logs scope by bundle id and resolved app executable, so use this instead of raw simctl log stream predicates.
On iOS physical devices, logs clear --restart relaunches the session app through devicectl process launch --console so stdout/stderr can be captured.
For iOS simulator launch-time stdout/stderr, use --launch-console on the direct app launch:
agent-device open MyApp --platform ios --relaunch --launch-console ./artifacts/app.console.log
--launch-console is only for direct iOS simulator app launches, not URL opens.
Expand Down
164 changes: 158 additions & 6 deletions src/daemon/__tests__/app-log.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,66 @@ import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { finished } from 'node:stream/promises';
import type { DeviceInfo } from '../../kernel/device.ts';
import { AppError } from '../../kernel/errors.ts';
import { withAppleToolProvider } from '../../platforms/apple/core/tool-provider.ts';
import type { ExecResult } from '../../utils/exec.ts';
import {
APP_LOG_PID_FILENAME,
assertAndroidPackageArgSafe,
buildAppleLogPredicate,
buildIosDeviceLogStreamArgs,
buildIosDeviceConsoleLaunchArgs,
buildIosSimulatorLogStreamArgs,
cleanupStaleAppLogProcesses,
runAppLogDoctor,
rotateAppLogIfNeeded,
} from '../app-log.ts';
import { startIosDeviceAppLog } from '../app-log-ios.ts';

const IOS_DEVICE_ID = '00008150-0000AAAA';
const IOS_DEVICE: DeviceInfo = {
platform: 'apple',
appleOs: 'ios',
id: IOS_DEVICE_ID,
name: 'iPhone',
kind: 'device',
};
const IOS_DEVICE_HELP_WITHOUT_CONSOLE_CAPTURE =
'USAGE: devicectl device [--verbose] [--quiet] <subcommand>\n\nSUBCOMMANDS:\n info\n process\n';
const IOS_DEVICE_CONSOLE_CAPTURE_HELP = `USAGE: devicectl device process launch [<options>] --device <uuid|ecid|serial_number|udid|name|dns_name> <bundle-identifier-or-path>

COMMAND OPTIONS:
--console Attaches the application to the console and waits for it to exit.
--terminate-existing Terminates any already-running instances of the app prior to launch.`;

type FakeDevicectlRun = (args: string[]) => Promise<ExecResult>;

async function withFakeDevicectl<T>(
run: FakeDevicectlRun,
fn: () => Promise<T>,
): Promise<{ result: T; calls: string[][] }> {
const calls: string[][] = [];
const result = await withAppleToolProvider(
{
runCommand: async () => ({ stdout: '', stderr: '', exitCode: 0 }),
devicectl: {
run: async (args) => {
calls.push(args);
return await run(args);
},
},
whichCommand: async () => false,
},
fn,
);
return { result, calls };
}

function makeAppLogWriteStream(prefix: string): fs.WriteStream {
const root = fs.mkdtempSync(path.join(os.tmpdir(), prefix));
return fs.createWriteStream(path.join(root, 'app.log'), { flags: 'a' });
}

test('buildAppleLogPredicate includes bundle-aware filters', () => {
const predicate = buildAppleLogPredicate('com.example.app');
Expand Down Expand Up @@ -64,17 +114,119 @@ test('cleanupStaleAppLogProcesses removes pid files even when pid is stale', ()
assert.equal(fs.existsSync(pidPath), false);
});

test('buildIosDeviceLogStreamArgs builds expected devicectl command args', () => {
assert.deepEqual(buildIosDeviceLogStreamArgs('00008150-0000AAAA'), [
test('buildIosDeviceConsoleLaunchArgs builds expected devicectl command args', () => {
assert.deepEqual(buildIosDeviceConsoleLaunchArgs(IOS_DEVICE_ID, 'com.example.app'), [
'devicectl',
'device',
'log',
'stream',
'process',
'launch',
'--device',
'00008150-0000AAAA',
IOS_DEVICE_ID,
'--console',
'--terminate-existing',
'com.example.app',
]);
});

test('startIosDeviceAppLog reports unsupported devicectl console capture before spawning', async () => {
const stream = makeAppLogWriteStream('agent-device-ios-device-log-');
const { calls } = await withFakeDevicectl(
async () => ({
stdout: IOS_DEVICE_HELP_WITHOUT_CONSOLE_CAPTURE,
stderr: '',
exitCode: 0,
}),
async () => {
await assert.rejects(
async () => await startIosDeviceAppLog(IOS_DEVICE_ID, 'com.example.app', stream, []),
(error: unknown) => {
assert.ok(error instanceof AppError);
assert.equal(error.code, 'UNSUPPORTED_OPERATION');
assert.match(error.message, /iOS physical-device app console capture is not supported/);
assert.equal(error.details?.backend, 'ios-device');
return true;
},
);
},
);

await finished(stream).catch(() => {});
assert.deepEqual(calls, [['device', 'process', 'launch', '--help']]);
});

test('startIosDeviceAppLog reports retryable failure when devicectl support probe fails', async () => {
const stream = makeAppLogWriteStream('agent-device-ios-device-log-timeout-');

await withFakeDevicectl(
async () => {
throw new Error('xcrun timed out after 5000ms');
},
async () => {
await assert.rejects(
async () => await startIosDeviceAppLog(IOS_DEVICE_ID, 'com.example.app', stream, []),
(error: unknown) => {
assert.ok(error instanceof AppError);
assert.equal(error.code, 'COMMAND_FAILED');
assert.match(error.message, /Could not verify iOS physical-device app console capture/);
assert.equal(error.details?.stderr, 'xcrun timed out after 5000ms');
return true;
},
);
},
);

await finished(stream).catch(() => {});
});

test('runAppLogDoctor reports supported iOS physical-device console capture', async () => {
const { result, calls } = await withFakeDevicectl(
async (args) => {
if (args.join(' ') === '--version') {
return { stdout: '506.6\n', stderr: '', exitCode: 0 };
}
return {
stdout: IOS_DEVICE_CONSOLE_CAPTURE_HELP,
stderr: '',
exitCode: 0,
};
},
async () => await runAppLogDoctor(IOS_DEVICE, 'com.example.app'),
);

assert.deepEqual(calls, [['--version'], ['device', 'process', 'launch', '--help']]);
assert.equal(result.checks.devicectlAvailable, true);
assert.equal(result.checks.devicectlConsoleCapture, true);
assert.equal(result.notes.length, 0);
});

test('startIosDeviceAppLog marks clean devicectl console exit as ended', async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-ios-device-console-'));
const fakeBinDir = path.join(root, 'bin');
fs.mkdirSync(fakeBinDir);
const fakeXcrun = path.join(fakeBinDir, 'xcrun');
fs.writeFileSync(fakeXcrun, '#!/bin/sh\nprintf "app output\\n"\nexit 0\n');
fs.chmodSync(fakeXcrun, 0o755);
const previousPath = process.env.PATH;
process.env.PATH = `${fakeBinDir}${path.delimiter}${previousPath ?? ''}`;
const stream = fs.createWriteStream(path.join(root, 'app.log'), { flags: 'a' });

try {
await withFakeDevicectl(
async () => ({ stdout: IOS_DEVICE_CONSOLE_CAPTURE_HELP, stderr: '', exitCode: 0 }),
async () => {
const appLog = await startIosDeviceAppLog(IOS_DEVICE_ID, 'com.example.app', stream, []);
assert.equal(appLog.getState(), 'active');
assert.equal((await appLog.wait).exitCode, 0);
assert.equal(appLog.getState(), 'ended');
},
);
} finally {
if (previousPath === undefined) delete process.env.PATH;
else process.env.PATH = previousPath;
await finished(stream).catch(() => {});
}
});

test('buildIosSimulatorLogStreamArgs streams logs inside the simulator at info level', () => {
assert.deepEqual(
buildIosSimulatorLogStreamArgs({
Expand Down
Loading
Loading