The 'not visible' tips were on the wrong code path (error:notvisible branch
which is rare). Hidden elements go through the missingState branch instead.
Fixed by adding visibility tip when missingState === 'visible'.
Also moved inline snapshot assertions before isError expects per convention.
Two improvements for agent experience with Playwright actions:
1. Descriptive timeout errors (via @xmorse/playwright-core 1.59.6):
When locator.click() times out, the error now explains WHY — e.g.
'Element is not visible', '<div id="overlay"> intercepts pointer events'.
The 'not visible' case includes tips about common causes (collapsed
<details>, inactive tab, CSS) and suggests { force: true }.
2. Faster action timeouts in executor:
- Action timeout (click, fill, hover): 2s (was 10s)
- Navigation timeout (goto, reload): 10s (unchanged)
- Code execution timeout: 10s (unchanged)
Agents now get fast failure with descriptive errors instead of waiting
10 seconds for a generic 'Timeout exceeded.'
Also removed the context.setDefaultTimeout(timeout) override in execute()
that was resetting action timeouts back to 10s on every call.
Tests added for three click failure scenarios: hidden element in collapsed
<details>, element covered by overlay, and display:none element.
Documents the full type generation pipeline (markdown docs → overrides.d.ts →
generator → types.d.ts) with an 8-step checklist and format examples for
methods, properties, and standalone types. Prevents future APIs from being
added to source without updating the type surface.
Updates submodule pointer to include public type declarations for
Page.onMouseAction, BrowserContext.getExistingCDPSession, and
MouseActionEvent — fixes all typecheck errors in playwriter.
The snapshot diff cache in executor.ts used only the Page object as key, so
calling snapshot({ page }) then snapshot({ page, locator }) would diff a
locator-scoped subtree against the previous full-page snapshot — producing
misleading diffs full of 'removed' lines that are simply outside the scope.
Fix: change lastSnapshots from WeakMap<Page, string> to a two-level
WeakMap<Page, Map<string, string>> keyed by locator.selector() (or '__page__'
for full-page). This matches the pattern already used by getCleanHTML in
clean-html.ts.
Also replaced (locator as any)._selector with the new typed
locator.selector() API from @xmorse/playwright-core 1.59.5 in both
executor.ts and clean-html.ts — zero `as any` casts.
Added locator-selector.test.ts with 28 inline snapshots covering CSS
selectors, getByRole, getByText, getByPlaceholder, getByLabel, getByAltText,
getByTestId, chained/filtered/described locators, and and/or combinators.
All `-e` arguments in bash examples now use single quotes instead of double
quotes. This prevents bash from interpreting `$`, backticks, and backslashes
inside JS code — a major pain point for agents that construct playwriter
commands programmatically.
Changes across skill docs, README, and website copies:
- **Single quotes for one-liners**: `playwriter -s 1 -e 'await page.goto("...")'`
- **Heredoc preferred for multiline**: `<<'EOF'` disables all bash expansion
- **Quoting rules summary**: explains when to use single quotes vs heredoc vs `$'...'`
- **Rewrote mistake #6**: now explains *why* double quotes break (bash `$` expansion
silently corrupts regex like `/\$[\d.]+/` and template literals)
- **Added locator-scoped snapshot section**: shows `snapshot({ locator: page.locator("main") })`
which dramatically reduces output size from ~150 lines to ~20 lines by snapshotting
only a subtree instead of the full page
- **Added single-quote tip** to README quick start and SKILL.md stub
The locator-scoped snapshot feature already existed but wasn't documented.
Discovered during real-world Cloudflare dashboard automation where full page
snapshots were dominated by 60+ sidebar navigation links.
Single-expression code ending with `;` (e.g. `await foo();`) was wrapped as
`return await (await foo();)` — the semicolon inside parens is a SyntaxError.
Refactored `shouldAutoReturn` into `getAutoReturnExpression` which returns the
exact expression source via acorn's `expr.start`/`expr.end` offsets, naturally
excluding the trailing semicolon. More robust than regex stripping since it uses
the parser we already have.
Closes#58
Closes#57
Highlights key differences: Playwriter uses your existing Chrome
(logins, extensions, cookies), can bypass captchas by disconnecting
the extension, lets you collaborate in the same browser window,
and gives full Playwright API + raw CDP access instead of a limited
shell command set.
The biggest context waste pattern is: screenshot → spawn image-understanding agent →
read description → repeat. snapshot() gives the same information (element names, roles,
locators, text) instantly as text.
Changes:
- Split 'snapshot before screenshot' into two rules: one for snapshot vs screenshot,
one explicitly saying snapshot replaces page.evaluate() DOM inspection
- Remove wrong claim that screenshotWithAccessibilityLabels returns images inline
in CLI context (it doesn't — that's MCP-only)
- Rewrite mistake #13 as numbered steps: snapshot first, try correct interaction
pattern second, investigate internals only as last resort
- Clarify when page.evaluate() IS still appropriate (state mutations, non-DOM data)
syncTabGroup was including 'connecting' tabs in the group unconditionally,
so when the relay WebSocket closed and all tabs transitioned to 'connecting',
the group stayed green. Now 'connecting' tabs are only included when the
relay is actually connected (connectionState === 'connected'). When the
relay is dead, 'connecting' tabs are excluded and the group gets cleaned up.
The onUpdated handler at line ~1601 already guards against the
ungroup→disconnect loop for 'connecting' tabs, so this is safe.
When an extension reconnects, clients can be rebound to a successor connection without reopening the Playwright socket. Command routing was still resolving via the extension id captured at client socket open, so rebound clients could hit stale routes and return Extension not connected.
Resolve the bound extension id from relay state on every incoming Playwright message, and prefer the newest same-stableKey successor when rebinding on extension close. This keeps reconnect handoff consistent between state transitions and runtime routing.
- Inline incrementExtensionMessageId: trivially messageId + 1, no value
as a separate function with one call site and one test
- Remove clearExtensionPendingRequests: redundant before removeExtension
which deletes the entire extension entry (including pendingRequests)
- Fold removeClientsForExtension into removeExtension: always called
together, callers can no longer forget to clean up bound clients
Net: -3 exported functions, -3 tests replaced by 1 new test verifying
removeExtension also removes bound playwright clients.
Introduce relay-state.ts with a single Zustand vanilla store holding all relay
server state (extensions, playwright clients, targets). All state transitions are
pure functions: data in, data out, no I/O inside setState().
Key changes:
- Move scattered mutable Maps (extensionConnections, extensionKeyIndex,
playwrightClients) into one RelayState atom managed by zustand/vanilla
- Co-locate I/O handles (ws, pingInterval, pendingRequests) on ExtensionEntry
instead of separate Maps, following the same pattern for PlaywrightClient
- Merge PlaywrightClientState + PlaywrightClientIO into single PlaywrightClient
type with { id, extensionId, ws } — eliminates the duplicate playwrightClientIO
map that had to stay in sync (and was already buggy: removeClientsForExtension
only cleaned playwrightClients but leaked playwrightClientIO entries)
- Derive extensionKeyIndex on demand via findExtensionByStableKey() linear scan
instead of maintaining a separate index Map
- Add relay-state.test.ts with 38 pure data-in/data-out tests covering all
state transitions and derivation helpers
Add hardware encoder detection with automatic fallback:
- h264_videotoolbox (macOS) → h264_nvenc/qsv/amf (Windows) → h264_nvenc/qsv (Linux) → libx264
- Probe caches result per process, 5s timeout per encoder test
- h264_vaapi excluded (needs -vaapi_device + hwupload in filter graph)
Add social-media-optimized encoding parameters:
- libx264: CRF 18 (near visually lossless for sharp text), preset fast,
deblock=-1,-1 (preserves text edges), no -tune flag (animation blurs
text, zerolatency is for real-time capture only)
- h264_videotoolbox: -q:v 80 (≈CRF 18 for screen content)
- Common: yuv420p, maxrate 25M (X.com cap), bufsize 50M, movflags +faststart
Skip redundant fps/scale filters on normal-speed passthrough segments
when output dimensions and frame rate match the probed input values.
Previously no -c:v or quality params were specified at all — ffmpeg
defaulted to libx264 -preset medium -crf 23, which is tuned for natural
video, not screen recordings, and purely CPU-based.
Tested on 48s 3440x1964 screen recording: 17s encode time with
VideoToolbox hardware acceleration, 5.5 Mbps output bitrate.
Instead of hard-blocking diff when search is present, change the
default: showDiffSinceLastCall defaults to !search. This way agents
don't get implicit diffs that swallow their search results, but can
still explicitly pass showDiffSinceLastCall: true to combine both.
Applied to snapshot(), getPageMarkdown(), and getCleanHTML().
Updated skill.md docs to document the new default behavior.
When both showDiffSinceLastCall and search were set, the diff block
returned early and the search filter was silently ignored. Now all
three functions (snapshot, getPageMarkdown, getCleanHTML) skip diffing
when a search string is present, ensuring the agent always gets
filtered results.
Clarify recording guidance to prefer interaction-driven flows (click, hover, typing) so ghost cursor motion is visible during demos.
Tune ghost cursor movement defaults for faster, human-like motion and align cursor encoder script path handling with __dirname plus path.resolve conventions.
Clarify recording guidance to prefer interaction-driven flows (click, hover, typing) so ghost cursor motion is visible during demos.
Tune ghost cursor movement defaults for faster, human-like motion and align cursor encoder script path handling with __dirname plus path.resolve conventions.
Timeout and abort error stack traces are internal noise (Promise.race,
setTimeout, AbortController internals) that waste context window space
for LLM agents and clutter CLI output. Now only error.message is shown
to the user for these error types. Full stacks are still logged to the
relay server log file for debugging.
Improve recording lifecycle safety by keeping stop and cancel cleanup aligned with successful relay operations, serializing per-page ghost cursor action application, and ensuring cursor transforms refresh when style or hotspot options change.
Harden demo video generation by preferring sane probed frame rates, capping speedup output FPS to source rate, and avoiding full-video speedup when execution timestamps are unavailable.
Extracts the sharp resize logic from screenshotWithAccessibilityLabels into a
standalone resizeImage() function exposed in the executor sandbox. Agents can
now shrink screenshots (or any image) to consume fewer context tokens.
Default behavior: fit within 1568×1568px (Claude-optimal), overwrite input file.
Also supports explicit width/height/maxDimension/fit/quality for custom resizing.
Replaces the old macOS-specific sips advice in skill.md.
Introduce a first-class ghost cursor system for recordings and screenshots with namespace APIs (, ) while preserving backward-compatible top-level recording helpers.
Improve reliability after relay restarts by consolidating extension wait logic around , reducing false disconnected states during and execute flows.
Refactor recording glue into dedicated modules (, in ) to keep executor orchestration thin and make cursor lifecycle state explicit and testable.
Prevents Chrome from hanging on the profile picker when agents restart it
programmatically. Applied to:
- getChromeRestartCommand() in screen-recording.ts (used for recording setup)
- All Chrome startup examples in skill.md (basic + recording flags)
- Both macOS, Linux, and Windows variants
Replace the macOS kill one-liner with a zsh-safe array pattern and add explicit post-kill verification commands for macOS and Windows. This prevents false positives where port 19988 remains occupied after an attempted shutdown.
The recording HTTP routes receive sessionId which can be either a CDP
session ID (pw-tab-*) from page.sessionId() or an executor session ID
(numeric). Previously, all sessionIds were looked up as executor sessions,
causing 'Session not found' for CDP session IDs.
New approach with resolveRecordingExtensionId():
- CDP session IDs (pw-tab-*): search all extension connections'
connectedTargets to find which extension owns that tab. Works correctly
with multiple Chrome profiles connected.
- Executor session IDs (numeric): look up executor metadata for extensionId
as before.
- null: fall back to default extension.
Applied consistently to all 4 recording routes (start/stop/status/cancel).
If recording is stopped and restarted inside the same execute() call,
the finally block could append an interval from the old recording into
the new recording's executionTimestamps. Fix by comparing against the
snapshotted recordingStartedAt instead of just checking non-null.
cdp-relay.ts:
- Fix pre-existing bug: recording HTTP routes (/recording/start, stop,
status, cancel) were treating CDP session IDs (pw-tab-*) as executor
session IDs, causing 'Session not found' errors. Now skip executor
lookup when sessionId is a CDP session ID and pass it through directly
to the extension for tab routing.
executor.ts:
- Use async IIFE instead of let+try for VM execution timestamp tracking.
Cleaner pattern: const result = await (async () => { try/finally })()
ffmpeg.ts + skill.md:
- Change default idle speed from 4x to 5x
Adds onMouseAction to playwright fork — an async callback on Page
that fires before every mouse action (move/down/up/wheel). Covers
both page.mouse.* and locator-initiated actions.
Test validates:
- Callback fires for page.mouse.click() with correct coordinates
- Callback fires for locator.click() with resolved element center
- Callback stops firing when set to null
ffmpeg.ts:
- Replace exec() with spawn() for ffmpeg/ffprobe — no shell interpreter,
no injection risk from filenames with special characters
- Filter invalid ranges in computeIdleSections() after clamping to video
bounds (handles timestamps exceeding video duration)
- Use path.basename() for timer IDs instead of split('/') for Windows compat
executor.ts:
- Use try/finally around VM execution so execution timestamps are recorded
even when code throws — the execution still occupied real time that should
not be sped up in the demo video
- Snapshot recordingStartedAt before try to avoid race if stopRecording()
is called inside the same execute() clearing the field
Add ffmpeg.ts with utilities for video manipulation:
- concatenateVideos: merge multiple video files with trim/scale/fps normalization
- speedUpSections: speed up arbitrary timestamp ranges in a single ffmpeg pass
- computeIdleSections: compute idle gaps from execution timestamps
- createDemoVideo: high-level wrapper that takes stopRecording() result and
produces a sped-up video where idle time (between execute() calls) is fast-forwarded
Executor changes:
- Track execution timestamps (start/end in seconds) while recording is active
- startRecording wrapper resets tracking state
- stopRecording wrapper returns executionTimestamps alongside path/duration/size
- cancelRecording wrapper clears tracking state
- Expose createDemoVideo in the sandbox VM context
The recording timeline looks like:
0s 5s 8s 12s 15s 20s 25s 30s
├───────┼────┼────┼────┼────┼────┼─────┤
▓▓▓▓▓ ▓▓▓▓▓ ▓▓▓▓▓▓▓▓▓
exec1 exec2 exec3
░░░░░░░ ░░░░░ ░░░░░
IDLE(4x) IDLE(4x) IDLE(4x)
A 1-second buffer (INTERACTION_BUFFER_SECONDS) is kept around each
execution at normal speed so viewers see context before/after actions.
Requires ffmpeg + ffprobe on the system.
Removed the preserveSystemColorScheme() workaround from executor.ts that was
mutating private Playwright internals via (context as any)._options to patch
around the forced light mode.
The root cause is now fixed in @xmorse/playwright-core@1.59.4: the default
colorScheme fallback in page.emulatedMedia() changed from 'light' to
'no-override', so Chrome respects the user's system color scheme.
Activated the previously-todo test that validates dark mode is preserved
after MCP connection (matchesDark: true, matchesLight: false).
Tommy D. Rossi