The previous .find() regex stopped matching because the error text starts with
a newline. Replaced with expect(result).toMatchInlineSnapshot() to capture the
full MCP tool call output (content + isError). Reduced click timeout from 5000ms
to 100ms — tests now run in ~350ms each instead of ~5s.
Recording now automatically fits the viewport to a target aspect ratio (default
16:9) before starting — shrinks to fit, never increases dimensions. Original
viewport is restored on stop/cancel.
Added maxDurationMs option (default 15 min) that auto-stops recording to prevent
accidentally filling disk. Set to 0 or Infinity to disable.
Both options are documented in skill.md with examples. fitToAspectRatio is a pure
utility function covered by inline snapshot tests for common screen sizes (1080p,
MacBook 16:10, 4:3, ultra-wide, square) and custom ratios (4:3, 1:1, 9:16).
This reverts the addGhostCursorInitScript approach after it proved unreliable in real usage.
What changed:\n- remove addGhostCursorInitScript/removeGhostCursorInitScript and related CDP init-script wiring\n- restore direct per-page ghost cursor injection fallback in applyGhostCursorMouseAction\n- remove recording controller state/cleanup tied to init-script identifiers\n- remove skill.md claim that ghost cursor persists automatically across MPA navigations\n- bump playwriter version to 0.0.85 and document the revert in changelog
This commit batches the pending workspace updates that were coupled in practice: extension reliability fixes, ghost cursor persistence across MPA navigations, timeout tuning, and aligned docs/versioning updates so behavior and guidance stay in sync.
Key updates:\n- extension: add MV3 keepalive via chrome.alarms, add alarms permission, and bump extension changelog/version to 0.0.73 to reduce silent relay disconnects while idle\n- playwriter runtime: raise default action timeout to 60s while keeping navigation timeout separate; increase relay /version fetch timeout to 2000ms\n- recording + cursor: add persistent cursor init scripts with Page.addScriptToEvaluateOnNewDocument, remove init script on teardown, and ensure cursor enablement before applying mouse actions\n- demo video defaults: use 0.5s side buffer (1s total) and default idle speed 6x for createDemoVideo/computeIdleSections\n- tests/docs: keep click-error tests deterministic with explicit 5000ms per-click timeout, remove outdated click-timeout section from skill mistakes, and update package/changelog entries
Four lessons from recording demo videos on MPA sites (GitHub, Hacker News):
1. recording.stop() result: examples now save the full result to state.recordingResult
instead of destructuring — executionTimestamps must not be lost between execute calls
or createDemoVideo cannot detect idle sections and you end up doing raw ffmpeg.
2. Ghost cursor MPA: document that cursor survives full-page navigation automatically
(via Page.addScriptToEvaluateOnNewDocument registered at recording.start time).
Previous docs implied you had to re-inject manually — which lost 2 recording attempts.
3. New mistake #13: SPA click timeouts — GitHub Turbo.js and similar SPA routers intercept
link clicks without firing standard load events. Playwright's click() sees a navigation
and waits for load, which never fires, so it times out even though the click succeeded.
Documents .catch(() => {}), { noWaitAfter: true }, and { timeout: 10000 } patterns.
4. createDemoVideo call: clarified that state.recordingResult must be saved (not just
destructured) and that createDemoVideo must run in its own execute call with high timeout.
The 'not visible' tips were on the wrong code path (error:notvisible branch
which is rare). Hidden elements go through the missingState branch instead.
Fixed by adding visibility tip when missingState === 'visible'.
Also moved inline snapshot assertions before isError expects per convention.
Two improvements for agent experience with Playwright actions:
1. Descriptive timeout errors (via @xmorse/playwright-core 1.59.6):
When locator.click() times out, the error now explains WHY — e.g.
'Element is not visible', '<div id="overlay"> intercepts pointer events'.
The 'not visible' case includes tips about common causes (collapsed
<details>, inactive tab, CSS) and suggests { force: true }.
2. Faster action timeouts in executor:
- Action timeout (click, fill, hover): 2s (was 10s)
- Navigation timeout (goto, reload): 10s (unchanged)
- Code execution timeout: 10s (unchanged)
Agents now get fast failure with descriptive errors instead of waiting
10 seconds for a generic 'Timeout exceeded.'
Also removed the context.setDefaultTimeout(timeout) override in execute()
that was resetting action timeouts back to 10s on every call.
Tests added for three click failure scenarios: hidden element in collapsed
<details>, element covered by overlay, and display:none element.
Documents the full type generation pipeline (markdown docs → overrides.d.ts →
generator → types.d.ts) with an 8-step checklist and format examples for
methods, properties, and standalone types. Prevents future APIs from being
added to source without updating the type surface.
Updates submodule pointer to include public type declarations for
Page.onMouseAction, BrowserContext.getExistingCDPSession, and
MouseActionEvent — fixes all typecheck errors in playwriter.
The snapshot diff cache in executor.ts used only the Page object as key, so
calling snapshot({ page }) then snapshot({ page, locator }) would diff a
locator-scoped subtree against the previous full-page snapshot — producing
misleading diffs full of 'removed' lines that are simply outside the scope.
Fix: change lastSnapshots from WeakMap<Page, string> to a two-level
WeakMap<Page, Map<string, string>> keyed by locator.selector() (or '__page__'
for full-page). This matches the pattern already used by getCleanHTML in
clean-html.ts.
Also replaced (locator as any)._selector with the new typed
locator.selector() API from @xmorse/playwright-core 1.59.5 in both
executor.ts and clean-html.ts — zero `as any` casts.
Added locator-selector.test.ts with 28 inline snapshots covering CSS
selectors, getByRole, getByText, getByPlaceholder, getByLabel, getByAltText,
getByTestId, chained/filtered/described locators, and and/or combinators.
All `-e` arguments in bash examples now use single quotes instead of double
quotes. This prevents bash from interpreting `$`, backticks, and backslashes
inside JS code — a major pain point for agents that construct playwriter
commands programmatically.
Changes across skill docs, README, and website copies:
- **Single quotes for one-liners**: `playwriter -s 1 -e 'await page.goto("...")'`
- **Heredoc preferred for multiline**: `<<'EOF'` disables all bash expansion
- **Quoting rules summary**: explains when to use single quotes vs heredoc vs `$'...'`
- **Rewrote mistake #6**: now explains *why* double quotes break (bash `$` expansion
silently corrupts regex like `/\$[\d.]+/` and template literals)
- **Added locator-scoped snapshot section**: shows `snapshot({ locator: page.locator("main") })`
which dramatically reduces output size from ~150 lines to ~20 lines by snapshotting
only a subtree instead of the full page
- **Added single-quote tip** to README quick start and SKILL.md stub
The locator-scoped snapshot feature already existed but wasn't documented.
Discovered during real-world Cloudflare dashboard automation where full page
snapshots were dominated by 60+ sidebar navigation links.
Single-expression code ending with `;` (e.g. `await foo();`) was wrapped as
`return await (await foo();)` — the semicolon inside parens is a SyntaxError.
Refactored `shouldAutoReturn` into `getAutoReturnExpression` which returns the
exact expression source via acorn's `expr.start`/`expr.end` offsets, naturally
excluding the trailing semicolon. More robust than regex stripping since it uses
the parser we already have.
Closes#58
Closes#57
Highlights key differences: Playwriter uses your existing Chrome
(logins, extensions, cookies), can bypass captchas by disconnecting
the extension, lets you collaborate in the same browser window,
and gives full Playwright API + raw CDP access instead of a limited
shell command set.
The biggest context waste pattern is: screenshot → spawn image-understanding agent →
read description → repeat. snapshot() gives the same information (element names, roles,
locators, text) instantly as text.
Changes:
- Split 'snapshot before screenshot' into two rules: one for snapshot vs screenshot,
one explicitly saying snapshot replaces page.evaluate() DOM inspection
- Remove wrong claim that screenshotWithAccessibilityLabels returns images inline
in CLI context (it doesn't — that's MCP-only)
- Rewrite mistake #13 as numbered steps: snapshot first, try correct interaction
pattern second, investigate internals only as last resort
- Clarify when page.evaluate() IS still appropriate (state mutations, non-DOM data)
syncTabGroup was including 'connecting' tabs in the group unconditionally,
so when the relay WebSocket closed and all tabs transitioned to 'connecting',
the group stayed green. Now 'connecting' tabs are only included when the
relay is actually connected (connectionState === 'connected'). When the
relay is dead, 'connecting' tabs are excluded and the group gets cleaned up.
The onUpdated handler at line ~1601 already guards against the
ungroup→disconnect loop for 'connecting' tabs, so this is safe.
When an extension reconnects, clients can be rebound to a successor connection without reopening the Playwright socket. Command routing was still resolving via the extension id captured at client socket open, so rebound clients could hit stale routes and return Extension not connected.
Resolve the bound extension id from relay state on every incoming Playwright message, and prefer the newest same-stableKey successor when rebinding on extension close. This keeps reconnect handoff consistent between state transitions and runtime routing.
- Inline incrementExtensionMessageId: trivially messageId + 1, no value
as a separate function with one call site and one test
- Remove clearExtensionPendingRequests: redundant before removeExtension
which deletes the entire extension entry (including pendingRequests)
- Fold removeClientsForExtension into removeExtension: always called
together, callers can no longer forget to clean up bound clients
Net: -3 exported functions, -3 tests replaced by 1 new test verifying
removeExtension also removes bound playwright clients.
Introduce relay-state.ts with a single Zustand vanilla store holding all relay
server state (extensions, playwright clients, targets). All state transitions are
pure functions: data in, data out, no I/O inside setState().
Key changes:
- Move scattered mutable Maps (extensionConnections, extensionKeyIndex,
playwrightClients) into one RelayState atom managed by zustand/vanilla
- Co-locate I/O handles (ws, pingInterval, pendingRequests) on ExtensionEntry
instead of separate Maps, following the same pattern for PlaywrightClient
- Merge PlaywrightClientState + PlaywrightClientIO into single PlaywrightClient
type with { id, extensionId, ws } — eliminates the duplicate playwrightClientIO
map that had to stay in sync (and was already buggy: removeClientsForExtension
only cleaned playwrightClients but leaked playwrightClientIO entries)
- Derive extensionKeyIndex on demand via findExtensionByStableKey() linear scan
instead of maintaining a separate index Map
- Add relay-state.test.ts with 38 pure data-in/data-out tests covering all
state transitions and derivation helpers
Add hardware encoder detection with automatic fallback:
- h264_videotoolbox (macOS) → h264_nvenc/qsv/amf (Windows) → h264_nvenc/qsv (Linux) → libx264
- Probe caches result per process, 5s timeout per encoder test
- h264_vaapi excluded (needs -vaapi_device + hwupload in filter graph)
Add social-media-optimized encoding parameters:
- libx264: CRF 18 (near visually lossless for sharp text), preset fast,
deblock=-1,-1 (preserves text edges), no -tune flag (animation blurs
text, zerolatency is for real-time capture only)
- h264_videotoolbox: -q:v 80 (≈CRF 18 for screen content)
- Common: yuv420p, maxrate 25M (X.com cap), bufsize 50M, movflags +faststart
Skip redundant fps/scale filters on normal-speed passthrough segments
when output dimensions and frame rate match the probed input values.
Previously no -c:v or quality params were specified at all — ffmpeg
defaulted to libx264 -preset medium -crf 23, which is tuned for natural
video, not screen recordings, and purely CPU-based.
Tested on 48s 3440x1964 screen recording: 17s encode time with
VideoToolbox hardware acceleration, 5.5 Mbps output bitrate.
Instead of hard-blocking diff when search is present, change the
default: showDiffSinceLastCall defaults to !search. This way agents
don't get implicit diffs that swallow their search results, but can
still explicitly pass showDiffSinceLastCall: true to combine both.
Applied to snapshot(), getPageMarkdown(), and getCleanHTML().
Updated skill.md docs to document the new default behavior.
When both showDiffSinceLastCall and search were set, the diff block
returned early and the search filter was silently ignored. Now all
three functions (snapshot, getPageMarkdown, getCleanHTML) skip diffing
when a search string is present, ensuring the agent always gets
filtered results.
Clarify recording guidance to prefer interaction-driven flows (click, hover, typing) so ghost cursor motion is visible during demos.
Tune ghost cursor movement defaults for faster, human-like motion and align cursor encoder script path handling with __dirname plus path.resolve conventions.
Clarify recording guidance to prefer interaction-driven flows (click, hover, typing) so ghost cursor motion is visible during demos.
Tune ghost cursor movement defaults for faster, human-like motion and align cursor encoder script path handling with __dirname plus path.resolve conventions.
Timeout and abort error stack traces are internal noise (Promise.race,
setTimeout, AbortController internals) that waste context window space
for LLM agents and clutter CLI output. Now only error.message is shown
to the user for these error types. Full stacks are still logged to the
relay server log file for debugging.
Improve recording lifecycle safety by keeping stop and cancel cleanup aligned with successful relay operations, serializing per-page ghost cursor action application, and ensuring cursor transforms refresh when style or hotspot options change.
Harden demo video generation by preferring sane probed frame rates, capping speedup output FPS to source rate, and avoiding full-video speedup when execution timestamps are unavailable.
Extracts the sharp resize logic from screenshotWithAccessibilityLabels into a
standalone resizeImage() function exposed in the executor sandbox. Agents can
now shrink screenshots (or any image) to consume fewer context tokens.
Default behavior: fit within 1568×1568px (Claude-optimal), overwrite input file.
Also supports explicit width/height/maxDimension/fit/quality for custom resizing.
Replaces the old macOS-specific sips advice in skill.md.
Introduce a first-class ghost cursor system for recordings and screenshots with namespace APIs (, ) while preserving backward-compatible top-level recording helpers.
Improve reliability after relay restarts by consolidating extension wait logic around , reducing false disconnected states during and execute flows.
Refactor recording glue into dedicated modules (, in ) to keep executor orchestration thin and make cursor lifecycle state explicit and testable.
Prevents Chrome from hanging on the profile picker when agents restart it
programmatically. Applied to:
- getChromeRestartCommand() in screen-recording.ts (used for recording setup)
- All Chrome startup examples in skill.md (basic + recording flags)
- Both macOS, Linux, and Windows variants
Replace the macOS kill one-liner with a zsh-safe array pattern and add explicit post-kill verification commands for macOS and Windows. This prevents false positives where port 19988 remains occupied after an attempted shutdown.
The recording HTTP routes receive sessionId which can be either a CDP
session ID (pw-tab-*) from page.sessionId() or an executor session ID
(numeric). Previously, all sessionIds were looked up as executor sessions,
causing 'Session not found' for CDP session IDs.
New approach with resolveRecordingExtensionId():
- CDP session IDs (pw-tab-*): search all extension connections'
connectedTargets to find which extension owns that tab. Works correctly
with multiple Chrome profiles connected.
- Executor session IDs (numeric): look up executor metadata for extensionId
as before.
- null: fall back to default extension.
Applied consistently to all 4 recording routes (start/stop/status/cancel).
If recording is stopped and restarted inside the same execute() call,
the finally block could append an interval from the old recording into
the new recording's executionTimestamps. Fix by comparing against the
snapshotted recordingStartedAt instead of just checking non-null.
cdp-relay.ts:
- Fix pre-existing bug: recording HTTP routes (/recording/start, stop,
status, cancel) were treating CDP session IDs (pw-tab-*) as executor
session IDs, causing 'Session not found' errors. Now skip executor
lookup when sessionId is a CDP session ID and pass it through directly
to the extension for tab routing.
executor.ts:
- Use async IIFE instead of let+try for VM execution timestamp tracking.
Cleaner pattern: const result = await (async () => { try/finally })()
ffmpeg.ts + skill.md:
- Change default idle speed from 4x to 5x
Adds onMouseAction to playwright fork — an async callback on Page
that fires before every mouse action (move/down/up/wheel). Covers
both page.mouse.* and locator-initiated actions.
Test validates:
- Callback fires for page.mouse.click() with correct coordinates
- Callback fires for locator.click() with resolved element center
- Callback stops firing when set to null
ffmpeg.ts:
- Replace exec() with spawn() for ffmpeg/ffprobe — no shell interpreter,
no injection risk from filenames with special characters
- Filter invalid ranges in computeIdleSections() after clamping to video
bounds (handles timestamps exceeding video duration)
- Use path.basename() for timer IDs instead of split('/') for Windows compat
executor.ts:
- Use try/finally around VM execution so execution timestamps are recorded
even when code throws — the execution still occupied real time that should
not be sped up in the demo video
- Snapshot recordingStartedAt before try to avoid race if stopRecording()
is called inside the same execute() clearing the field
Tommy D. Rossi