feat: harden relay sessions and recording UX across extension + playwriter
This commit batches the pending workspace updates that were coupled in practice: extension reliability fixes, ghost cursor persistence across MPA navigations, timeout tuning, and aligned docs/versioning updates so behavior and guidance stay in sync. Key updates:\n- extension: add MV3 keepalive via chrome.alarms, add alarms permission, and bump extension changelog/version to 0.0.73 to reduce silent relay disconnects while idle\n- playwriter runtime: raise default action timeout to 60s while keeping navigation timeout separate; increase relay /version fetch timeout to 2000ms\n- recording + cursor: add persistent cursor init scripts with Page.addScriptToEvaluateOnNewDocument, remove init script on teardown, and ensure cursor enablement before applying mouse actions\n- demo video defaults: use 0.5s side buffer (1s total) and default idle speed 6x for createDemoVideo/computeIdleSections\n- tests/docs: keep click-error tests deterministic with explicit 5000ms per-click timeout, remove outdated click-timeout section from skill mistakes, and update package/changelog entries
This commit is contained in:
@@ -1,5 +1,11 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 0.0.73
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- **Service worker keepalive via chrome.alarms**: Added `chrome.alarms` keepalive to prevent Chrome MV3 from terminating the service worker when idle. Without this, the `maintainLoop` stops, the WebSocket closes, and the extension silently disconnects from the relay server — causing `session new` to fail with "Extension did not connect within timeout."
|
||||||
|
|
||||||
## 0.0.72
|
## 0.0.72
|
||||||
|
|
||||||
### Bug Fixes
|
### Bug Fixes
|
||||||
|
|||||||
@@ -1,9 +1,10 @@
|
|||||||
{
|
{
|
||||||
"manifest_version": 3,
|
"manifest_version": 3,
|
||||||
"name": "Playwriter",
|
"name": "Playwriter",
|
||||||
"version": "0.0.72",
|
"version": "0.0.73",
|
||||||
"description": "Automate your Browser using Cursor, Claude, VS Code. More capable and context efficient than Playwright MCP.",
|
"description": "Automate your Browser using Cursor, Claude, VS Code. More capable and context efficient than Playwright MCP.",
|
||||||
"permissions": [
|
"permissions": [
|
||||||
|
"alarms",
|
||||||
"debugger",
|
"debugger",
|
||||||
"tabGroups",
|
"tabGroups",
|
||||||
"contextMenus",
|
"contextMenus",
|
||||||
|
|||||||
@@ -1483,6 +1483,18 @@ async function onActionClicked(tab: chrome.tabs.Tab): Promise<void> {
|
|||||||
resetDebugger()
|
resetDebugger()
|
||||||
connectionManager.maintainLoop()
|
connectionManager.maintainLoop()
|
||||||
|
|
||||||
|
// MV3 service workers can be terminated by Chrome despite setInterval.
|
||||||
|
// chrome.alarms is the only reliable way to keep a service worker alive.
|
||||||
|
// Without this, the maintainLoop stops, the WebSocket closes, and the
|
||||||
|
// extension silently disconnects from the relay server.
|
||||||
|
chrome.alarms.create('keepalive', { periodInMinutes: 0.25 })
|
||||||
|
chrome.alarms.onAlarm.addListener((alarm) => {
|
||||||
|
if (alarm.name === 'keepalive') {
|
||||||
|
// No-op — the alarm firing is enough to wake the service worker.
|
||||||
|
// The maintainLoop will handle reconnection if needed.
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
chrome.contextMenus
|
chrome.contextMenus
|
||||||
.remove('playwriter-pin-element')
|
.remove('playwriter-pin-element')
|
||||||
.catch(() => {})
|
.catch(() => {})
|
||||||
|
|||||||
@@ -1,7 +1,37 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## 0.0.84
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
- **Remove click-timeout section from `skill.md` mistakes list**: Deleted the dedicated post-click timeout section per docs cleanup request and renumbered the following mistake item.
|
||||||
|
|
||||||
|
## 0.0.83
|
||||||
|
|
||||||
|
### Docs
|
||||||
|
|
||||||
|
- **Correct click-timeout guidance in `skill.md`**: Replaced the incorrect "SPA does not fire load" explanation with the actual behavior: click succeeds, then Playwright times out during post-click navigation waiting when action timeout budget is exceeded.
|
||||||
|
|
||||||
|
## 0.0.82
|
||||||
|
|
||||||
|
### Improvements
|
||||||
|
|
||||||
|
- **Increase default action timeout to 60s**: Raised `context.setDefaultTimeout` from `5000ms` to `60000ms` to avoid false click failures on slower real-world flows where the interaction succeeds but Playwright post-action navigation waiting exceeds short budgets.
|
||||||
|
|
||||||
|
## 0.0.81
|
||||||
|
|
||||||
|
### Improvements
|
||||||
|
|
||||||
|
- **Increase default action timeout to 5s**: Raised `context.setDefaultTimeout` from `2000ms` to `5000ms` to reduce false `locator.click()` timeouts on slower navigation flows (for example GitHub/Turbo links) where the click succeeds but post-click navigation settling exceeds 2 seconds. Default navigation timeout remains `10000ms`.
|
||||||
|
|
||||||
## 0.0.80
|
## 0.0.80
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- **Relaxed relay server version check timeout**: Increased `getRelayServerVersion` timeout from 500ms to 2000ms to prevent false "server not running" detections that kill a healthy relay server and disconnect the extension. This was causing intermittent `session new` failures when the relay was briefly busy (e.g. processing recording chunks).
|
||||||
|
|
||||||
|
## 0.0.80 (previous)
|
||||||
|
|
||||||
### Improvements
|
### Improvements
|
||||||
|
|
||||||
- **Descriptive click timeout errors**: When `locator.click()` times out due to actionability failures, the error now includes the reason (e.g. "Element is not visible", "Element is not stable", "<button> intercepts pointer events") instead of just "Timeout exceeded."
|
- **Descriptive click timeout errors**: When `locator.click()` times out due to actionability failures, the error now includes the reason (e.g. "Element is not visible", "Element is not stable", "<button> intercepts pointer events") instead of just "Timeout exceeded."
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "playwriter",
|
"name": "playwriter",
|
||||||
"description": "",
|
"description": "",
|
||||||
"version": "0.0.80",
|
"version": "0.0.84",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"main": "dist/index.js",
|
"main": "dist/index.js",
|
||||||
"types": "dist/index.d.ts",
|
"types": "dist/index.d.ts",
|
||||||
|
|||||||
@@ -592,9 +592,10 @@ export class PlaywrightExecutor {
|
|||||||
const contexts = browser.contexts()
|
const contexts = browser.contexts()
|
||||||
const context = contexts.length > 0 ? contexts[0] : await browser.newContext()
|
const context = contexts.length > 0 ? contexts[0] : await browser.newContext()
|
||||||
|
|
||||||
// Action timeout (click, fill, hover, etc.) is short for fast agent failure.
|
// Action timeout (click, fill, hover, etc.) is longer to tolerate slower
|
||||||
// Navigation timeout (goto, reload) is longer since page loads are slower.
|
// SPA/Turbo navigations and post-click settling on real sites.
|
||||||
context.setDefaultTimeout(2000)
|
// Navigation timeout (goto, reload) remains separate.
|
||||||
|
context.setDefaultTimeout(60000)
|
||||||
context.setDefaultNavigationTimeout(10000)
|
context.setDefaultNavigationTimeout(10000)
|
||||||
|
|
||||||
context.on('page', (page) => {
|
context.on('page', (page) => {
|
||||||
@@ -674,9 +675,10 @@ export class PlaywrightExecutor {
|
|||||||
const contexts = browser.contexts()
|
const contexts = browser.contexts()
|
||||||
const context = contexts.length > 0 ? contexts[0] : await browser.newContext()
|
const context = contexts.length > 0 ? contexts[0] : await browser.newContext()
|
||||||
|
|
||||||
// Action timeout (click, fill, hover, etc.) is short for fast agent failure.
|
// Action timeout (click, fill, hover, etc.) is longer to tolerate slower
|
||||||
// Navigation timeout (goto, reload) is longer since page loads are slower.
|
// SPA/Turbo navigations and post-click settling on real sites.
|
||||||
context.setDefaultTimeout(2000)
|
// Navigation timeout (goto, reload) remains separate.
|
||||||
|
context.setDefaultTimeout(60000)
|
||||||
context.setDefaultNavigationTimeout(10000)
|
context.setDefaultNavigationTimeout(10000)
|
||||||
|
|
||||||
context.on('page', (page) => {
|
context.on('page', (page) => {
|
||||||
|
|||||||
@@ -14,8 +14,8 @@ import path from 'node:path'
|
|||||||
// Constants
|
// Constants
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
/** Seconds of normal-speed buffer kept before and after each execution */
|
/** Seconds of normal-speed buffer kept before and after each execution (0.5s each side = 1s total) */
|
||||||
export const INTERACTION_BUFFER_SECONDS = 1
|
export const INTERACTION_BUFFER_SECONDS = 0.5
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Hardware encoder detection
|
// Hardware encoder detection
|
||||||
@@ -635,13 +635,13 @@ export interface ExecutionTimestamp {
|
|||||||
export function computeIdleSections({
|
export function computeIdleSections({
|
||||||
executionTimestamps,
|
executionTimestamps,
|
||||||
totalDurationMs,
|
totalDurationMs,
|
||||||
speed = 5,
|
speed = 6,
|
||||||
bufferSeconds = INTERACTION_BUFFER_SECONDS,
|
bufferSeconds = INTERACTION_BUFFER_SECONDS,
|
||||||
}: {
|
}: {
|
||||||
executionTimestamps: ExecutionTimestamp[]
|
executionTimestamps: ExecutionTimestamp[]
|
||||||
/** Total recording duration in milliseconds (from stopRecording result) */
|
/** Total recording duration in milliseconds (from stopRecording result) */
|
||||||
totalDurationMs: number
|
totalDurationMs: number
|
||||||
/** Speed multiplier for idle sections (default 5) */
|
/** Speed multiplier for idle sections (default 6) */
|
||||||
speed?: number
|
speed?: number
|
||||||
/** Override the default buffer around each execution (seconds) */
|
/** Override the default buffer around each execution (seconds) */
|
||||||
bufferSeconds?: number
|
bufferSeconds?: number
|
||||||
@@ -711,7 +711,7 @@ export interface CreateDemoVideoOptions {
|
|||||||
durationMs: number
|
durationMs: number
|
||||||
/** Execution timestamps (from stopRecording result) */
|
/** Execution timestamps (from stopRecording result) */
|
||||||
executionTimestamps: ExecutionTimestamp[]
|
executionTimestamps: ExecutionTimestamp[]
|
||||||
/** Speed multiplier for idle sections (default 5) */
|
/** Speed multiplier for idle sections (default 6) */
|
||||||
speed?: number
|
speed?: number
|
||||||
/** Output file path (defaults to recordingPath with `-demo` suffix) */
|
/** Output file path (defaults to recordingPath with `-demo` suffix) */
|
||||||
outputFile?: string
|
outputFile?: string
|
||||||
@@ -722,8 +722,8 @@ export interface CreateDemoVideoOptions {
|
|||||||
* Create a demo video from a recording by speeding up idle sections
|
* Create a demo video from a recording by speeding up idle sections
|
||||||
* (gaps between execute() calls) while keeping interactions at normal speed.
|
* (gaps between execute() calls) while keeping interactions at normal speed.
|
||||||
*
|
*
|
||||||
* A 1-second buffer (INTERACTION_BUFFER_SECONDS) is preserved around each
|
* A 0.5-second buffer (INTERACTION_BUFFER_SECONDS) is preserved on each side of
|
||||||
* interaction so viewers see context before and after each action.
|
* an interaction (1 second total) so viewers see context before and after each action.
|
||||||
*
|
*
|
||||||
* Requires `ffmpeg` and `ffprobe` installed on the system.
|
* Requires `ffmpeg` and `ffprobe` installed on the system.
|
||||||
*
|
*
|
||||||
@@ -736,7 +736,7 @@ export async function createDemoVideo(
|
|||||||
recordingPath,
|
recordingPath,
|
||||||
durationMs,
|
durationMs,
|
||||||
executionTimestamps,
|
executionTimestamps,
|
||||||
speed = 5,
|
speed = 6,
|
||||||
signal,
|
signal,
|
||||||
} = options
|
} = options
|
||||||
|
|
||||||
|
|||||||
@@ -247,7 +247,8 @@ function applyRuntimeVisualOptions(): void {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (runtime.options.style === 'minimal') {
|
if (runtime.options.style === 'minimal') {
|
||||||
const triangleSvg = `<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 24 24"><path fill="${runtime.options.color}" d="m23.284 19.124l-6.866-6.895a.4.4 0 0 1-.118-.296a.43.43 0 0 1 .163-.282l4.439-3.077a1.48 1.48 0 0 0 .621-1.48a1.48 1.48 0 0 0-1.036-1.198L1.623.302a1.14 1.14 0 0 0-1.11.282A1.13 1.13 0 0 0 .29 1.649L5.928 20.44a1.48 1.48 0 0 0 1.183 1.035a1.48 1.48 0 0 0 1.48-.621l3.078-4.44a.37.37 0 0 1 .31-.118a.43.43 0 0 1 .296.104l6.91 6.91a1.48 1.48 0 0 0 2.087 0l2.086-2.086a1.48 1.48 0 0 0-.074-2.101"/></svg>`
|
// White fill with dark border stroke, like a standard macOS cursor
|
||||||
|
const triangleSvg = `<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="-1 -1 26 26"><path fill="white" stroke="${runtime.options.color}" stroke-width="1.5" stroke-linejoin="round" d="m23.284 19.124l-6.866-6.895a.4.4 0 0 1-.118-.296a.43.43 0 0 1 .163-.282l4.439-3.077a1.48 1.48 0 0 0 .621-1.48a1.48 1.48 0 0 0-1.036-1.198L1.623.302a1.14 1.14 0 0 0-1.11.282A1.13 1.13 0 0 0 .29 1.649L5.928 20.44a1.48 1.48 0 0 0 1.183 1.035a1.48 1.48 0 0 0 1.48-.621l3.078-4.44a.37.37 0 0 1 .31-.118a.43.43 0 0 1 .296.104l6.91 6.91a1.48 1.48 0 0 0 2.087 0l2.086-2.086a1.48 1.48 0 0 0-.074-2.101"/></svg>`
|
||||||
const triangleDataUrl = `url("data:image/svg+xml,${encodeURIComponent(triangleSvg)}")`
|
const triangleDataUrl = `url("data:image/svg+xml,${encodeURIComponent(triangleSvg)}")`
|
||||||
runtime.cursorElement.style.borderRadius = '0'
|
runtime.cursorElement.style.borderRadius = '0'
|
||||||
runtime.cursorElement.style.border = 'none'
|
runtime.cursorElement.style.border = 'none'
|
||||||
@@ -258,7 +259,7 @@ function applyRuntimeVisualOptions(): void {
|
|||||||
runtime.cursorElement.style.backgroundPosition = 'left top'
|
runtime.cursorElement.style.backgroundPosition = 'left top'
|
||||||
runtime.cursorElement.style.backdropFilter = 'none'
|
runtime.cursorElement.style.backdropFilter = 'none'
|
||||||
runtime.cursorElement.style.boxShadow = 'none'
|
runtime.cursorElement.style.boxShadow = 'none'
|
||||||
runtime.cursorElement.style.filter = 'drop-shadow(0 1px 1px rgba(0, 0, 0, 0.3))'
|
runtime.cursorElement.style.filter = 'drop-shadow(0 1px 2px rgba(0, 0, 0, 0.4))'
|
||||||
runtime.cursorElement.style.opacity = getBaseOpacity()
|
runtime.cursorElement.style.opacity = getBaseOpacity()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,12 +1,19 @@
|
|||||||
/**
|
/**
|
||||||
* Node-side ghost cursor helpers.
|
* Node-side ghost cursor helpers.
|
||||||
* Injects the browser bundle and forwards mouse action events to the page overlay.
|
* Injects the browser bundle and forwards mouse action events to the page overlay.
|
||||||
|
*
|
||||||
|
* Two injection strategies:
|
||||||
|
* 1. One-shot: page.evaluate() — used by enableGhostCursor() for single pages
|
||||||
|
* 2. Persistent: Page.addScriptToEvaluateOnNewDocument — used by
|
||||||
|
* addGhostCursorInitScript() so the cursor survives MPA navigations.
|
||||||
|
* Chrome re-runs the script on every new document automatically.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import fs from 'node:fs'
|
import fs from 'node:fs'
|
||||||
import path from 'node:path'
|
import path from 'node:path'
|
||||||
import { fileURLToPath } from 'node:url'
|
import { fileURLToPath } from 'node:url'
|
||||||
import type { Page, MouseActionEvent } from '@xmorse/playwright-core'
|
import type { Page, MouseActionEvent } from '@xmorse/playwright-core'
|
||||||
|
import { getCDPSessionForPage, type ICDPSession } from './cdp-session.js'
|
||||||
|
|
||||||
export interface GhostCursorClientOptions {
|
export interface GhostCursorClientOptions {
|
||||||
style?: 'minimal' | 'dot' | 'screenstudio'
|
style?: 'minimal' | 'dot' | 'screenstudio'
|
||||||
@@ -23,6 +30,7 @@ interface GhostCursorBrowserApi {
|
|||||||
enable: (options?: GhostCursorClientOptions) => void
|
enable: (options?: GhostCursorClientOptions) => void
|
||||||
disable: () => void
|
disable: () => void
|
||||||
applyMouseAction: (event: MouseActionEvent) => void
|
applyMouseAction: (event: MouseActionEvent) => void
|
||||||
|
isEnabled: () => boolean
|
||||||
}
|
}
|
||||||
|
|
||||||
let ghostCursorCode: string | null = null
|
let ghostCursorCode: string | null = null
|
||||||
@@ -76,34 +84,84 @@ export async function disableGhostCursor(options: { page: Page }): Promise<void>
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Register the ghost cursor bundle as a persistent init script via CDP
|
||||||
|
* Page.addScriptToEvaluateOnNewDocument. Chrome re-runs this on every new
|
||||||
|
* document (navigation), so the cursor survives MPA page loads.
|
||||||
|
*
|
||||||
|
* The script auto-enables when the DOM is ready (DOMContentLoaded or
|
||||||
|
* requestAnimationFrame fallback) so the cursor is visible immediately
|
||||||
|
* on each new page.
|
||||||
|
*
|
||||||
|
* Returns the CDP identifier needed to remove it later.
|
||||||
|
*/
|
||||||
|
export async function addGhostCursorInitScript(options: {
|
||||||
|
page: Page
|
||||||
|
cursorOptions?: GhostCursorClientOptions
|
||||||
|
}): Promise<{ cdp: ICDPSession; identifier: string }> {
|
||||||
|
const { page, cursorOptions } = options
|
||||||
|
const cdp = await getCDPSessionForPage({ page })
|
||||||
|
const code = getGhostCursorCode()
|
||||||
|
|
||||||
|
// Wrap the bundle: inject, then auto-enable once DOM is ready.
|
||||||
|
// DOMContentLoaded may have already fired if the script runs late,
|
||||||
|
// so check document.readyState first.
|
||||||
|
const optionsJson = JSON.stringify(cursorOptions ?? {})
|
||||||
|
const wrappedCode = `
|
||||||
|
${code}
|
||||||
|
;(function() {
|
||||||
|
function autoEnable() {
|
||||||
|
var api = globalThis.__playwriterGhostCursor;
|
||||||
|
if (api) {
|
||||||
|
var opts = ${optionsJson};
|
||||||
|
api.enable(Object.keys(opts).length ? opts : undefined);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (document.readyState === 'loading') {
|
||||||
|
document.addEventListener('DOMContentLoaded', autoEnable, { once: true });
|
||||||
|
} else {
|
||||||
|
requestAnimationFrame(autoEnable);
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
`
|
||||||
|
|
||||||
|
const result = await cdp.send('Page.addScriptToEvaluateOnNewDocument', {
|
||||||
|
source: wrappedCode,
|
||||||
|
})
|
||||||
|
|
||||||
|
return { cdp, identifier: result.identifier }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Remove a previously registered ghost cursor init script.
|
||||||
|
*/
|
||||||
|
export async function removeGhostCursorInitScript(options: {
|
||||||
|
cdp: ICDPSession
|
||||||
|
identifier: string
|
||||||
|
}): Promise<void> {
|
||||||
|
const { cdp, identifier } = options
|
||||||
|
await cdp.send('Page.removeScriptToEvaluateOnNewDocument', { identifier })
|
||||||
|
}
|
||||||
|
|
||||||
export async function applyGhostCursorMouseAction(options: {
|
export async function applyGhostCursorMouseAction(options: {
|
||||||
page: Page
|
page: Page
|
||||||
event: MouseActionEvent
|
event: MouseActionEvent
|
||||||
}): Promise<void> {
|
}): Promise<void> {
|
||||||
const { page, event } = options
|
const { page, event } = options
|
||||||
|
|
||||||
const applied = await page.evaluate(
|
|
||||||
({ serializedEvent }) => {
|
|
||||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
|
||||||
if (!api) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
api.applyMouseAction(serializedEvent)
|
|
||||||
return true
|
|
||||||
},
|
|
||||||
{ serializedEvent: event },
|
|
||||||
)
|
|
||||||
|
|
||||||
if (applied) {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
await ensureGhostCursorInjected({ page })
|
|
||||||
await page.evaluate(
|
await page.evaluate(
|
||||||
({ serializedEvent }) => {
|
({ serializedEvent }) => {
|
||||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
||||||
api?.applyMouseAction(serializedEvent)
|
if (!api) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure cursor is enabled (may be a freshly injected init script
|
||||||
|
// where DOMContentLoaded hasn't fired yet)
|
||||||
|
if (!api.isEnabled()) {
|
||||||
|
api.enable()
|
||||||
|
}
|
||||||
|
api.applyMouseAction(serializedEvent)
|
||||||
},
|
},
|
||||||
{ serializedEvent: event },
|
{ serializedEvent: event },
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,10 +1,21 @@
|
|||||||
/**
|
/**
|
||||||
* Encapsulates ghost cursor lifecycle for recording sessions.
|
* Encapsulates ghost cursor lifecycle for recording sessions.
|
||||||
* Keeps onMouseAction chaining/restoration isolated from executor logic.
|
* Keeps onMouseAction chaining/restoration isolated from executor logic.
|
||||||
|
*
|
||||||
|
* Uses Page.addScriptToEvaluateOnNewDocument so the cursor persists across
|
||||||
|
* MPA navigations — Chrome re-runs the init script on every new document.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import type { BrowserContext, Page } from '@xmorse/playwright-core'
|
import type { BrowserContext, Page } from '@xmorse/playwright-core'
|
||||||
import { applyGhostCursorMouseAction, disableGhostCursor, enableGhostCursor, type GhostCursorClientOptions } from './ghost-cursor.js'
|
import type { ICDPSession } from './cdp-session.js'
|
||||||
|
import {
|
||||||
|
addGhostCursorInitScript,
|
||||||
|
applyGhostCursorMouseAction,
|
||||||
|
disableGhostCursor,
|
||||||
|
enableGhostCursor,
|
||||||
|
removeGhostCursorInitScript,
|
||||||
|
type GhostCursorClientOptions,
|
||||||
|
} from './ghost-cursor.js'
|
||||||
|
|
||||||
interface RecordingGhostCursorLogger {
|
interface RecordingGhostCursorLogger {
|
||||||
error: (...args: unknown[]) => void
|
error: (...args: unknown[]) => void
|
||||||
@@ -15,9 +26,15 @@ interface RecordingTargetOptions {
|
|||||||
sessionId?: string
|
sessionId?: string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
interface InitScriptHandle {
|
||||||
|
cdp: ICDPSession
|
||||||
|
identifier: string
|
||||||
|
}
|
||||||
|
|
||||||
export class RecordingGhostCursorController {
|
export class RecordingGhostCursorController {
|
||||||
private readonly previousMouseActionByPage = new WeakMap<Page, Page['onMouseAction']>()
|
private readonly previousMouseActionByPage = new WeakMap<Page, Page['onMouseAction']>()
|
||||||
private readonly cursorApplyQueueByPage = new WeakMap<Page, Promise<void>>()
|
private readonly cursorApplyQueueByPage = new WeakMap<Page, Promise<void>>()
|
||||||
|
private readonly initScriptByPage = new WeakMap<Page, InitScriptHandle>()
|
||||||
private readonly logger: RecordingGhostCursorLogger
|
private readonly logger: RecordingGhostCursorLogger
|
||||||
|
|
||||||
constructor(options: { logger: RecordingGhostCursorLogger }) {
|
constructor(options: { logger: RecordingGhostCursorLogger }) {
|
||||||
@@ -52,7 +69,14 @@ export class RecordingGhostCursorController {
|
|||||||
const { page } = options
|
const { page } = options
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
// Register persistent init script so cursor survives MPA navigations
|
||||||
|
const handle = await addGhostCursorInitScript({ page })
|
||||||
|
this.initScriptByPage.set(page, handle)
|
||||||
|
|
||||||
|
// Also enable on the current page immediately (init script only runs
|
||||||
|
// on future navigations, not the current document)
|
||||||
await enableGhostCursor({ page })
|
await enableGhostCursor({ page })
|
||||||
|
|
||||||
if (!this.previousMouseActionByPage.has(page)) {
|
if (!this.previousMouseActionByPage.has(page)) {
|
||||||
this.previousMouseActionByPage.set(page, page.onMouseAction)
|
this.previousMouseActionByPage.set(page, page.onMouseAction)
|
||||||
}
|
}
|
||||||
@@ -88,6 +112,17 @@ export class RecordingGhostCursorController {
|
|||||||
this.previousMouseActionByPage.delete(page)
|
this.previousMouseActionByPage.delete(page)
|
||||||
this.cursorApplyQueueByPage.delete(page)
|
this.cursorApplyQueueByPage.delete(page)
|
||||||
|
|
||||||
|
// Remove the persistent init script so future navigations don't inject cursor
|
||||||
|
const handle = this.initScriptByPage.get(page)
|
||||||
|
if (handle) {
|
||||||
|
try {
|
||||||
|
await removeGhostCursorInitScript(handle)
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error('[playwriter] Failed to remove ghost cursor init script', error)
|
||||||
|
}
|
||||||
|
this.initScriptByPage.delete(page)
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
await disableGhostCursor({ page })
|
await disableGhostCursor({ page })
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ export type ExtensionStatus = {
|
|||||||
export async function getRelayServerVersion(port: number = RELAY_PORT): Promise<string | null> {
|
export async function getRelayServerVersion(port: number = RELAY_PORT): Promise<string | null> {
|
||||||
try {
|
try {
|
||||||
const response = await fetch(`http://127.0.0.1:${port}/version`, {
|
const response = await fetch(`http://127.0.0.1:${port}/version`, {
|
||||||
signal: AbortSignal.timeout(500),
|
signal: AbortSignal.timeout(2000),
|
||||||
})
|
})
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
return null
|
return null
|
||||||
|
|||||||
@@ -1039,14 +1039,14 @@ describe('Relay Core Tests', () => {
|
|||||||
name: 'execute',
|
name: 'execute',
|
||||||
arguments: {
|
arguments: {
|
||||||
code: js`
|
code: js`
|
||||||
await state.errorTestPage.click('#hidden-btn');
|
await state.errorTestPage.click('#hidden-btn', { timeout: 5000 });
|
||||||
`,
|
`,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
const text = (result as any).content[0].text
|
const text = (result as any).content[0].text
|
||||||
// Strip stack traces and call logs to only match the descriptive error line
|
// Strip stack traces and call logs to only match the descriptive error line
|
||||||
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('not visible') || l.includes('not stable'))
|
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('not visible') || l.includes('not stable'))
|
||||||
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 2000ms exceeded. Element is not visible — it may be hidden by CSS, inside a collapsed <details>, inactive tab, or closed accordion. Try: interact with the page to reveal it first, or use { force: true } to skip visibility checks"`)
|
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 5000ms exceeded. Element is not visible — it may be hidden by CSS, inside a collapsed <details>, inactive tab, or closed accordion. Try: interact with the page to reveal it first, or use { force: true } to skip visibility checks"`)
|
||||||
expect((result as any).isError).toBe(true)
|
expect((result as any).isError).toBe(true)
|
||||||
// Cleanup
|
// Cleanup
|
||||||
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
||||||
@@ -1071,13 +1071,13 @@ describe('Relay Core Tests', () => {
|
|||||||
name: 'execute',
|
name: 'execute',
|
||||||
arguments: {
|
arguments: {
|
||||||
code: js`
|
code: js`
|
||||||
await state.errorTestPage.click('#covered-btn');
|
await state.errorTestPage.click('#covered-btn', { timeout: 5000 });
|
||||||
`,
|
`,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
const text = (result as any).content[0].text
|
const text = (result as any).content[0].text
|
||||||
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('intercepts'))
|
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('intercepts'))
|
||||||
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 2000ms exceeded. <div id="overlay">Overlay</div> intercepts pointer events"`)
|
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 5000ms exceeded. <div id=\"overlay\">Overlay</div> intercepts pointer events"`)
|
||||||
expect((result as any).isError).toBe(true)
|
expect((result as any).isError).toBe(true)
|
||||||
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
||||||
}, 30000)
|
}, 30000)
|
||||||
@@ -1096,13 +1096,13 @@ describe('Relay Core Tests', () => {
|
|||||||
name: 'execute',
|
name: 'execute',
|
||||||
arguments: {
|
arguments: {
|
||||||
code: js`
|
code: js`
|
||||||
await state.errorTestPage.click('#invisible');
|
await state.errorTestPage.click('#invisible', { timeout: 5000 });
|
||||||
`,
|
`,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
const text = (result as any).content[0].text
|
const text = (result as any).content[0].text
|
||||||
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('not visible'))
|
const errorLine = text.split('\n').find((l: string) => l.includes('Timeout') || l.includes('not visible'))
|
||||||
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 2000ms exceeded. Element is not visible — it may be hidden by CSS, inside a collapsed <details>, inactive tab, or closed accordion. Try: interact with the page to reveal it first, or use { force: true } to skip visibility checks"`)
|
expect(errorLine).toMatchInlineSnapshot(`"Error executing code: page.click: Timeout 5000ms exceeded. Element is not visible — it may be hidden by CSS, inside a collapsed <details>, inactive tab, or closed accordion. Try: interact with the page to reveal it first, or use { force: true } to skip visibility checks"`)
|
||||||
expect((result as any).isError).toBe(true)
|
expect((result as any).isError).toBe(true)
|
||||||
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
await client.callTool({ name: 'execute', arguments: { code: js`await state.errorTestPage.close(); delete state.errorTestPage;` } })
|
||||||
}, 30000)
|
}, 30000)
|
||||||
|
|||||||
+1
-25
@@ -440,31 +440,7 @@ await state.page.evaluate(() => document.querySelector('button').click())
|
|||||||
await state.page.getByRole('radio', { name: 'Node.js' }).click()
|
await state.page.getByRole('radio', { name: 'Node.js' }).click()
|
||||||
```
|
```
|
||||||
|
|
||||||
**13. SPA/Turbo navigation makes `click()` time out**
|
**13. Over-investigating instead of just interacting**
|
||||||
Sites using Turbo (GitHub), Next.js soft nav, or other SPA routers intercept link clicks and update the URL without firing the browser's standard `load` event. Playwright's `click()` sees the navigation start and waits for it to complete — but the load event never fires, so it times out even though the navigation succeeded.
|
|
||||||
|
|
||||||
Symptoms: `locator.click: Timeout Xms exceeded` with logs showing `navigated to "..."` underneath — the click worked, Playwright just timed out waiting for cleanup.
|
|
||||||
|
|
||||||
Fix: add `.catch(() => {})` on the click, or raise `timeout`, or use `{ noWaitAfter: true }` then wait manually:
|
|
||||||
|
|
||||||
```js
|
|
||||||
// Pattern 1: swallow the navigation-wait timeout (click still fired)
|
|
||||||
await state.page.locator('a.notification-link').first().click({ timeout: 10000 }).catch(() => {})
|
|
||||||
await state.page.waitForTimeout(1500) // give SPA time to render
|
|
||||||
|
|
||||||
// Pattern 2: skip the post-click navigation wait entirely
|
|
||||||
await state.page.locator('a.notification-link').first().click({ noWaitAfter: true })
|
|
||||||
await state.page.waitForLoadState('domcontentloaded').catch(() => {})
|
|
||||||
await state.page.waitForTimeout(1000)
|
|
||||||
|
|
||||||
// Pattern 3: catch at waitForLoadState instead
|
|
||||||
await state.page.locator('a').first().click()
|
|
||||||
await state.page.waitForLoadState('domcontentloaded').catch(() => {})
|
|
||||||
```
|
|
||||||
|
|
||||||
Always verify the URL afterwards with `console.log(state.page.url())` — the navigation typically did happen.
|
|
||||||
|
|
||||||
**14. Over-investigating instead of just interacting**
|
|
||||||
When something doesn't respond to a click, do NOT start inspecting CDP event listeners, React fibers, canvas pixel data, or writing `page.evaluate()` to read class names and bounding boxes. This wastes massive context. Instead:
|
When something doesn't respond to a click, do NOT start inspecting CDP event listeners, React fibers, canvas pixel data, or writing `page.evaluate()` to read class names and bounding boxes. This wastes massive context. Instead:
|
||||||
|
|
||||||
1. Take a `snapshot()` — it shows every interactive element and what to click
|
1. Take a `snapshot()` — it shows every interactive element and what to click
|
||||||
|
|||||||
Reference in New Issue
Block a user