diff --git a/playwriter/CHANGELOG.md b/playwriter/CHANGELOG.md index 274d678..6503328 100644 --- a/playwriter/CHANGELOG.md +++ b/playwriter/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## 0.0.79 + +### Improvements + +- **Faster ghost cursor motion defaults**: Reduced min/max movement durations and increased base movement speed so pointer travel feels snappier while preserving smooth easing. +- **Recording docs now emphasize interaction-driven navigation**: Updated skill guidance to prefer click/type/hover flows during recordings so ghost cursor motion is visible and human-like instead of bypassed by direct `goto` jumps. + ## 0.0.78 ### Features diff --git a/playwriter/package.json b/playwriter/package.json index dbe6ab4..8194a59 100644 --- a/playwriter/package.json +++ b/playwriter/package.json @@ -1,7 +1,7 @@ { "name": "playwriter", "description": "", - "version": "0.0.78", + "version": "0.0.79", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", diff --git a/playwriter/scripts/encode-screenstudio-cursor.ts b/playwriter/scripts/encode-screenstudio-cursor.ts index d46e46e..7849908 100644 --- a/playwriter/scripts/encode-screenstudio-cursor.ts +++ b/playwriter/scripts/encode-screenstudio-cursor.ts @@ -6,9 +6,10 @@ import fs from 'node:fs' import path from 'node:path' import { fileURLToPath } from 'node:url' -const currentDir = path.dirname(fileURLToPath(import.meta.url)) -const sourcePath = path.join(currentDir, '..', 'src', 'assets', 'cursors', 'screen-studio', 'pointer-macos-tahoe.svg') -const outputPath = path.join(currentDir, '..', 'src', 'assets', 'cursors', 'screen-studio', 'pointer-macos-tahoe-data-url.ts') +const __filename = fileURLToPath(import.meta.url) +const __dirname = path.dirname(__filename) +const sourcePath = path.resolve(__dirname, '../src/assets/cursors/screen-studio/pointer-macos-tahoe.svg') +const outputPath = path.resolve(__dirname, '../src/assets/cursors/screen-studio/pointer-macos-tahoe-data-url.ts') function main() { const svg = fs.readFileSync(sourcePath, 'utf-8').trim() diff --git a/playwriter/src/ghost-cursor-client.ts b/playwriter/src/ghost-cursor-client.ts index 831de1c..c536614 100644 --- a/playwriter/src/ghost-cursor-client.ts +++ b/playwriter/src/ghost-cursor-client.ts @@ -71,10 +71,10 @@ const DEFAULT_OPTIONS: GhostCursorRuntimeOptions = { color: '#111827', size: 22, zIndex: 2147483647, - easing: 'cubic-bezier(0.22, 1, 0.36, 1)', - minDurationMs: 40, - maxDurationMs: 450, - speedPxPerMs: 2.2, + easing: 'cubic-bezier(0.16, 1, 0.3, 1)', + minDurationMs: 24, + maxDurationMs: 320, + speedPxPerMs: 4, } const runtime: GhostCursorRuntimeState = { diff --git a/playwriter/src/skill.md b/playwriter/src/skill.md index 4ee43a8..db112bd 100644 --- a/playwriter/src/skill.md +++ b/playwriter/src/skill.md @@ -408,32 +408,23 @@ await loginPage.waitForURL('**/callback**') // Original page should now be authenticated ``` -**11. Click times out or does nothing — snapshot first to find the blocking dialog** -When a click times out or has no visible effect, the most common cause is a **modal or overlay intercepting pointer events**. Do not retry with different selectors or `{ force: true }` — snapshot immediately to find the blocker, then interact with it using its own snapshot locators: +**11. Click times out or does nothing — snapshot to find the blocker** +When a click times out, a **modal or overlay** is likely intercepting pointer events. Do not retry with different selectors or `{ force: true }` — snapshot to find the blocker: ```js -// BAD: click timed out → retry with force:true (still blocked by the overlay) -await state.page.locator('button[name="Create Project"]').click({ force: true }) - -// GOOD: click timed out → snapshot first -const snap = await snapshot({ page: state.page, search: /dialog|modal/i }) -// Found: dialog > heading "Do you use a framework?" → interact with it properly +// click timed out → don't retry blindly, find what's blocking +await snapshot({ page: state.page, search: /dialog|modal/i }) +// Found modal → interact with it properly (don't just close via X, it may reappear) await state.page.getByRole('radio', { name: 'Nope, Vanilla' }).click() -await state.page.getByRole('button', { name: 'Configure SDK' }).click() -// Now the button is unblocked -await state.page.getByRole('button', { name: 'Create Project' }).click() ``` -**12. `dispatchEvent` and `{ force: true }` do not work on React SPAs** -React uses a synthetic event system. Raw DOM `dispatchEvent(new MouseEvent(...))` and Playwright's `{ force: true }` bypass actionability checks but **do not trigger React event handlers** — component state won't update. If a click appears to succeed but nothing changes, you are clicking the wrong DOM node. Use the snapshot to find the real interactive element (`role=radio`, `role=button`), not a heading or wrapper div: +**12. Never use `dispatchEvent` or `{ force: true }` to bypass blockers** +`dispatchEvent(new MouseEvent(...))` and `{ force: true }` bypass Playwright checks but **do not trigger React/Vue/Svelte handlers** — state won't update. If a click "succeeds" but nothing changes, you're clicking the wrong node: ```js -// BAD: H3 heading is not what React listens to — state won't update -await state.page.evaluate(() => h3El.dispatchEvent(new MouseEvent('click', { bubbles: true }))) +// BAD: heading click bypasses overlay but React ignores it await state.page.locator('h3:has-text("Node.js")').click({ force: true }) - -// GOOD: snapshot shows the real interactive element -// - role=radio[name="Node.js"] ← React's event handler is here +// GOOD: snapshot shows the real interactive element is a radio, not the heading await state.page.getByRole('radio', { name: 'Node.js' }).click() ``` @@ -480,28 +471,16 @@ Each interactive line ends with a Playwright locator you can pass to `state.page If multiple elements share the same locator, a `>> nth=N` suffix is added (0-based) to make it unique. -**Use snapshot locators directly — never invent selectors.** The locator string shown in the snapshot IS the selector. Use it immediately with `getByRole` or `locator()`. Do not guess CSS selectors, `heading >> text=...`, or `getByText` when the snapshot already gives you the exact string: +**Use snapshot locators directly — never invent selectors.** The snapshot output IS the selector. Do not guess CSS selectors or `getByText` when the snapshot already gives you the exact match: ```js -// Snapshot shows: - role=radio[name="Nope, Vanilla"] +// Snapshot shows: role=radio[name="Nope, Vanilla"] → use it directly await state.page.getByRole('radio', { name: 'Nope, Vanilla' }).click() - -// Snapshot shows: - role=button[name="Configure SDK"] -await state.page.getByRole('button', { name: 'Configure SDK' }).click() - -// Snapshot shows: - role=link[name="SIGN IN"] +// Snapshot shows: role=link[name="SIGN IN"] → or pass raw string to locator() await state.page.locator('role=link[name="SIGN IN"]').click() ``` -**SPA CSS text-transform case mismatch**: accessibility snapshots reflect the visual text, which may be uppercase due to CSS `text-transform`. The actual DOM value (and what `getByRole` matches on) may differ. Use case-insensitive regex to be safe: - -```js -// Snapshot shows heading "NODE.JS" but real DOM value is "Node.js" -// BAD: exact string may not match -await state.page.getByRole('heading', { name: 'NODE.JS' }) -// GOOD: case-insensitive regex always works -await state.page.getByRole('heading', { name: /node\.js/i }) -``` +**Beware CSS text-transform**: snapshots show visual text (`heading "NODE.JS"`) but DOM may be `"Node.js"`. Use case-insensitive regex: `getByRole('heading', { name: /node\.js/i })`. If a screenshot shows ref labels like `e3`, resolve them using the last snapshot: diff --git a/website/public/SKILL.md b/website/public/SKILL.md index 4afa9bf..150c407 100644 --- a/website/public/SKILL.md +++ b/website/public/SKILL.md @@ -408,6 +408,35 @@ await loginPage.waitForURL('**/callback**') // Original page should now be authenticated ``` +**11. Click times out or does nothing — snapshot first to find the blocking dialog** +When a click times out or has no visible effect, the most common cause is a **modal or overlay intercepting pointer events**. Do not retry with different selectors or `{ force: true }` — snapshot immediately to find the blocker, then interact with it using its own snapshot locators: + +```js +// BAD: click timed out → retry with force:true (still blocked by the overlay) +await state.page.locator('button[name="Create Project"]').click({ force: true }) + +// GOOD: click timed out → snapshot first +const snap = await snapshot({ page: state.page, search: /dialog|modal/i }) +// Found: dialog > heading "Do you use a framework?" → interact with it properly +await state.page.getByRole('radio', { name: 'Nope, Vanilla' }).click() +await state.page.getByRole('button', { name: 'Configure SDK' }).click() +// Now the button is unblocked +await state.page.getByRole('button', { name: 'Create Project' }).click() +``` + +**12. `dispatchEvent` and `{ force: true }` do not work on React SPAs** +React uses a synthetic event system. Raw DOM `dispatchEvent(new MouseEvent(...))` and Playwright's `{ force: true }` bypass actionability checks but **do not trigger React event handlers** — component state won't update. If a click appears to succeed but nothing changes, you are clicking the wrong DOM node. Use the snapshot to find the real interactive element (`role=radio`, `role=button`), not a heading or wrapper div: + +```js +// BAD: H3 heading is not what React listens to — state won't update +await state.page.evaluate(() => h3El.dispatchEvent(new MouseEvent('click', { bubbles: true }))) +await state.page.locator('h3:has-text("Node.js")').click({ force: true }) + +// GOOD: snapshot shows the real interactive element +// - role=radio[name="Node.js"] ← React's event handler is here +await state.page.getByRole('radio', { name: 'Node.js' }).click() +``` + ## checking page state After any action (click, submit, navigate), verify what happened. Always print URL first, then snapshot: @@ -451,6 +480,29 @@ Each interactive line ends with a Playwright locator you can pass to `state.page If multiple elements share the same locator, a `>> nth=N` suffix is added (0-based) to make it unique. +**Use snapshot locators directly — never invent selectors.** The locator string shown in the snapshot IS the selector. Use it immediately with `getByRole` or `locator()`. Do not guess CSS selectors, `heading >> text=...`, or `getByText` when the snapshot already gives you the exact string: + +```js +// Snapshot shows: - role=radio[name="Nope, Vanilla"] +await state.page.getByRole('radio', { name: 'Nope, Vanilla' }).click() + +// Snapshot shows: - role=button[name="Configure SDK"] +await state.page.getByRole('button', { name: 'Configure SDK' }).click() + +// Snapshot shows: - role=link[name="SIGN IN"] +await state.page.locator('role=link[name="SIGN IN"]').click() +``` + +**SPA CSS text-transform case mismatch**: accessibility snapshots reflect the visual text, which may be uppercase due to CSS `text-transform`. The actual DOM value (and what `getByRole` matches on) may differ. Use case-insensitive regex to be safe: + +```js +// Snapshot shows heading "NODE.JS" but real DOM value is "Node.js" +// BAD: exact string may not match +await state.page.getByRole('heading', { name: 'NODE.JS' }) +// GOOD: case-insensitive regex always works +await state.page.getByRole('heading', { name: /node\.js/i }) +``` + If a screenshot shows ref labels like `e3`, resolve them using the last snapshot: ```js @@ -861,10 +913,21 @@ await screenshotWithAccessibilityLabels({ page: state.page }) Labels are color-coded: yellow=links, orange=buttons, coral=inputs, pink=checkboxes, peach=sliders, salmon=menus, amber=tabs. +**resizeImage** - shrink an image so it consumes fewer tokens when read back into context. Overwrites the input file by default. Fits within 1568×1568px (Claude-optimal). Always outputs JPEG. + +```js +// Shrink screenshot in-place for LLM ingestion +await resizeImage({ input: './screenshot.png' }) +``` + +Also supports explicit dimensions: `width`, `height`, `maxDimension`, `fit` ('inside' | 'cover' | 'contain' | 'fill'), `quality` (1-100, default 80), `output` (defaults to overwriting input). + **recording.start / recording.stop** - record the page as a video at native FPS (30-60fps). Uses `chrome.tabCapture` in the extension context, so **recording survives page navigation**. Video is saved as mp4. While recording is active, Playwriter automatically overlays a smooth ghost cursor that follows automated mouse actions (`page.mouse.*`, `locator.click()`, hover flows) using `page.onMouseAction` from the Playwright fork. +For demos where cursor movement should be visible and human-like, drive the page with interaction methods (`locator.click()`, `page.click()`, `page.mouse.move()`, `press`, typing). Avoid skipping interactions with direct state jumps (for example, `goto(itemUrl)` instead of clicking the link) when your goal is to show realistic pointer motion in the recording. + **Note**: Recording requires the user to have clicked the Playwriter extension icon on the tab. This grants `activeTab` permission needed for `chrome.tabCapture`. Recording works on tabs where the icon was clicked - if you need to record a new tab, ask the user to click the icon on it first. ```js @@ -964,7 +1027,11 @@ Always use `scale: 'css'` to avoid 2-4x larger images on high-DPI displays: await state.page.screenshot({ path: 'shot.png', scale: 'css' }) ``` -If you want to read back the image file into context make sure to resize it first, scaling down the image to make sure max size is 1500px. for example with `sips --resampleHeightWidthMax 1500 input.png --out output.png` on macOS. +If you want to read back the image file into context, resize it first so it consumes fewer tokens: + +```js +await resizeImage({ input: './shot.png' }) +``` ## page.evaluate