feat: add ghost cursor APIs and harden reconnect flow
Introduce a first-class ghost cursor system for recordings and screenshots with namespace APIs (, ) while preserving backward-compatible top-level recording helpers. Improve reliability after relay restarts by consolidating extension wait logic around , reducing false disconnected states during and execute flows. Refactor recording glue into dedicated modules (, in ) to keep executor orchestration thin and make cursor lifecycle state explicit and testable.
This commit is contained in:
@@ -53,13 +53,18 @@ to test CLI changes without publishing:
|
||||
|
||||
```bash
|
||||
# mac/linux: kill any existing relay on 19988
|
||||
lsof -ti :19988 | xargs kill
|
||||
PIDS=($(lsof -ti :19988))
|
||||
if [ ${#PIDS[@]} -gt 0 ]; then kill "${PIDS[@]}"; fi
|
||||
# verify port is free (must print nothing)
|
||||
lsof -ti :19988
|
||||
|
||||
# windows (powershell): kill any existing relay on 19988
|
||||
Get-NetTCPConnection -LocalPort 19988 | ForEach-Object { Stop-Process -Id $_.OwningProcess -Force }
|
||||
# verify port is free (must print nothing)
|
||||
Get-NetTCPConnection -LocalPort 19988 -ErrorAction SilentlyContinue
|
||||
|
||||
tsx playwriter/src/cli.ts -s 1 -e "await page.goto('https://example.com')"
|
||||
tsx playwriter/src/cli.ts -s 1 -e "console.log(await accessibilitySnapshot({ page }))"
|
||||
tsx playwriter/src/cli.ts -s 1 -e "console.log(await snapshot({ page }))"
|
||||
tsx playwriter/src/cli.ts session new
|
||||
tsx playwriter/src/cli.ts -s 1 -e "await page.click('button')"
|
||||
```
|
||||
@@ -365,36 +370,35 @@ you can open files when i ask me "open in zed the line where ..." using the comm
|
||||
- if you encounter typescript lint errors for an npm package, read the node_modules/package/\*.d.ts files to understand the typescript types of the package. if you cannot understand them, ask me to help you with it.
|
||||
|
||||
- NEVER silently suppress errors in catch {} blocks if they contain more than one function call
|
||||
|
||||
```ts
|
||||
// BAD. DO NOT DO THIS
|
||||
let favicon: string | undefined
|
||||
let favicon: string | undefined;
|
||||
if (docsConfig?.favicon) {
|
||||
if (typeof docsConfig.favicon === 'string') {
|
||||
favicon = docsConfig.favicon
|
||||
if (typeof docsConfig.favicon === "string") {
|
||||
favicon = docsConfig.favicon;
|
||||
} else if (docsConfig.favicon?.light) {
|
||||
// Use light favicon as default, could be enhanced with theme detection
|
||||
favicon = docsConfig.favicon.light
|
||||
favicon = docsConfig.favicon.light;
|
||||
}
|
||||
}
|
||||
// DO THIS. use an iife. Immediately Invoked Function Expression
|
||||
const favicon: string = (() => {
|
||||
if (!docsConfig?.favicon) {
|
||||
return ''
|
||||
return "";
|
||||
}
|
||||
if (typeof docsConfig.favicon === 'string') {
|
||||
return docsConfig.favicon
|
||||
if (typeof docsConfig.favicon === "string") {
|
||||
return docsConfig.favicon;
|
||||
}
|
||||
if (docsConfig.favicon?.light) {
|
||||
// Use light favicon as default, could be enhanced with theme detection
|
||||
return docsConfig.favicon.light
|
||||
return docsConfig.favicon.light;
|
||||
}
|
||||
return ''
|
||||
})()
|
||||
return "";
|
||||
})();
|
||||
// if you already know the type use it:
|
||||
const favicon: string = () => {
|
||||
// ...
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
- when a package has to import files from another packages in the workspace never add a new tsconfig path, instead add that package as a workspace dependency using `pnpm i "package@workspace:*"`
|
||||
@@ -411,12 +415,12 @@ always specify the type when creating arrays, especially for empty arrays. if yo
|
||||
|
||||
```ts
|
||||
// BAD: Type will be never[]
|
||||
const items = []
|
||||
const items = [];
|
||||
|
||||
// GOOD: Specify the expected type
|
||||
const items: string[] = []
|
||||
const numbers: number[] = []
|
||||
const users: User[] = []
|
||||
const items: string[] = [];
|
||||
const numbers: number[] = [];
|
||||
const users: User[] = [];
|
||||
```
|
||||
|
||||
remember to always add the explicit type to avoid unexpected type inference.
|
||||
@@ -574,7 +578,7 @@ to understand how the code you are writing works, you should add inline snapshot
|
||||
|
||||
- for very long snapshots you should use `toMatchFileSnapshot(filename)` instead of `toMatchInlineSnapshot()`. put the snapshot files in a snapshots/ directory and use the appropriate extension for the file based on the content
|
||||
|
||||
never test client react components. only React and browser independent code.
|
||||
never test client react components. only React and browser independent code.
|
||||
|
||||
most tests should be simple calls to functions with some expect calls, no mocks. test files should be called the same as the file where the tested function is being exported from.
|
||||
|
||||
@@ -591,13 +595,12 @@ sometimes tests work directly on database data, using prisma. to run these tests
|
||||
never write tests yourself that call prisma or interact with database or emails. for these, ask the user to write them for you.
|
||||
|
||||
changelogs.md
|
||||
|
||||
# writing docs
|
||||
|
||||
when generating a .md or .mdx file to document things, always add a frontmatter with title and description. also add a prompt field with the exact prompt used to generate the doc. use @ to reference files and urls and provide any context necessary to be able to recreate this file from scratch using a model. if you used urls also reference them. reference all files you had to read to create the doc. use yaml | syntax to add this prompt and never go over the column width of 80
|
||||
|
||||
# github
|
||||
|
||||
|
||||
you can use the `gh` cli to do operations on github for the current repository. For example: open issues, open PRs, check actions status, read workflow logs, etc.
|
||||
|
||||
## creating issues and pull requests
|
||||
@@ -680,7 +683,6 @@ This will download the source code in ./opensrc. which should be put in .gitigno
|
||||
you can control the browser using the playwright mcp tools. these tools let you control the browser to get information or accomplish actions
|
||||
|
||||
if i ask you to test something in the browser, know that the website dev server is already running at http://localhost:7664 for website and :7777 for docs-website (but docs-website needs to use the website domain specifically, for example name-hash.localhost:7777)
|
||||
|
||||
# zod
|
||||
|
||||
when you need to create a complex type that comes from a prisma table, do not create a new schema that tries to recreate the prisma table structure. instead just use `z.any() as ZodType<PrismaTable>)` to get type safety but leave any in the schema. this gets most of the benefits of zod without having to define a new zod schema that can easily go out of sync.
|
||||
@@ -690,40 +692,17 @@ when you need to create a complex type that comes from a prisma table, do not cr
|
||||
you MUST use the built in zod v4 toJSONSchema and not the npm package `zod-to-json-schema` which is outdated and does not support zod v4.
|
||||
|
||||
```ts
|
||||
import { toJSONSchema } from 'zod'
|
||||
import { toJSONSchema } from "zod";
|
||||
|
||||
const mySchema = z.object({
|
||||
id: z.string().uuid(),
|
||||
name: z.string().min(3).max(100),
|
||||
age: z.number().min(0).optional(),
|
||||
})
|
||||
});
|
||||
|
||||
const jsonSchema = toJSONSchema(mySchema, {
|
||||
removeAdditionalStrategy: 'strict',
|
||||
})
|
||||
removeAdditionalStrategy: "strict",
|
||||
});
|
||||
```
|
||||
|
||||
github.md
|
||||
|
||||
<!-- opensrc:start -->
|
||||
|
||||
## Source Code Reference
|
||||
|
||||
Source code for dependencies is available in `opensrc/` for deeper understanding of implementation details.
|
||||
|
||||
See `opensrc/sources.json` for the list of available packages and their versions.
|
||||
|
||||
Use this source code when you need to understand how a package works internally, not just its types/interface.
|
||||
|
||||
### Fetching Additional Source Code
|
||||
|
||||
To fetch source code for a package or repository you need to understand, run:
|
||||
|
||||
```bash
|
||||
npx opensrc <package> # npm package (e.g., npx opensrc zod)
|
||||
npx opensrc pypi:<package> # Python package (e.g., npx opensrc pypi:requests)
|
||||
npx opensrc crates:<package> # Rust crate (e.g., npx opensrc crates:serde)
|
||||
npx opensrc <owner>/<repo> # GitHub repo (e.g., npx opensrc vercel/ai)
|
||||
```
|
||||
|
||||
<!-- opensrc:end -->
|
||||
|
||||
@@ -1,5 +1,46 @@
|
||||
# Changelog
|
||||
|
||||
## 0.0.75
|
||||
|
||||
### Improvements
|
||||
|
||||
- **Switch minimal cursor to triangular pointer icon**: Updated the `minimal` ghost cursor style to use a stylized triangular SVG pointer (with subtle drop shadow) instead of the circular indicator, while keeping `dot` and `screenstudio` styles available.
|
||||
|
||||
## 0.0.74
|
||||
|
||||
### Improvements
|
||||
|
||||
- **Switch default ghost cursor to a stylized minimal look**: Updated cursor rendering defaults to a cleaner minimal style while preserving `dot` and `screenstudio` options for explicit overrides.
|
||||
|
||||
## 0.0.73
|
||||
|
||||
### Improvements
|
||||
|
||||
- **Simplify recording integration in executor**: Moved ghost-cursor-aware recording wrappers out of `executor.ts` into `screen-recording.ts` via `createRecordingApi(...)`, reducing executor complexity while preserving existing `recording.*` and backward-compatible top-level recording helpers.
|
||||
|
||||
## 0.0.72
|
||||
|
||||
### Improvements
|
||||
|
||||
- **Reduce false "extension disconnected" on relay restarts**: `playwriter session new` now waits longer for extension reconnect and adds a short polling grace window before failing, preventing transient post-restart races from surfacing as hard disconnect errors.
|
||||
|
||||
## 0.0.71
|
||||
|
||||
### Features
|
||||
|
||||
- **Add `recording` and `ghostCursor` namespaces in execute context**: New `recording.start/stop/isRecording/cancel` and `ghostCursor.show/hide` APIs are now exposed for cleaner scripting while keeping `startRecording`, `stopRecording`, `isRecording`, and `cancelRecording` as backward-compatible aliases.
|
||||
- **Manual cursor overlay controls**: Cursor overlay can now be shown/hidden explicitly outside recording flows for screenshot and demo generation.
|
||||
|
||||
## 0.0.70
|
||||
|
||||
### Features
|
||||
|
||||
- **Ghost cursor overlay during recording**: Playwriter now auto-enables a smooth in-page ghost cursor when `startRecording()` is called, driven by `page.onMouseAction` callbacks from the Playwright fork so both `page.mouse.*` and `locator.click()` actions are visualized.
|
||||
|
||||
### Tests
|
||||
|
||||
- **Add ghost-cursor integration coverage**: Extended `on-mouse-action.test.ts` to verify callback-driven cursor animation and teardown in real extension-connected runs.
|
||||
|
||||
## 0.0.69
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "playwriter",
|
||||
"description": "",
|
||||
"version": "0.0.69",
|
||||
"version": "0.0.75",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
|
||||
@@ -47,6 +47,11 @@ const BUNDLES: BundleConfig[] = [
|
||||
type: 'source',
|
||||
entry: 'a11y-client.ts',
|
||||
},
|
||||
{
|
||||
name: 'ghost-cursor-client',
|
||||
type: 'source',
|
||||
entry: 'ghost-cursor-client.ts',
|
||||
},
|
||||
|
||||
// Wrapper bundles (npm packages → globalThis)
|
||||
{
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
/**
|
||||
* Encodes the extracted Screen Studio cursor SVG as a data URL TS module.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const currentDir = path.dirname(fileURLToPath(import.meta.url))
|
||||
const sourcePath = path.join(currentDir, '..', 'src', 'assets', 'cursors', 'screen-studio', 'pointer-macos-tahoe.svg')
|
||||
const outputPath = path.join(currentDir, '..', 'src', 'assets', 'cursors', 'screen-studio', 'pointer-macos-tahoe-data-url.ts')
|
||||
|
||||
function main() {
|
||||
const svg = fs.readFileSync(sourcePath, 'utf-8').trim()
|
||||
const base64 = Buffer.from(svg, 'utf-8').toString('base64')
|
||||
const dataUrl = `data:image/svg+xml;base64,${base64}`
|
||||
|
||||
const output = `/**\n * Generated from pointer-macos-tahoe.svg via scripts/encode-screenstudio-cursor.ts.\n */\n\nexport const SCREENSTUDIO_POINTER_MACOS_TAHOE_DATA_URL = '${dataUrl}'\n`
|
||||
fs.writeFileSync(outputPath, output)
|
||||
console.log(`Wrote ${outputPath}`)
|
||||
}
|
||||
|
||||
main()
|
||||
@@ -0,0 +1,5 @@
|
||||
/**
|
||||
* Generated from pointer-macos-tahoe.svg via scripts/encode-screenstudio-cursor.ts.
|
||||
*/
|
||||
|
||||
export const SCREENSTUDIO_POINTER_MACOS_TAHOE_DATA_URL = 'data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iNjE4IiBoZWlnaHQ9Ijk1OCIgdmlld0JveD0iMCAwIDYxOCA5NTgiIGZpbGw9Im5vbmUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CjxnIGZpbHRlcj0idXJsKCNmaWx0ZXIwX2RfMzg0XzI3KSI+CjxwYXRoIGZpbGwtcnVsZT0iZXZlbm9kZCIgY2xpcC1ydWxlPSJldmVub2RkIiBkPSJNMTI3LjA2MiAzNy4wMzMxTDU0MC42OTYgNDUxLjU1NUM1OTIuNjUzIDUwMy42NiA1NTUuNzk0IDU5Mi41NzQgNDgyLjIyNiA1OTIuNTc0TDQyMS44MzEgNTkyLjU2OUw0ODEuODIxIDczNS4wNTRDNDkyLjMzMSA3NjAuMDIxIDQ5Mi40NzkgNzg3LjY1MiA0ODIuMjY1IDgxMi43NjdDNDcyLjAwMiA4MzcuOTMyIDQ1Mi41NjEgODU3LjU3IDQyNy40OTYgODY4LjA4QzQxNC44NjQgODczLjM1OSA0MDEuNjQgODc2LjAyNCAzODguMTIxIDg3Ni4wMjRDMzQ3LjExNyA4NzYuMDI0IDMxMC4zNTggODUxLjYgMjk0LjQ3IDgxMy44MDRMMjMxLjQyIDY2My45MThMMTkwLjM2OCA3MDAuMzM3QzEzNy4wMjkgNzQ3LjUwOCA1MyA3MDkuNjYzIDUzIDYzOC40MTNWNjcuNjc0NEM1MyAyOC45OTAzIDk5LjcyNjggOS42NDgyOCAxMjcuMDYyIDM3LjAzMzFaIiBmaWxsPSJ3aGl0ZSIvPgo8cGF0aCBmaWxsLXJ1bGU9ImV2ZW5vZGQiIGNsaXAtcnVsZT0iZXZlbm9kZCIgZD0iTTEwMi4zMTYgOTkuNjUyQzEwMi4zMTYgOTMuMTg4MiAxMTAuMTYyIDg5LjkzMTYgMTE0LjcwMSA5NC41MjA0TDUwNC44OTcgNDg1LjU1NUM1MjYuMTY0IDUwNi44NzEgNTExLjA2NSA1NDMuMjM2IDQ4MC45NjcgNTQzLjIzNkwzNDcuNTQ2IDU0My4xNjFMNDM2LjM0MiA3NTQuMTQzQzQ0Ny41NDIgNzgwLjc4OCA0MzUuMDA5IDgxMS40MjkgNDA4LjQxNCA4MjIuNTgxQzM4MS43MiA4MzMuNzgxIDM1MS4xMjggODIxLjI5OCAzMzkuOTc3IDc5NC43MDJMMjUwLjI5MyA1ODEuMzUyTDE1OC41MTcgNjYyLjY0NEMxMzcuOTkxIDY4MC44MDEgMTA2LjMxOSA2NjguMTQ1IDEwMi42NjQgNjQyLjMyM0wxMDIuMzE2IDYzNy4zMzFWOTkuNjUyWiIgZmlsbD0iYmxhY2siLz4KPC9nPgo8ZGVmcz4KPGZpbHRlciBpZD0iZmlsdGVyMF9kXzM4NF8yNyIgeD0iMC4zNCIgeT0iMC43OTkyMTkiIHdpZHRoPSI2MTcuMzIiIGhlaWdodD0iOTU3LjE0NCIgZmlsdGVyVW5pdHM9InVzZXJTcGFjZU9uVXNlIiBjb2xvci1pbnRlcnBvbGF0aW9uLWZpbHRlcnM9InNSR0IiPgo8ZmVGbG9vZCBmbG9vZC1vcGFjaXR5PSIwIiByZXN1bHQ9IkJhY2tncm91bmRJbWFnZUZpeCIvPgo8ZmVDb2xvck1hdHJpeCBpbj0iU291cmNlQWxwaGEiIHR5cGU9Im1hdHJpeCIgdmFsdWVzPSIwIDAgMCAwIDAgMCAwIDAgMCAwIDAgMCAwIDAgMCAwIDAgMCAxMjcgMCIgcmVzdWx0PSJoYXJkQWxwaGEiLz4KPGZlT2Zmc2V0IGR5PSIyOS4yNiIvPgo8ZmVHYXVzc2lhbkJsdXIgc3RkRGV2aWF0aW9uPSIyNi4zMyIvPgo8ZmVDb21wb3NpdGUgaW4yPSJoYXJkQWxwaGEiIG9wZXJhdG9yPSJvdXQiLz4KPGZlQ29sb3JNYXRyaXggdHlwZT0ibWF0cml4IiB2YWx1ZXM9IjAgMCAwIDAgMCAwIDAgMCAwIDAgMCAwIDAgMCAwIDAgMCAwIDAuNjUgMCIvPgo8ZmVCbGVuZCBtb2RlPSJub3JtYWwiIGluMj0iQmFja2dyb3VuZEltYWdlRml4IiByZXN1bHQ9ImVmZmVjdDFfZHJvcFNoYWRvd18zODRfMjciLz4KPGZlQmxlbmQgbW9kZT0ibm9ybWFsIiBpbj0iU291cmNlR3JhcGhpYyIgaW4yPSJlZmZlY3QxX2Ryb3BTaGFkb3dfMzg0XzI3IiByZXN1bHQ9InNoYXBlIi8+CjwvZmlsdGVyPgo8L2RlZnM+Cjwvc3ZnPg=='
|
||||
@@ -0,0 +1,18 @@
|
||||
<svg width="618" height="958" viewBox="0 0 618 958" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<g filter="url(#filter0_d_384_27)">
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M127.062 37.0331L540.696 451.555C592.653 503.66 555.794 592.574 482.226 592.574L421.831 592.569L481.821 735.054C492.331 760.021 492.479 787.652 482.265 812.767C472.002 837.932 452.561 857.57 427.496 868.08C414.864 873.359 401.64 876.024 388.121 876.024C347.117 876.024 310.358 851.6 294.47 813.804L231.42 663.918L190.368 700.337C137.029 747.508 53 709.663 53 638.413V67.6744C53 28.9903 99.7268 9.64828 127.062 37.0331Z" fill="white"/>
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M102.316 99.652C102.316 93.1882 110.162 89.9316 114.701 94.5204L504.897 485.555C526.164 506.871 511.065 543.236 480.967 543.236L347.546 543.161L436.342 754.143C447.542 780.788 435.009 811.429 408.414 822.581C381.72 833.781 351.128 821.298 339.977 794.702L250.293 581.352L158.517 662.644C137.991 680.801 106.319 668.145 102.664 642.323L102.316 637.331V99.652Z" fill="black"/>
|
||||
</g>
|
||||
<defs>
|
||||
<filter id="filter0_d_384_27" x="0.34" y="0.799219" width="617.32" height="957.144" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB">
|
||||
<feFlood flood-opacity="0" result="BackgroundImageFix"/>
|
||||
<feColorMatrix in="SourceAlpha" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0" result="hardAlpha"/>
|
||||
<feOffset dy="29.26"/>
|
||||
<feGaussianBlur stdDeviation="26.33"/>
|
||||
<feComposite in2="hardAlpha" operator="out"/>
|
||||
<feColorMatrix type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.65 0"/>
|
||||
<feBlend mode="normal" in2="BackgroundImageFix" result="effect1_dropShadow_384_27"/>
|
||||
<feBlend mode="normal" in="SourceGraphic" in2="effect1_dropShadow_384_27" result="shape"/>
|
||||
</filter>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.7 KiB |
+28
-16
@@ -5,6 +5,7 @@ import path from 'node:path'
|
||||
import util from 'node:util'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { cac } from '@xmorse/cac'
|
||||
import pc from 'picocolors'
|
||||
|
||||
// Prevent Buffers from dumping hex bytes in util.inspect output.
|
||||
Buffer.prototype[util.inspect.custom] = function () {
|
||||
@@ -15,9 +16,10 @@ import { VERSION, LOG_FILE_PATH, LOG_CDP_FILE_PATH, parseRelayHost } from './uti
|
||||
import {
|
||||
ensureRelayServer,
|
||||
RELAY_PORT,
|
||||
waitForExtension,
|
||||
waitForConnectedExtensions,
|
||||
getExtensionOutdatedWarning,
|
||||
getExtensionStatus,
|
||||
type ExtensionStatus,
|
||||
} from './relay-client.js'
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url))
|
||||
@@ -26,15 +28,6 @@ const cliRelayEnv = { PLAYWRITER_AUTO_ENABLE: '1' }
|
||||
|
||||
const cli = cac('playwriter')
|
||||
|
||||
type ExtensionStatus = {
|
||||
extensionId: string
|
||||
stableKey?: string
|
||||
browser: string | null
|
||||
profile: { email: string; id: string } | null
|
||||
activeTargets: number
|
||||
playwriterVersion?: string | null
|
||||
}
|
||||
|
||||
cli
|
||||
.command('', 'Start the MCP server or controls the browser with -e')
|
||||
.option('--host <host>', 'Remote relay server host to connect to (or use PLAYWRITER_HOST env var)')
|
||||
@@ -136,8 +129,12 @@ async function executeCode(options: {
|
||||
if (!host && !process.env.PLAYWRITER_HOST) {
|
||||
const restarted = await ensureRelayServer({ logger: console, env: cliRelayEnv })
|
||||
if (restarted) {
|
||||
const connected = await waitForExtension({ logger: console, timeoutMs: 10000 })
|
||||
if (!connected) {
|
||||
const connectedExtensions = await waitForConnectedExtensions({
|
||||
logger: console,
|
||||
timeoutMs: 10000,
|
||||
pollIntervalMs: 250,
|
||||
})
|
||||
if (connectedExtensions.length === 0) {
|
||||
console.error('Warning: Extension not connected. Commands may fail.')
|
||||
}
|
||||
}
|
||||
@@ -212,15 +209,30 @@ cli
|
||||
.option('--host <host>', 'Remote relay server host')
|
||||
.option('--browser <stableKey>', 'Stable browser key when multiple browsers are connected')
|
||||
.action(async (options: { host?: string; browser?: string }) => {
|
||||
if (!options.host && !process.env.PLAYWRITER_HOST) {
|
||||
const isLocal = !options.host && !process.env.PLAYWRITER_HOST
|
||||
|
||||
let extensions: ExtensionStatus[] = []
|
||||
|
||||
if (isLocal) {
|
||||
await ensureRelayServer({ logger: console, env: cliRelayEnv })
|
||||
await waitForExtension({
|
||||
timeoutMs: 3000,
|
||||
extensions = await waitForConnectedExtensions({
|
||||
timeoutMs: 12000,
|
||||
pollIntervalMs: 250,
|
||||
logger: console,
|
||||
})
|
||||
|
||||
if (extensions.length === 0) {
|
||||
console.log(pc.dim('Waiting briefly for extension to reconnect...'))
|
||||
extensions = await waitForConnectedExtensions({
|
||||
timeoutMs: 10000,
|
||||
pollIntervalMs: 250,
|
||||
logger: console,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
extensions = await fetchExtensionsStatus(options.host)
|
||||
}
|
||||
|
||||
const extensions = await fetchExtensionsStatus(options.host)
|
||||
if (extensions.length === 0) {
|
||||
console.error('No connected browsers detected. Click the Playwriter extension icon.')
|
||||
process.exit(1)
|
||||
|
||||
+60
-30
@@ -33,8 +33,10 @@ import { createGhostBrowserChrome, type GhostBrowserCommandResult } from './ghos
|
||||
export type { SnapshotFormat }
|
||||
import { getCleanHTML, type GetCleanHTMLOptions } from './clean-html.js'
|
||||
import { getPageMarkdown, type GetPageMarkdownOptions } from './page-markdown.js'
|
||||
import { startRecording, stopRecording, isRecording, cancelRecording } from './screen-recording.js'
|
||||
import { createRecordingApi } from './screen-recording.js'
|
||||
import { createDemoVideo } from './ffmpeg.js'
|
||||
import { type GhostCursorClientOptions } from './ghost-cursor.js'
|
||||
import { RecordingGhostCursorController } from './recording-ghost-cursor.js'
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url)
|
||||
const __dirname = path.dirname(__filename)
|
||||
@@ -948,18 +950,51 @@ export class PlaywrightExecutor {
|
||||
// Recording uses chrome.tabCapture which requires activeTab permission.
|
||||
// This permission is granted when the user clicks the Playwriter extension icon on a tab.
|
||||
const relayPort = this.cdpConfig.port || 19988
|
||||
// Recording will work on any tab where the user has clicked the icon.
|
||||
const withRecordingDefaults = <T extends { page?: Page; sessionId?: string }, R>(
|
||||
fn: (opts: T & { relayPort: number; sessionId?: string }) => Promise<R>,
|
||||
) => {
|
||||
return async (options: T = {} as T) => {
|
||||
const targetPage = options.page || page
|
||||
// Use Playwright's exposed tab session ID directly.
|
||||
const sessionId = options.sessionId || targetPage.sessionId() || undefined
|
||||
return fn({ page: targetPage, sessionId, relayPort, ...options })
|
||||
}
|
||||
}
|
||||
const self = this
|
||||
const recordingGhostCursor = new RecordingGhostCursorController({
|
||||
logger: {
|
||||
error: (...args: unknown[]) => {
|
||||
self.logger.error(...args)
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
const showGhostCursor = async (options?: ({ page?: Page } & GhostCursorClientOptions)) => {
|
||||
const targetPage = options?.page || page
|
||||
const cursorOptions: GhostCursorClientOptions | undefined = (() => {
|
||||
if (!options) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
const { page: _ignoredPage, ...rest } = options
|
||||
return rest
|
||||
})()
|
||||
|
||||
await recordingGhostCursor.show({ page: targetPage, cursorOptions })
|
||||
}
|
||||
|
||||
const hideGhostCursor = async (options?: { page?: Page }) => {
|
||||
const targetPage = options?.page || page
|
||||
await recordingGhostCursor.hide({ page: targetPage })
|
||||
}
|
||||
|
||||
const recordingApi = createRecordingApi({
|
||||
context,
|
||||
defaultPage: page,
|
||||
relayPort,
|
||||
ghostCursorController: recordingGhostCursor,
|
||||
onStart: () => {
|
||||
self.recordingStartedAt = Date.now()
|
||||
self.executionTimestamps = []
|
||||
},
|
||||
onFinish: () => {
|
||||
self.recordingStartedAt = null
|
||||
self.executionTimestamps = []
|
||||
},
|
||||
getExecutionTimestamps: () => {
|
||||
return self.executionTimestamps
|
||||
},
|
||||
})
|
||||
|
||||
// Ghost Browser API - creates chrome object that mirrors Ghost Browser's APIs
|
||||
// See extension/src/ghost-browser-api.d.ts for full API documentation
|
||||
@@ -994,26 +1029,21 @@ export class PlaywrightExecutor {
|
||||
formatStylesAsText,
|
||||
getReactSource: getReactSourceFn,
|
||||
screenshotWithAccessibilityLabels: screenshotWithAccessibilityLabelsFn,
|
||||
startRecording: async (opts?: Parameters<typeof startRecording>[0]) => {
|
||||
const result = await withRecordingDefaults(startRecording)(opts)
|
||||
self.recordingStartedAt = Date.now()
|
||||
self.executionTimestamps = []
|
||||
return result
|
||||
ghostCursor: {
|
||||
show: showGhostCursor,
|
||||
hide: hideGhostCursor,
|
||||
},
|
||||
stopRecording: async (opts?: Parameters<typeof stopRecording>[0]) => {
|
||||
const result = await withRecordingDefaults(stopRecording)(opts)
|
||||
const executionTimestamps = [...self.executionTimestamps]
|
||||
self.recordingStartedAt = null
|
||||
self.executionTimestamps = []
|
||||
return { ...result, executionTimestamps }
|
||||
},
|
||||
isRecording: withRecordingDefaults(isRecording),
|
||||
cancelRecording: async (opts?: Parameters<typeof cancelRecording>[0]) => {
|
||||
const result = await withRecordingDefaults(cancelRecording)(opts)
|
||||
self.recordingStartedAt = null
|
||||
self.executionTimestamps = []
|
||||
return result
|
||||
recording: {
|
||||
start: recordingApi.start,
|
||||
stop: recordingApi.stop,
|
||||
isRecording: recordingApi.isRecording,
|
||||
cancel: recordingApi.cancel,
|
||||
},
|
||||
// Backward-compatible aliases
|
||||
startRecording: recordingApi.start,
|
||||
stopRecording: recordingApi.stop,
|
||||
isRecording: recordingApi.isRecording,
|
||||
cancelRecording: recordingApi.cancel,
|
||||
createDemoVideo,
|
||||
resetPlaywright: async () => {
|
||||
const { page: newPage, context: newContext } = await self.reset()
|
||||
|
||||
@@ -0,0 +1,367 @@
|
||||
/**
|
||||
* Browser-side ghost cursor renderer.
|
||||
* Injected into the page to visualize automated mouse actions with smooth easing.
|
||||
*/
|
||||
|
||||
import { SCREENSTUDIO_POINTER_MACOS_TAHOE_DATA_URL } from './assets/cursors/screen-studio/pointer-macos-tahoe-data-url.js'
|
||||
|
||||
type GhostCursorActionType = 'move' | 'down' | 'up' | 'wheel'
|
||||
type GhostCursorButton = 'left' | 'right' | 'middle' | 'none'
|
||||
type GhostCursorStyle = 'minimal' | 'dot' | 'screenstudio'
|
||||
|
||||
interface GhostCursorAction {
|
||||
type: GhostCursorActionType
|
||||
x: number
|
||||
y: number
|
||||
button: GhostCursorButton
|
||||
}
|
||||
|
||||
export interface GhostCursorClientOptions {
|
||||
style?: GhostCursorStyle
|
||||
color?: string
|
||||
size?: number
|
||||
zIndex?: number
|
||||
easing?: string
|
||||
minDurationMs?: number
|
||||
maxDurationMs?: number
|
||||
speedPxPerMs?: number
|
||||
}
|
||||
|
||||
interface GhostCursorRuntimeOptions {
|
||||
style: GhostCursorStyle
|
||||
color: string
|
||||
size: number
|
||||
zIndex: number
|
||||
easing: string
|
||||
minDurationMs: number
|
||||
maxDurationMs: number
|
||||
speedPxPerMs: number
|
||||
}
|
||||
|
||||
interface GhostCursorRuntimeState {
|
||||
cursorElement: ReturnType<typeof createCursorElement> | null
|
||||
options: GhostCursorRuntimeOptions
|
||||
x: number
|
||||
y: number
|
||||
scale: number
|
||||
hasPosition: boolean
|
||||
enabled: boolean
|
||||
}
|
||||
|
||||
interface GhostCursorApi {
|
||||
enable: (options?: GhostCursorClientOptions) => void
|
||||
disable: () => void
|
||||
applyMouseAction: (action: GhostCursorAction) => void
|
||||
isEnabled: () => boolean
|
||||
}
|
||||
|
||||
declare global {
|
||||
var __playwriterGhostCursor: GhostCursorApi | undefined
|
||||
}
|
||||
|
||||
const CURSOR_ID = '__playwriter_ghost_cursor__'
|
||||
const SCREENSTUDIO_POINTER_ASPECT_RATIO = 618 / 958
|
||||
const SCREENSTUDIO_HOTSPOT_X_RATIO = 0.14
|
||||
const SCREENSTUDIO_HOTSPOT_Y_RATIO = 0.06
|
||||
const MINIMAL_TRIANGLE_HOTSPOT_X_RATIO = 0.07
|
||||
const MINIMAL_TRIANGLE_HOTSPOT_Y_RATIO = 0.06
|
||||
|
||||
const DEFAULT_OPTIONS: GhostCursorRuntimeOptions = {
|
||||
style: 'minimal',
|
||||
color: '#111827',
|
||||
size: 22,
|
||||
zIndex: 2147483647,
|
||||
easing: 'cubic-bezier(0.22, 1, 0.36, 1)',
|
||||
minDurationMs: 40,
|
||||
maxDurationMs: 450,
|
||||
speedPxPerMs: 2.2,
|
||||
}
|
||||
|
||||
const runtime: GhostCursorRuntimeState = {
|
||||
cursorElement: null,
|
||||
options: DEFAULT_OPTIONS,
|
||||
x: 0,
|
||||
y: 0,
|
||||
scale: 1,
|
||||
hasPosition: false,
|
||||
enabled: false,
|
||||
}
|
||||
|
||||
function clamp(options: { value: number; min: number; max: number }): number {
|
||||
const { value, min, max } = options
|
||||
return Math.min(max, Math.max(min, value))
|
||||
}
|
||||
|
||||
function mergeOptions(options?: GhostCursorClientOptions): GhostCursorRuntimeOptions {
|
||||
if (!options) {
|
||||
return DEFAULT_OPTIONS
|
||||
}
|
||||
|
||||
return {
|
||||
style: options.style ?? DEFAULT_OPTIONS.style,
|
||||
color: options.color ?? DEFAULT_OPTIONS.color,
|
||||
size: options.size ?? DEFAULT_OPTIONS.size,
|
||||
zIndex: options.zIndex ?? DEFAULT_OPTIONS.zIndex,
|
||||
easing: options.easing ?? DEFAULT_OPTIONS.easing,
|
||||
minDurationMs: options.minDurationMs ?? DEFAULT_OPTIONS.minDurationMs,
|
||||
maxDurationMs: options.maxDurationMs ?? DEFAULT_OPTIONS.maxDurationMs,
|
||||
speedPxPerMs: options.speedPxPerMs ?? DEFAULT_OPTIONS.speedPxPerMs,
|
||||
}
|
||||
}
|
||||
|
||||
function getCursorDimensions(): { width: number; height: number } {
|
||||
if (runtime.options.style === 'screenstudio') {
|
||||
const height = runtime.options.size
|
||||
const width = Math.max(10, Math.round(height * SCREENSTUDIO_POINTER_ASPECT_RATIO))
|
||||
return { width, height }
|
||||
}
|
||||
|
||||
if (runtime.options.style === 'minimal') {
|
||||
const size = Math.max(12, runtime.options.size)
|
||||
return { width: size, height: size }
|
||||
}
|
||||
|
||||
return { width: runtime.options.size, height: runtime.options.size }
|
||||
}
|
||||
|
||||
function getHotspotOffsetPx(): { x: number; y: number } {
|
||||
const dimensions = getCursorDimensions()
|
||||
|
||||
if (runtime.options.style === 'screenstudio') {
|
||||
return {
|
||||
x: Math.round(dimensions.width * SCREENSTUDIO_HOTSPOT_X_RATIO),
|
||||
y: Math.round(dimensions.height * SCREENSTUDIO_HOTSPOT_Y_RATIO),
|
||||
}
|
||||
}
|
||||
|
||||
if (runtime.options.style === 'minimal') {
|
||||
return {
|
||||
x: Math.round(dimensions.width * MINIMAL_TRIANGLE_HOTSPOT_X_RATIO),
|
||||
y: Math.round(dimensions.height * MINIMAL_TRIANGLE_HOTSPOT_Y_RATIO),
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
x: Math.round(dimensions.width / 2),
|
||||
y: Math.round(dimensions.height / 2),
|
||||
}
|
||||
}
|
||||
|
||||
function getBaseOpacity(): string {
|
||||
if (runtime.options.style === 'screenstudio') {
|
||||
return '0.95'
|
||||
}
|
||||
|
||||
if (runtime.options.style === 'minimal') {
|
||||
return '1'
|
||||
}
|
||||
|
||||
return '0.72'
|
||||
}
|
||||
|
||||
function applyTransform(): void {
|
||||
if (!runtime.cursorElement) {
|
||||
return
|
||||
}
|
||||
|
||||
const hotspot = getHotspotOffsetPx()
|
||||
runtime.cursorElement.style.transform = `translate3d(${runtime.x - hotspot.x}px, ${runtime.y - hotspot.y}px, 0) scale(${runtime.scale})`
|
||||
}
|
||||
|
||||
function computeDurationMs(options: { targetX: number; targetY: number }): number {
|
||||
if (!runtime.hasPosition) {
|
||||
return 0
|
||||
}
|
||||
|
||||
const dx = options.targetX - runtime.x
|
||||
const dy = options.targetY - runtime.y
|
||||
const distance = Math.hypot(dx, dy)
|
||||
const rawDurationMs = distance / runtime.options.speedPxPerMs
|
||||
|
||||
return clamp({
|
||||
value: rawDurationMs,
|
||||
min: runtime.options.minDurationMs,
|
||||
max: runtime.options.maxDurationMs,
|
||||
})
|
||||
}
|
||||
|
||||
function createCursorElement() {
|
||||
const element = document.createElement('div')
|
||||
element.id = CURSOR_ID
|
||||
element.setAttribute('aria-hidden', 'true')
|
||||
element.style.position = 'fixed'
|
||||
element.style.left = '0'
|
||||
element.style.top = '0'
|
||||
element.style.pointerEvents = 'none'
|
||||
element.style.zIndex = `${runtime.options.zIndex}`
|
||||
element.style.opacity = getBaseOpacity()
|
||||
element.style.transitionProperty = 'transform, opacity'
|
||||
element.style.transitionTimingFunction = runtime.options.easing
|
||||
element.style.transitionDuration = '0ms'
|
||||
element.style.willChange = 'transform'
|
||||
|
||||
runtime.cursorElement = element
|
||||
applyRuntimeVisualOptions()
|
||||
|
||||
return element
|
||||
}
|
||||
|
||||
function ensureCursorElement() {
|
||||
const existing = document.getElementById(CURSOR_ID)
|
||||
if (existing) {
|
||||
runtime.cursorElement = existing
|
||||
return existing
|
||||
}
|
||||
|
||||
const element = createCursorElement()
|
||||
runtime.cursorElement = element
|
||||
const root = document.documentElement || document.body
|
||||
root.appendChild(element)
|
||||
return element
|
||||
}
|
||||
|
||||
function applyRuntimeVisualOptions(): void {
|
||||
if (!runtime.cursorElement) {
|
||||
return
|
||||
}
|
||||
|
||||
const dimensions = getCursorDimensions()
|
||||
runtime.cursorElement.style.width = `${dimensions.width}px`
|
||||
runtime.cursorElement.style.height = `${dimensions.height}px`
|
||||
runtime.cursorElement.style.zIndex = `${runtime.options.zIndex}`
|
||||
runtime.cursorElement.style.transitionTimingFunction = runtime.options.easing
|
||||
|
||||
if (runtime.options.style === 'screenstudio') {
|
||||
runtime.cursorElement.style.borderRadius = '0'
|
||||
runtime.cursorElement.style.border = 'none'
|
||||
runtime.cursorElement.style.backgroundColor = 'transparent'
|
||||
runtime.cursorElement.style.backgroundImage = `url("${SCREENSTUDIO_POINTER_MACOS_TAHOE_DATA_URL}")`
|
||||
runtime.cursorElement.style.backgroundRepeat = 'no-repeat'
|
||||
runtime.cursorElement.style.backgroundPosition = 'left top'
|
||||
runtime.cursorElement.style.backgroundSize = 'contain'
|
||||
runtime.cursorElement.style.backdropFilter = 'none'
|
||||
runtime.cursorElement.style.filter = 'none'
|
||||
runtime.cursorElement.style.boxShadow = 'none'
|
||||
runtime.cursorElement.style.opacity = getBaseOpacity()
|
||||
return
|
||||
}
|
||||
|
||||
if (runtime.options.style === 'minimal') {
|
||||
const triangleSvg = `<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 24 24"><path fill="${runtime.options.color}" d="m23.284 19.124l-6.866-6.895a.4.4 0 0 1-.118-.296a.43.43 0 0 1 .163-.282l4.439-3.077a1.48 1.48 0 0 0 .621-1.48a1.48 1.48 0 0 0-1.036-1.198L1.623.302a1.14 1.14 0 0 0-1.11.282A1.13 1.13 0 0 0 .29 1.649L5.928 20.44a1.48 1.48 0 0 0 1.183 1.035a1.48 1.48 0 0 0 1.48-.621l3.078-4.44a.37.37 0 0 1 .31-.118a.43.43 0 0 1 .296.104l6.91 6.91a1.48 1.48 0 0 0 2.087 0l2.086-2.086a1.48 1.48 0 0 0-.074-2.101"/></svg>`
|
||||
const triangleDataUrl = `url("data:image/svg+xml,${encodeURIComponent(triangleSvg)}")`
|
||||
runtime.cursorElement.style.borderRadius = '0'
|
||||
runtime.cursorElement.style.border = 'none'
|
||||
runtime.cursorElement.style.backgroundColor = 'transparent'
|
||||
runtime.cursorElement.style.backgroundImage = triangleDataUrl
|
||||
runtime.cursorElement.style.backgroundRepeat = 'no-repeat'
|
||||
runtime.cursorElement.style.backgroundSize = 'contain'
|
||||
runtime.cursorElement.style.backgroundPosition = 'left top'
|
||||
runtime.cursorElement.style.backdropFilter = 'none'
|
||||
runtime.cursorElement.style.boxShadow = 'none'
|
||||
runtime.cursorElement.style.filter = 'drop-shadow(0 1px 1px rgba(0, 0, 0, 0.3))'
|
||||
runtime.cursorElement.style.opacity = getBaseOpacity()
|
||||
return
|
||||
}
|
||||
|
||||
runtime.cursorElement.style.borderRadius = '999px'
|
||||
runtime.cursorElement.style.border = 'none'
|
||||
runtime.cursorElement.style.backgroundColor = runtime.options.color
|
||||
runtime.cursorElement.style.backgroundImage = 'none'
|
||||
runtime.cursorElement.style.backdropFilter = 'none'
|
||||
runtime.cursorElement.style.filter = 'none'
|
||||
runtime.cursorElement.style.boxShadow = '0 2px 10px rgba(0, 0, 0, 0.18), inset 0 0 0 2px rgba(255, 255, 255, 0.55)'
|
||||
runtime.cursorElement.style.opacity = getBaseOpacity()
|
||||
}
|
||||
|
||||
function moveCursor(options: { x: number; y: number }): void {
|
||||
if (!runtime.enabled) {
|
||||
return
|
||||
}
|
||||
|
||||
const element = ensureCursorElement()
|
||||
const durationMs = computeDurationMs({ targetX: options.x, targetY: options.y })
|
||||
element.style.transitionDuration = `${Math.round(durationMs)}ms`
|
||||
|
||||
runtime.x = options.x
|
||||
runtime.y = options.y
|
||||
runtime.hasPosition = true
|
||||
applyTransform()
|
||||
}
|
||||
|
||||
function setPressed(options: { pressed: boolean }): void {
|
||||
if (!runtime.enabled) {
|
||||
return
|
||||
}
|
||||
|
||||
const element = ensureCursorElement()
|
||||
runtime.scale = options.pressed
|
||||
? runtime.options.style === 'screenstudio'
|
||||
? 0.94
|
||||
: runtime.options.style === 'minimal'
|
||||
? 0.93
|
||||
: 0.82
|
||||
: 1
|
||||
element.style.opacity = options.pressed ? '1' : getBaseOpacity()
|
||||
applyTransform()
|
||||
}
|
||||
|
||||
function enable(options?: GhostCursorClientOptions): void {
|
||||
runtime.options = mergeOptions(options)
|
||||
runtime.enabled = true
|
||||
ensureCursorElement()
|
||||
applyRuntimeVisualOptions()
|
||||
|
||||
if (!runtime.hasPosition) {
|
||||
runtime.x = Math.round(window.innerWidth / 2)
|
||||
runtime.y = Math.round(window.innerHeight / 2)
|
||||
runtime.scale = 1
|
||||
runtime.hasPosition = true
|
||||
applyTransform()
|
||||
}
|
||||
}
|
||||
|
||||
function disable(): void {
|
||||
runtime.enabled = false
|
||||
runtime.scale = 1
|
||||
runtime.hasPosition = false
|
||||
|
||||
if (runtime.cursorElement) {
|
||||
runtime.cursorElement.remove()
|
||||
runtime.cursorElement = null
|
||||
}
|
||||
}
|
||||
|
||||
function applyMouseAction(action: GhostCursorAction): void {
|
||||
if (!runtime.enabled) {
|
||||
return
|
||||
}
|
||||
|
||||
if (action.type === 'move' || action.type === 'wheel') {
|
||||
moveCursor({ x: action.x, y: action.y })
|
||||
return
|
||||
}
|
||||
|
||||
if (action.type === 'down') {
|
||||
moveCursor({ x: action.x, y: action.y })
|
||||
setPressed({ pressed: true })
|
||||
return
|
||||
}
|
||||
|
||||
if (action.type === 'up') {
|
||||
moveCursor({ x: action.x, y: action.y })
|
||||
setPressed({ pressed: false })
|
||||
}
|
||||
}
|
||||
|
||||
const api: GhostCursorApi = {
|
||||
enable,
|
||||
disable,
|
||||
applyMouseAction,
|
||||
isEnabled: () => {
|
||||
return runtime.enabled
|
||||
},
|
||||
}
|
||||
|
||||
globalThis.__playwriterGhostCursor = api
|
||||
|
||||
export {}
|
||||
@@ -0,0 +1,110 @@
|
||||
/**
|
||||
* Node-side ghost cursor helpers.
|
||||
* Injects the browser bundle and forwards mouse action events to the page overlay.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import type { Page, MouseActionEvent } from '@xmorse/playwright-core'
|
||||
|
||||
export interface GhostCursorClientOptions {
|
||||
style?: 'minimal' | 'dot' | 'screenstudio'
|
||||
color?: string
|
||||
size?: number
|
||||
zIndex?: number
|
||||
easing?: string
|
||||
minDurationMs?: number
|
||||
maxDurationMs?: number
|
||||
speedPxPerMs?: number
|
||||
}
|
||||
|
||||
interface GhostCursorBrowserApi {
|
||||
enable: (options?: GhostCursorClientOptions) => void
|
||||
disable: () => void
|
||||
applyMouseAction: (event: MouseActionEvent) => void
|
||||
}
|
||||
|
||||
let ghostCursorCode: string | null = null
|
||||
|
||||
function getGhostCursorCode(): string {
|
||||
if (ghostCursorCode) {
|
||||
return ghostCursorCode
|
||||
}
|
||||
|
||||
const currentDir = path.dirname(fileURLToPath(import.meta.url))
|
||||
const bundlePath = path.join(currentDir, '..', 'dist', 'ghost-cursor-client.js')
|
||||
ghostCursorCode = fs.readFileSync(bundlePath, 'utf-8')
|
||||
return ghostCursorCode
|
||||
}
|
||||
|
||||
async function ensureGhostCursorInjected(options: { page: Page }): Promise<void> {
|
||||
const { page } = options
|
||||
const hasGhostCursor = await page.evaluate(() => {
|
||||
return Boolean((globalThis as { __playwriterGhostCursor?: unknown }).__playwriterGhostCursor)
|
||||
})
|
||||
|
||||
if (hasGhostCursor) {
|
||||
return
|
||||
}
|
||||
|
||||
const code = getGhostCursorCode()
|
||||
await page.evaluate(code)
|
||||
}
|
||||
|
||||
export async function enableGhostCursor(options: {
|
||||
page: Page
|
||||
cursorOptions?: GhostCursorClientOptions
|
||||
}): Promise<void> {
|
||||
const { page, cursorOptions } = options
|
||||
await ensureGhostCursorInjected({ page })
|
||||
|
||||
await page.evaluate(
|
||||
({ optionsFromNode }) => {
|
||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
||||
api?.enable(optionsFromNode)
|
||||
},
|
||||
{ optionsFromNode: cursorOptions },
|
||||
)
|
||||
}
|
||||
|
||||
export async function disableGhostCursor(options: { page: Page }): Promise<void> {
|
||||
const { page } = options
|
||||
await page.evaluate(() => {
|
||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
||||
api?.disable()
|
||||
})
|
||||
}
|
||||
|
||||
export async function applyGhostCursorMouseAction(options: {
|
||||
page: Page
|
||||
event: MouseActionEvent
|
||||
}): Promise<void> {
|
||||
const { page, event } = options
|
||||
|
||||
const applied = await page.evaluate(
|
||||
({ serializedEvent }) => {
|
||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
||||
if (!api) {
|
||||
return false
|
||||
}
|
||||
|
||||
api.applyMouseAction(serializedEvent)
|
||||
return true
|
||||
},
|
||||
{ serializedEvent: event },
|
||||
)
|
||||
|
||||
if (applied) {
|
||||
return
|
||||
}
|
||||
|
||||
await ensureGhostCursorInjected({ page })
|
||||
await page.evaluate(
|
||||
({ serializedEvent }) => {
|
||||
const api = (globalThis as { __playwriterGhostCursor?: GhostCursorBrowserApi }).__playwriterGhostCursor
|
||||
api?.applyMouseAction(serializedEvent)
|
||||
},
|
||||
{ serializedEvent: event },
|
||||
)
|
||||
}
|
||||
@@ -4,9 +4,10 @@
|
||||
* and locator-initiated actions like page.locator().click().
|
||||
*/
|
||||
import { chromium } from '@xmorse/playwright-core'
|
||||
import type { MouseActionEvent } from '@xmorse/playwright-core'
|
||||
import type { MouseActionEvent, Page } from '@xmorse/playwright-core'
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest'
|
||||
import { getCdpUrl } from './utils.js'
|
||||
import { enableGhostCursor, applyGhostCursorMouseAction, disableGhostCursor } from './ghost-cursor.js'
|
||||
import {
|
||||
setupTestContext,
|
||||
cleanupTestContext,
|
||||
@@ -113,6 +114,62 @@ describe('onMouseAction callback', () => {
|
||||
await safeCloseCDPBrowser(directBrowser)
|
||||
}, 30000)
|
||||
|
||||
it('should animate ghost cursor from onMouseAction callback', async () => {
|
||||
const browserContext = testCtx!.browserContext
|
||||
|
||||
const directBrowser = await chromium.connectOverCDP(getCdpUrl({ port: TEST_PORT }))
|
||||
let targetPage: Page | null = null
|
||||
try {
|
||||
const contexts = directBrowser.contexts()
|
||||
const pages = contexts[0].pages()
|
||||
targetPage = pages.find((p) => p.url().startsWith('data:'))!
|
||||
expect(targetPage).toBeDefined()
|
||||
const pageForTest = targetPage
|
||||
|
||||
await enableGhostCursor({ page: pageForTest })
|
||||
pageForTest.onMouseAction = async (event) => {
|
||||
await applyGhostCursorMouseAction({ page: pageForTest, event })
|
||||
}
|
||||
|
||||
await pageForTest.mouse.click(140, 120)
|
||||
|
||||
const cursorState = await pageForTest.evaluate(() => {
|
||||
const cursorElement = document.getElementById('__playwriter_ghost_cursor__')
|
||||
if (!cursorElement) {
|
||||
return { exists: false, transform: '' }
|
||||
}
|
||||
return {
|
||||
exists: true,
|
||||
transform: cursorElement.getAttribute('style') || '',
|
||||
}
|
||||
})
|
||||
|
||||
expect(cursorState.exists).toBe(true)
|
||||
const translateMatch = cursorState.transform.match(/translate3d\(([-\d.]+)px, ([-\d.]+)px, 0px\)/)
|
||||
expect(translateMatch).toBeTruthy()
|
||||
const translateX = Number(translateMatch![1])
|
||||
const translateY = Number(translateMatch![2])
|
||||
// Screen Studio cursor uses a hotspot offset, so CSS position is slightly above/left of click target.
|
||||
expect(translateX).toBeLessThanOrEqual(140)
|
||||
expect(translateY).toBeLessThanOrEqual(120)
|
||||
expect(translateX).toBeGreaterThan(120)
|
||||
expect(translateY).toBeGreaterThan(100)
|
||||
|
||||
await disableGhostCursor({ page: pageForTest })
|
||||
|
||||
const hasGhostCursor = await pageForTest.evaluate(() => {
|
||||
return Boolean(document.getElementById('__playwriter_ghost_cursor__'))
|
||||
})
|
||||
expect(hasGhostCursor).toBe(false)
|
||||
} finally {
|
||||
if (targetPage) {
|
||||
targetPage.onMouseAction = null
|
||||
await disableGhostCursor({ page: targetPage })
|
||||
}
|
||||
await safeCloseCDPBrowser(directBrowser)
|
||||
}
|
||||
}, 30000)
|
||||
|
||||
it('should not fire when onMouseAction is set to null', async () => {
|
||||
const browserContext = testCtx!.browserContext
|
||||
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* Encapsulates ghost cursor lifecycle for recording sessions.
|
||||
* Keeps onMouseAction chaining/restoration isolated from executor logic.
|
||||
*/
|
||||
|
||||
import type { BrowserContext, Page } from '@xmorse/playwright-core'
|
||||
import { applyGhostCursorMouseAction, disableGhostCursor, enableGhostCursor, type GhostCursorClientOptions } from './ghost-cursor.js'
|
||||
|
||||
interface RecordingGhostCursorLogger {
|
||||
error: (...args: unknown[]) => void
|
||||
}
|
||||
|
||||
interface RecordingTargetOptions {
|
||||
page?: Page
|
||||
sessionId?: string
|
||||
}
|
||||
|
||||
export class RecordingGhostCursorController {
|
||||
private readonly previousMouseActionByPage = new WeakMap<Page, Page['onMouseAction']>()
|
||||
private readonly logger: RecordingGhostCursorLogger
|
||||
|
||||
constructor(options: { logger: RecordingGhostCursorLogger }) {
|
||||
this.logger = options.logger
|
||||
}
|
||||
|
||||
resolveRecordingTargetPage(options: {
|
||||
context: BrowserContext
|
||||
defaultPage: Page
|
||||
target?: RecordingTargetOptions
|
||||
}): Page {
|
||||
const { context, defaultPage, target } = options
|
||||
|
||||
if (target?.page) {
|
||||
return target.page
|
||||
}
|
||||
|
||||
if (target?.sessionId) {
|
||||
const pageForSession = context.pages().find((candidatePage) => {
|
||||
return candidatePage.sessionId() === target.sessionId
|
||||
})
|
||||
|
||||
if (pageForSession) {
|
||||
return pageForSession
|
||||
}
|
||||
}
|
||||
|
||||
return defaultPage
|
||||
}
|
||||
|
||||
async enableForRecording(options: { page: Page }): Promise<void> {
|
||||
const { page } = options
|
||||
|
||||
try {
|
||||
await enableGhostCursor({ page })
|
||||
if (!this.previousMouseActionByPage.has(page)) {
|
||||
this.previousMouseActionByPage.set(page, page.onMouseAction)
|
||||
}
|
||||
|
||||
const previousMouseAction = this.previousMouseActionByPage.get(page)
|
||||
page.onMouseAction = async (event) => {
|
||||
void applyGhostCursorMouseAction({ page, event }).catch((error) => {
|
||||
this.logger.error('[playwriter] Failed to apply ghost cursor action', error)
|
||||
})
|
||||
|
||||
if (!previousMouseAction) {
|
||||
return
|
||||
}
|
||||
|
||||
await previousMouseAction(event)
|
||||
}
|
||||
} catch (error) {
|
||||
page.onMouseAction = this.previousMouseActionByPage.get(page) ?? null
|
||||
this.previousMouseActionByPage.delete(page)
|
||||
this.logger.error('[playwriter] Failed to enable ghost cursor', error)
|
||||
}
|
||||
}
|
||||
|
||||
async disableForRecording(options: { page: Page }): Promise<void> {
|
||||
const { page } = options
|
||||
page.onMouseAction = this.previousMouseActionByPage.get(page) ?? null
|
||||
this.previousMouseActionByPage.delete(page)
|
||||
|
||||
try {
|
||||
await disableGhostCursor({ page })
|
||||
} catch (error) {
|
||||
this.logger.error('[playwriter] Failed to disable ghost cursor', error)
|
||||
}
|
||||
}
|
||||
|
||||
async show(options: { page: Page; cursorOptions?: GhostCursorClientOptions }): Promise<void> {
|
||||
const { page, cursorOptions } = options
|
||||
await enableGhostCursor({ page, cursorOptions })
|
||||
}
|
||||
|
||||
async hide(options: { page: Page }): Promise<void> {
|
||||
const { page } = options
|
||||
await disableGhostCursor({ page })
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,15 @@ const __dirname = path.dirname(__filename)
|
||||
|
||||
export const RELAY_PORT = Number(process.env.PLAYWRITER_PORT) || 19988
|
||||
|
||||
export type ExtensionStatus = {
|
||||
extensionId: string
|
||||
stableKey?: string
|
||||
browser: string | null
|
||||
profile: { email: string; id: string } | null
|
||||
activeTargets: number
|
||||
playwriterVersion: string | null
|
||||
}
|
||||
|
||||
export async function getRelayServerVersion(port: number = RELAY_PORT): Promise<string | null> {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/version`, {
|
||||
@@ -46,33 +55,81 @@ export async function getExtensionStatus(
|
||||
}
|
||||
}
|
||||
|
||||
export async function getExtensionsStatus(port: number = RELAY_PORT): Promise<ExtensionStatus[]> {
|
||||
try {
|
||||
const response = await fetch(`http://127.0.0.1:${port}/extensions/status`, {
|
||||
signal: AbortSignal.timeout(2000),
|
||||
})
|
||||
if (!response.ok) {
|
||||
const fallback = await fetch(`http://127.0.0.1:${port}/extension/status`, {
|
||||
signal: AbortSignal.timeout(2000),
|
||||
})
|
||||
if (!fallback.ok) {
|
||||
return []
|
||||
}
|
||||
|
||||
const fallbackData = (await fallback.json()) as {
|
||||
connected: boolean
|
||||
activeTargets: number
|
||||
browser: string | null
|
||||
profile: { email: string; id: string } | null
|
||||
playwriterVersion?: string | null
|
||||
}
|
||||
|
||||
if (!fallbackData?.connected) {
|
||||
return []
|
||||
}
|
||||
|
||||
return [
|
||||
{
|
||||
extensionId: 'default',
|
||||
stableKey: undefined,
|
||||
browser: fallbackData.browser,
|
||||
profile: fallbackData.profile,
|
||||
activeTargets: fallbackData.activeTargets,
|
||||
playwriterVersion: fallbackData.playwriterVersion || null,
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
const data = (await response.json()) as {
|
||||
extensions: ExtensionStatus[]
|
||||
}
|
||||
|
||||
return data.extensions || []
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for the extension to connect to the relay server.
|
||||
* Returns true if connected within timeout, false otherwise.
|
||||
* Wait for at least one extension to appear in extensions status.
|
||||
* Returns connected extension entries, or [] on timeout.
|
||||
*/
|
||||
export async function waitForExtension(
|
||||
export async function waitForConnectedExtensions(
|
||||
options: {
|
||||
port?: number
|
||||
timeoutMs?: number
|
||||
pollIntervalMs?: number
|
||||
logger?: { log: (...args: any[]) => void }
|
||||
} = {},
|
||||
): Promise<boolean> {
|
||||
const { port = RELAY_PORT, timeoutMs = 5000, logger } = options
|
||||
): Promise<ExtensionStatus[]> {
|
||||
const { port = RELAY_PORT, timeoutMs = 5000, pollIntervalMs = 200, logger } = options
|
||||
const startTime = Date.now()
|
||||
|
||||
logger?.log(pc.dim('Waiting for extension to connect...'))
|
||||
|
||||
while (Date.now() - startTime < timeoutMs) {
|
||||
const status = await getExtensionStatus(port)
|
||||
if (status?.connected) {
|
||||
const extensions = await getExtensionsStatus(port)
|
||||
if (extensions.length > 0) {
|
||||
logger?.log(pc.green('Extension connected'))
|
||||
return true
|
||||
return extensions
|
||||
}
|
||||
await sleep(200)
|
||||
await sleep(pollIntervalMs)
|
||||
}
|
||||
|
||||
logger?.log(pc.yellow('Extension did not connect within timeout'))
|
||||
return false
|
||||
return []
|
||||
}
|
||||
|
||||
async function killRelayServer(options: { port: number; waitForFreeMs?: number }): Promise<void> {
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import type { Page } from '@xmorse/playwright-core'
|
||||
import type { BrowserContext, Page } from '@xmorse/playwright-core'
|
||||
import type {
|
||||
StartRecordingResult,
|
||||
StopRecordingResult,
|
||||
@@ -16,6 +16,7 @@ import type {
|
||||
CancelRecordingResult,
|
||||
} from './protocol.js'
|
||||
import { EXTENSION_IDS } from './utils.js'
|
||||
import { RecordingGhostCursorController } from './recording-ghost-cursor.js'
|
||||
|
||||
/**
|
||||
* Generate a shell command to quit and restart Chrome with flags that allow automatic tab capture.
|
||||
@@ -86,6 +87,132 @@ export interface RecordingState {
|
||||
tabId?: number
|
||||
}
|
||||
|
||||
export interface ExecutionTimestamp {
|
||||
start: number
|
||||
end: number
|
||||
}
|
||||
|
||||
interface RecordingTargetOptions {
|
||||
page?: Page
|
||||
sessionId?: string
|
||||
}
|
||||
|
||||
interface CreateRecordingApiOptions {
|
||||
context: BrowserContext
|
||||
defaultPage: Page
|
||||
relayPort: number
|
||||
ghostCursorController: RecordingGhostCursorController
|
||||
onStart: () => void
|
||||
onFinish: () => void
|
||||
getExecutionTimestamps: () => ExecutionTimestamp[]
|
||||
}
|
||||
|
||||
interface StartRecordingWithDefaultsOptions extends Omit<StartRecordingOptions, 'relayPort'> {}
|
||||
interface StopRecordingWithDefaultsOptions extends Omit<StopRecordingOptions, 'relayPort'> {}
|
||||
interface IsRecordingWithDefaultsOptions {
|
||||
page?: Page
|
||||
sessionId?: string
|
||||
}
|
||||
interface CancelRecordingWithDefaultsOptions {
|
||||
page?: Page
|
||||
sessionId?: string
|
||||
}
|
||||
|
||||
function resolveRecordingTargetPage(options: {
|
||||
context: BrowserContext
|
||||
defaultPage: Page
|
||||
ghostCursorController: RecordingGhostCursorController
|
||||
target?: RecordingTargetOptions
|
||||
}): Page {
|
||||
return options.ghostCursorController.resolveRecordingTargetPage({
|
||||
context: options.context,
|
||||
defaultPage: options.defaultPage,
|
||||
target: options.target,
|
||||
})
|
||||
}
|
||||
|
||||
function withRecordingDefaults<T extends { page?: Page; sessionId?: string }, R>(options: {
|
||||
relayPort: number
|
||||
defaultPage: Page
|
||||
fn: (opts: T & { relayPort: number; sessionId?: string }) => Promise<R>
|
||||
}): (input?: T) => Promise<R> {
|
||||
const { relayPort, defaultPage, fn } = options
|
||||
return async (input: T = {} as T) => {
|
||||
const targetPage = input.page || defaultPage
|
||||
const sessionId = input.sessionId || targetPage.sessionId() || undefined
|
||||
return fn({ page: targetPage, sessionId, relayPort, ...input })
|
||||
}
|
||||
}
|
||||
|
||||
export function createRecordingApi(options: CreateRecordingApiOptions): {
|
||||
start: (opts?: StartRecordingWithDefaultsOptions) => Promise<RecordingState>
|
||||
stop: (opts?: StopRecordingWithDefaultsOptions) => Promise<{ path: string; duration: number; size: number; executionTimestamps: ExecutionTimestamp[] }>
|
||||
isRecording: (opts?: IsRecordingWithDefaultsOptions) => Promise<RecordingState>
|
||||
cancel: (opts?: CancelRecordingWithDefaultsOptions) => Promise<void>
|
||||
} {
|
||||
const { context, defaultPage, relayPort, ghostCursorController, onStart, onFinish, getExecutionTimestamps } = options
|
||||
|
||||
const startWithDefaults = withRecordingDefaults<StartRecordingWithDefaultsOptions, RecordingState>({
|
||||
relayPort,
|
||||
defaultPage,
|
||||
fn: startRecording,
|
||||
})
|
||||
const stopWithDefaults = withRecordingDefaults<StopRecordingWithDefaultsOptions, { path: string; duration: number; size: number }>({
|
||||
relayPort,
|
||||
defaultPage,
|
||||
fn: stopRecording,
|
||||
})
|
||||
const isRecordingWithDefaults = async (opts: IsRecordingWithDefaultsOptions = {}): Promise<RecordingState> => {
|
||||
const targetPage = opts.page || defaultPage
|
||||
const sessionId = opts.sessionId || targetPage.sessionId() || undefined
|
||||
return isRecording({ page: targetPage, sessionId, relayPort })
|
||||
}
|
||||
|
||||
const cancelWithDefaults = async (opts: CancelRecordingWithDefaultsOptions = {}): Promise<void> => {
|
||||
const targetPage = opts.page || defaultPage
|
||||
const sessionId = opts.sessionId || targetPage.sessionId() || undefined
|
||||
await cancelRecording({ page: targetPage, sessionId, relayPort })
|
||||
}
|
||||
|
||||
const start = async (opts?: StartRecordingWithDefaultsOptions): Promise<RecordingState> => {
|
||||
const targetPage = resolveRecordingTargetPage({ context, defaultPage, ghostCursorController, target: opts })
|
||||
const result = await startWithDefaults(opts)
|
||||
onStart()
|
||||
await ghostCursorController.enableForRecording({ page: targetPage })
|
||||
return result
|
||||
}
|
||||
|
||||
const stop = async (
|
||||
opts?: StopRecordingWithDefaultsOptions,
|
||||
): Promise<{ path: string; duration: number; size: number; executionTimestamps: ExecutionTimestamp[] }> => {
|
||||
const targetPage = resolveRecordingTargetPage({ context, defaultPage, ghostCursorController, target: opts })
|
||||
try {
|
||||
const result = await stopWithDefaults(opts)
|
||||
return { ...result, executionTimestamps: [...getExecutionTimestamps()] }
|
||||
} finally {
|
||||
onFinish()
|
||||
await ghostCursorController.disableForRecording({ page: targetPage })
|
||||
}
|
||||
}
|
||||
|
||||
const cancel = async (opts?: CancelRecordingWithDefaultsOptions): Promise<void> => {
|
||||
const targetPage = resolveRecordingTargetPage({ context, defaultPage, ghostCursorController, target: opts })
|
||||
try {
|
||||
await cancelWithDefaults(opts)
|
||||
} finally {
|
||||
onFinish()
|
||||
await ghostCursorController.disableForRecording({ page: targetPage })
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
start,
|
||||
stop,
|
||||
isRecording: isRecordingWithDefaults,
|
||||
cancel,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Start recording the page.
|
||||
* The recording is handled by the extension, so it survives page navigation.
|
||||
|
||||
+28
-11
@@ -861,13 +861,15 @@ await screenshotWithAccessibilityLabels({ page: state.page })
|
||||
|
||||
Labels are color-coded: yellow=links, orange=buttons, coral=inputs, pink=checkboxes, peach=sliders, salmon=menus, amber=tabs.
|
||||
|
||||
**startRecording / stopRecording** - record the page as a video at native FPS (30-60fps). Uses `chrome.tabCapture` in the extension context, so **recording survives page navigation**. Video is saved as mp4.
|
||||
**recording.start / recording.stop** - record the page as a video at native FPS (30-60fps). Uses `chrome.tabCapture` in the extension context, so **recording survives page navigation**. Video is saved as mp4.
|
||||
|
||||
While recording is active, Playwriter automatically overlays a smooth ghost cursor that follows automated mouse actions (`page.mouse.*`, `locator.click()`, hover flows) using `page.onMouseAction` from the Playwright fork.
|
||||
|
||||
**Note**: Recording requires the user to have clicked the Playwriter extension icon on the tab. This grants `activeTab` permission needed for `chrome.tabCapture`. Recording works on tabs where the icon was clicked - if you need to record a new tab, ask the user to click the icon on it first.
|
||||
|
||||
```js
|
||||
// Start recording - outputPath must be specified upfront
|
||||
await startRecording({
|
||||
await recording.start({
|
||||
page: state.page,
|
||||
outputPath: './recording.mp4',
|
||||
frameRate: 30, // default: 30
|
||||
@@ -881,7 +883,7 @@ await state.page.waitForLoadState('domcontentloaded')
|
||||
await state.page.goBack()
|
||||
|
||||
// Stop and get result
|
||||
const { path, duration, size } = await stopRecording({ page: state.page })
|
||||
const { path, duration, size } = await recording.stop({ page: state.page })
|
||||
console.log(`Saved ${size} bytes, duration: ${duration}ms`)
|
||||
```
|
||||
|
||||
@@ -889,17 +891,32 @@ Additional recording utilities:
|
||||
|
||||
```js
|
||||
// Check if recording is active
|
||||
const { isRecording, startedAt } = await isRecording({ page: state.page })
|
||||
const { isRecording, startedAt } = await recording.isRecording({ page: state.page })
|
||||
|
||||
// Cancel recording without saving
|
||||
await cancelRecording({ page: state.page })
|
||||
await recording.cancel({ page: state.page })
|
||||
```
|
||||
|
||||
**ghostCursor.show / ghostCursor.hide** - manually show or hide the in-page cursor overlay. Useful for screenshots and demos even when recording is not running.
|
||||
|
||||
```js
|
||||
// Show cursor in the center (or keep current position if already visible)
|
||||
await ghostCursor.show({ page: state.page })
|
||||
|
||||
// Optional styles: 'minimal' (default triangular pointer), 'dot', 'screenstudio'
|
||||
await ghostCursor.show({ page: state.page, style: 'minimal' })
|
||||
|
||||
// Hide cursor overlay
|
||||
await ghostCursor.hide({ page: state.page })
|
||||
```
|
||||
|
||||
`startRecording`, `stopRecording`, `isRecording`, and `cancelRecording` remain available as backward-compatible aliases.
|
||||
|
||||
**Key difference from getDisplayMedia**: This approach uses `chrome.tabCapture` which runs in the extension context, not the page. The recording persists across navigations because the extension holds the `MediaRecorder`, not the page's JavaScript context.
|
||||
|
||||
**createDemoVideo** - create a polished demo video from a recording by automatically speeding up idle sections (time between execute() calls) while keeping interactions at normal speed. Useful for creating demo videos of agent workflows without long pauses.
|
||||
|
||||
While recording is active, playwriter tracks when each `execute()` call starts and ends. `stopRecording()` returns these timestamps alongside the video file. `createDemoVideo` uses this data to identify idle gaps and speed them up with ffmpeg in a single pass.
|
||||
While recording is active, playwriter tracks when each `execute()` call starts and ends. `recording.stop()` returns these timestamps alongside the video file. `createDemoVideo` uses this data to identify idle gaps and speed them up with ffmpeg in a single pass.
|
||||
|
||||
A 1-second buffer is preserved around each interaction so viewers see context before and after each action.
|
||||
|
||||
@@ -907,7 +924,7 @@ Requires `ffmpeg` and `ffprobe` installed on the system.
|
||||
|
||||
```js
|
||||
// Start recording
|
||||
await startRecording({ page: state.page, outputPath: './recording.mp4' })
|
||||
await recording.start({ page: state.page, outputPath: './recording.mp4' })
|
||||
```
|
||||
|
||||
```js
|
||||
@@ -917,13 +934,13 @@ await startRecording({ page: state.page, outputPath: './recording.mp4' })
|
||||
|
||||
```js
|
||||
// Stop recording — executionTimestamps is included in the result
|
||||
const recording = await stopRecording({ page: state.page })
|
||||
const recordingResult = await recording.stop({ page: state.page })
|
||||
|
||||
// Create demo video — idle gaps are sped up 4x (default)
|
||||
const demoPath = await createDemoVideo({
|
||||
recordingPath: recording.path,
|
||||
durationMs: recording.duration,
|
||||
executionTimestamps: recording.executionTimestamps,
|
||||
recordingPath: recordingResult.path,
|
||||
durationMs: recordingResult.duration,
|
||||
executionTimestamps: recordingResult.executionTimestamps,
|
||||
speed: 5, // optional, default 5x for idle sections
|
||||
// outputFile: './demo.mp4', // optional, defaults to recording-demo.mp4
|
||||
})
|
||||
|
||||
@@ -6,5 +6,5 @@
|
||||
"types": ["node", "chrome"]
|
||||
},
|
||||
"include": ["src"],
|
||||
"exclude": ["src/a11y-client.ts"]
|
||||
"exclude": ["src/a11y-client.ts", "src/ghost-cursor-client.ts"]
|
||||
}
|
||||
|
||||
@@ -903,6 +903,9 @@ await recording.cancel({ page: state.page })
|
||||
// Show cursor in the center (or keep current position if already visible)
|
||||
await ghostCursor.show({ page: state.page })
|
||||
|
||||
// Optional styles: 'minimal' (default triangular pointer), 'dot', 'screenstudio'
|
||||
await ghostCursor.show({ page: state.page, style: 'minimal' })
|
||||
|
||||
// Hide cursor overlay
|
||||
await ghostCursor.hide({ page: state.page })
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user