Commit Graph

150 Commits

Author SHA1 Message Date
Tommy D. Rossi 36f3801451 fix(extension): guard contextMenus calls when API is unavailable
Prevent service worker startup crashes in environments where chrome.contextMenus is undefined. Use optional chaining for create, update, and onClicked registration so missing context menu APIs degrade gracefully instead of throwing uncaught TypeError.
2026-03-12 15:36:12 +01:00
Tommy D. Rossi 1ced689153 extension: remove chrome.alarms keepalive and alarms permission
The alarm added in 0.0.73 was a no-op — the maintainLoop while(true) + sleep(1000)
and setInterval(checkMemory, 5000) already keep the service worker alive.
Removing it drops the unnecessary alarms permission from the manifest.

Bumps extension to 0.0.75.
2026-03-10 21:28:17 +01:00
Tommy D. Rossi 96458f41f0 fix: reload script hangs — use open instead of osascript
osascript 'tell application Chrome to open location' blocks indefinitely
waiting for Chrome to finish handling the chrome:// URL. The open command
returns immediately.
2026-03-04 22:02:58 +01:00
Tommy D. Rossi 905df6eec5 fix: route Target.detachFromTarget on root CDP session (#40)
When Playwright sends Target.detachFromTarget via the root browser session
(no top-level sessionId), the extension couldn't find the target tab because
it only checked msg.params.sessionId for routing. This caused 'No tab found'
errors that cascaded into disconnects and instability.

- Add getTabForCommand() helper with params.sessionId fallback so any command
  referencing a session in its params can be routed when the top-level
  sessionId is absent
- No-op Target.detachFromTarget for stale/unknown sessions instead of throwing
- Always re-apply tab group color on every sync to prevent Chrome resetting
  it to white
- Replace silent .catch() with error log in aria-snapshot OOPIF detach
- Add regression test using raw WebSocket to verify routing without sessionId

Extension bumped to 0.0.74.
2026-03-04 21:57:27 +01:00
Tommy D. Rossi f793dd8ac4 feat: harden relay sessions and recording UX across extension + playwriter
This commit batches the pending workspace updates that were coupled in practice: extension reliability fixes, ghost cursor persistence across MPA navigations, timeout tuning, and aligned docs/versioning updates so behavior and guidance stay in sync.

Key updates:\n- extension: add MV3 keepalive via chrome.alarms, add alarms permission, and bump extension changelog/version to 0.0.73 to reduce silent relay disconnects while idle\n- playwriter runtime: raise default action timeout to 60s while keeping navigation timeout separate; increase relay /version fetch timeout to 2000ms\n- recording + cursor: add persistent cursor init scripts with Page.addScriptToEvaluateOnNewDocument, remove init script on teardown, and ensure cursor enablement before applying mouse actions\n- demo video defaults: use 0.5s side buffer (1s total) and default idle speed 6x for createDemoVideo/computeIdleSections\n- tests/docs: keep click-error tests deterministic with explicit 5000ms per-click timeout, remove outdated click-timeout section from skill mistakes, and update package/changelog entries
2026-02-28 22:12:07 +01:00
Tommy D. Rossi c9809967e5 fix: hide tab group when relay dies instead of keeping it green
syncTabGroup was including 'connecting' tabs in the group unconditionally,
so when the relay WebSocket closed and all tabs transitioned to 'connecting',
the group stayed green. Now 'connecting' tabs are only included when the
relay is actually connected (connectionState === 'connected'). When the
relay is dead, 'connecting' tabs are excluded and the group gets cleaned up.

The onUpdated handler at line ~1601 already guards against the
ungroup→disconnect loop for 'connecting' tabs, so this is safe.
2026-02-27 09:04:43 +01:00
Tommy D. Rossi 95ff2a797d fix: scope CDP tab sessions and simplify recording sessionId routing 2026-02-25 16:52:41 +01:00
Tommy D. Rossi f87b0243cd format 2026-02-22 15:21:38 +01:00
Tommy D. Rossi 539aeaac91 rename accessibilitySnapshot to snapshot for shorter agent usage
The function is now exposed as `snapshot()` in the executor sandbox scope.
`accessibilitySnapshot` remains as a backward-compatible alias pointing to
the same function, so existing agents and saved commands continue to work.

Updated all code examples across skill.md, README, PLAYWRITER_AGENTS,
docs, tests, and the extension welcome page to use the shorter name.
2026-02-18 17:13:57 +01:00
Tommy D. Rossi 44370833f7 feat: bundle playwriter version in extension, warn when CLI/MCP is outdated
The extension now reports which playwriter version it was built with via a
`?v=` query param on its WebSocket connection to the relay. The relay stores
this and exposes it in `/extension/status` and `/extensions/status` endpoints.

CLI and MCP compare the extension's bundled version against their own VERSION.
If the extension was built with a newer playwriter, a warning is shown:

  Playwriter 0.0.60 is outdated (extension requires 0.0.62).
  Run `npm install -g playwriter@latest` or update the playwriter package.

This is fully backward compatible:
- Old extensions without `?v=` get `playwriterVersion: null`, no warning
- No WS protocol changes (version is in URL query params)
- Warning fires once per executor/session lifetime

Flow:
  vite build reads playwriter/package.json → injects __PLAYWRITER_VERSION__
  → extension sends ?v=X.Y.Z on WS connect → relay stores in ExtensionInfo
  → /extension/status returns playwriterVersion → CLI/executor checks & warns
2026-02-16 19:42:58 +01:00
Tommy D. Rossi e6f1e8c089 fix extension runtime session routing for oopif iframe locators
Route Runtime.enable/disable through the incoming child sessionId instead of always using the tab root session so OOPIF iframe targets receive execution contexts reliably. Add a focused relay regression test for empty-src cross-origin iframe attach flow and update extension version/changelog for release tracking.
2026-02-11 16:15:02 +01:00
Tommy D. Rossi 5a3f344a0b Update manifest.json 2026-02-07 09:49:41 +01:00
Tommy D. Rossi 78734bf1e8 fix: prevent tab group add/remove infinite loop when dragging tabs
When a user drags a tab into the playwriter group, connectTab() sets
its state to 'connecting'. syncTabGroup() previously only kept
'connected' tabs in the group, so it would immediately ungroup the
'connecting' tab. This ungroup fired chrome.tabs.onUpdated, which
called disconnectTab(). When the original connectTab() resolved, it
re-added the tab, triggering onUpdated again — creating an infinite
add/remove/add/remove loop.

Fix: include 'connecting' tabs in syncTabGroup's group membership
filter. This way syncTabGroup won't fight the user's intent by
removing tabs that are still being set up. Error-state tabs are
still correctly excluded from the group.
2026-02-06 15:39:35 +01:00
Tommy D. Rossi 1a59082e12 feat: auto-connect tabs created via ghostPublicAPI.openTab
Tabs created via Ghost Browser's openTab API now automatically
attach to Playwriter's debugger, making them immediately available
in context.pages() without requiring manual extension icon click.
2026-02-04 17:47:57 +01:00
Tommy D. Rossi 321acf9dbc Update ghost-browser-api.d.ts 2026-02-04 17:24:36 +01:00
Tommy D. Rossi 44514f00c8 feat: add multi-browser support
- Extension connections are now tracked per-connection with isolated state
- Each connection has its own connectedTargets, pendingRequests, messageId
- Sessions are bound to specific extensionId via SessionMetadata
- New /extensions/status endpoint for listing all connected browsers
- CLI session new shows browser selection when multiple connected
- Browser detection via navigator.userAgentData (Ghost, Brave, Edge, etc)
- Added identity permission for Chrome profile email detection
2026-02-04 16:43:09 +01:00
Tommy D. Rossi c623c91891 extract Ghost Browser code to shared ghost-browser.ts
- Constants, types, proxy factory, and handler in one file
- Works in both Node.js (executor) and Chrome (extension) environments
- executor.ts imports createGhostBrowserChrome factory
- background.ts imports handleGhostBrowserCommand handler
2026-02-04 13:16:29 +01:00
Tommy D. Rossi f2faee50c0 move Ghost Browser docs to .d.ts, shorten skill.md section 2026-02-04 13:14:46 +01:00
Tommy D. Rossi a1178cbd35 add Ghost Browser API integration
Exposes chrome.ghostPublicAPI, chrome.ghostProxies, and chrome.projects
in the executor sandbox when running in Ghost Browser.

- protocol.ts: add GhostBrowserCommandMessage type
- background.ts: handle ghost-browser messages, call chrome APIs
- cdp-relay.ts: route ghost-browser commands to extension
- executor.ts: inject chrome object with Proxy-based API namespaces
- skill.md: document Ghost Browser integration with examples

Enables multi-identity automation for managing multiple accounts,
rotating proxies per tab/identity, and isolated cookie sessions.
2026-02-04 13:09:41 +01:00
Tommy D. Rossi a52ec9a8f2 add Ghost Browser API type definitions
Extracted from Ghost Browser.app binary. Includes:
- chrome.ghostPublicAPI: tab/identity management
- chrome.ghostProxies: full proxy control per tab/identity/session
- chrome.projects: identity/workspace/session management
- Extended chrome.tabs with Ghost-specific properties
2026-02-04 12:55:03 +01:00
Tommy D. Rossi a9a40e1183 fix screen recording: resolve relative paths, buffer chunks, wait for encoder 2026-01-29 23:30:01 +01:00
Tommy D. Rossi db54c35b83 mp4 2026-01-29 20:30:10 +01:00
Tommy D. Rossi 4251562ff7 no MCP in name 2026-01-29 20:23:52 +01:00
Tommy D. Rossi 7e4dc3fd18 Update welcome.html 2026-01-27 18:39:55 +01:00
Tommy D. Rossi 5dc39cc4fe style(extension): tweak welcome page text opacity and remove redundant copy 2026-01-27 18:39:43 +01:00
Tommy D. Rossi 653f00c3a3 feat(extension): redesign welcome page with Prism.js syntax highlighting 2026-01-27 18:38:41 +01:00
Tommy D. Rossi 67d017237e fix(cdp): isolate extension builds and keep oopif auto-attach 2026-01-27 16:05:15 +01:00
Tommy D. Rossi a8412343e1 fix merge artifacts. fix build 2026-01-27 13:54:26 +01:00
Tommy D. Rossi 53450ee36f refactor: remove dependency injection, use direct imports
- Export store, connectionManager, logger, sendMessage, getTabBySessionId from background.ts
- recording.ts now imports directly instead of using initRecording()
- Simplify RecordingRelay constructor to take params directly instead of interface
2026-01-27 13:35:23 +01:00
Tommy D. Rossi 4a8caf19bd refactor: extract recording code into separate modules with proper types
- Create extension/src/offscreen-types.ts with Chrome tabCapture constraint types
- Create extension/src/recording.ts extracting recording functions from background.ts
- Create playwriter/src/recording-relay.ts extracting recording logic from cdp-relay.ts
- Remove 'as any' usages by adding proper MediaStream constraint types
- Add typesafe message types for offscreen document communication
- Remove duplicate OUR_EXTENSION_IDS constant (uses EXTENSION_IDS from utils.ts)
2026-01-27 13:35:23 +01:00
Tommy D. Rossi 54d3f02a5b feat: support concurrent screen recordings across multiple tabs
- Change offscreen.ts from single recording variable to Map<tabId, RecordingState>
- Update all recording message handlers to route by tabId
- Track sessionId in relay server for proper recording lookup when stopping
- All recording operations now consistently require tabId
2026-01-27 13:35:23 +01:00
Tommy D. Rossi c9e8361a54 fix: address PR review comments
- Fix misleading comment about user gesture requirement in background.ts
- Remove unused 'target' variable in /recording/stop endpoint
- Fix /recording/cancel to not delete all recordings (handled by event)
- Remove redundant security middlewares (CORS handles it)
2026-01-27 13:35:22 +01:00
Tommy D. Rossi 050c6d5b50 feat: use MP4 format for screen recording instead of WebM 2026-01-27 13:35:22 +01:00
Tommy D. Rossi df63f418e5 feat: add Chrome flags for automated recording without user interaction
- Add getChromeRestartCommand() with --allowlisted-extension-id and --auto-accept-this-tab-capture flags
- Move EXTENSION_IDS to utils.ts for cleaner imports
- Add sessionId support for multi-tab recording
- Add helpful error message when activeTab permission is missing
- Add null check for empty extension result
- Document offscreen recording flow with ASCII diagram
2026-01-27 13:35:22 +01:00
Tommy D. Rossi 8449006e59 refactor: simplify screen recording code
- Fix vite output path (was broken lib/background.mjs, now background.js)
- Import recording types from protocol.ts instead of duplicating in background.ts
- Remove dead getSessionId() and unused sessionId params from screen-recording.ts
- Replace 4 executor wrapper functions with single withRecordingDefaults helper
- Fix binary chunk routing with lastRecordingMetadataTabId tracking
- Add ExtensionStopRecordingResult type for proper extension/relay separation

Saves ~56 lines while improving correctness of multi-tab recording.
2026-01-27 13:35:22 +01:00
Tommy D. Rossi 3b5427ebe8 update notes on permissions for recording 2026-01-27 13:35:22 +01:00
Tommy D. Rossi 09f502cc43 Update background.ts 2026-01-27 13:35:22 +01:00
Tommy D. Rossi b52928cc90 refactor: move HTML files to src/, use vite HTML entry points
- Move welcome.html to src/
- Use vite HTML inputs instead of static copy for offscreen.html and welcome.html
- Source HTML references .ts, vite transforms to .js
- Update paths in background.ts to src/
2026-01-27 13:35:22 +01:00
Tommy D. Rossi f8d6640819 refactor: use vite rollupOptions instead of esbuild for offscreen.ts 2026-01-27 13:35:22 +01:00
Tommy D. Rossi 74497c2a1a feat: implement screen recording via chrome.tabCapture with offscreen document
- Use chrome.tabCapture.getMediaStreamId() from service worker
- Pass stream ID to offscreen document for MediaRecorder
- Recording survives page navigation (unlike getDisplayMedia)
- Requires user to click extension icon first (activeTab permission)
- Add esbuild to build offscreen.ts separately
- Update prompt docs with recording instructions
2026-01-27 13:35:22 +01:00
Tommy D. Rossi e56e27e5e5 WIP: screen recording via chrome.tabCapture
Work in progress for navigation-safe screen recording.

Implemented:
- tabCapture permission in manifest
- Recording handlers in extension background.ts
- Recording endpoints in relay server
- Binary WebSocket streaming for video chunks
- Protocol types for recording messages
- Executor integration

Needs:
- Offscreen document implementation (tabCapture not available in service workers)
- Connect background.ts to offscreen document via messages
- Update manifest for offscreen document

The offscreen.ts/html files are scaffolding for this next step.
2026-01-27 13:35:22 +01:00
Tommy D. Rossi 81988c3305 show cli examples in welcome 2026-01-27 13:29:03 +01:00
Tommy D. Rossi d4107d1ddd fix(extension): detach child sessions on parent detach 2026-01-26 15:04:37 +01:00
Tommy D. Rossi d51d8b8e72 feat(cdp): allow iframe targets and serialize extension builds 2026-01-26 14:56:17 +01:00
Tommy D. Rossi 4d2606253b add source param in CDP messages 2026-01-26 13:07:59 +01:00
Tommy D. Rossi dc771ddc1d sync tab state for automated tabs 2026-01-26 12:56:27 +01:00
Tommy D. Rossi 0fb9d2c229 Clamp extension log payloads 2026-01-25 15:01:40 +01:00
Tommy D. Rossi d4e49b2d5d nn 2026-01-25 14:58:00 +01:00
Tommy D. Rossi a15b06082b Add stable extension ID for development builds
Inject manifest key via vite plugin for dev/test builds (not production).
This ensures all developers get the same extension ID: pebbngnfojnignonigcnkdilknapkgid

- Update vite.config.mts to inject key when !PRODUCTION
- Update release script to use PRODUCTION=true
- Replace old hardcoded dev extension ID in all files
2026-01-25 11:19:51 +01:00
Tommy D. Rossi f9869a59f2 refactor: move welcome.html to src/, simplify vite build config, cleanup cli.ts formatting 2026-01-23 23:57:25 +01:00