playwriter
Code Issues

format

Browse Source
This commit is contained in:
Tommy D. Rossi
2026-02-22 15:21:38 +01:00
parent e475c2f582
commit f87b0243cd
101 changed files with 14997 additions and 11339 deletions
Download Patch File Download Diff File Expand all files Collapse all files
slop/aria-snapshot-analysis.md
+57 -19
View File
@@ -26,9 +26,11 @@ The aria snapshot feature in playwriter extracts an accessibility tree from the
### Key Functions
#### `getAriaSnapshot()`
**Location:** Line 749-1033 in `aria-snapshot.ts`
Main entry point that returns `AriaSnapshotResult` containing:
- `snapshot`: String representation of the accessibility tree
- `tree`: Structured tree with nodes
- `refs`: Array of references to interactive elements
@@ -36,14 +38,15 @@ Main entry point that returns `AriaSnapshotResult` containing:
- `getRefsForLocators()`: Get refs for Playwright locators
**Signature:**
```typescript
export async function getAriaSnapshot({
page,
locator,
refFilter,
wsUrl,
interactiveOnly = false,
cdp
export async function getAriaSnapshot({
page,
locator,
refFilter,
wsUrl,
interactiveOnly = false,
cdp,
}: {
page: Page
locator?: Locator
@@ -60,14 +63,16 @@ export async function getAriaSnapshot({
**Command:** `DOM.getFlattenedDocument`
**Location:** Line 772
```typescript
const { nodes: domNodes } = await session.send('DOM.getFlattenedDocument', {
depth: -1,
pierce: true
}) as Protocol.DOM.GetFlattenedDocumentResponse
const { nodes: domNodes } = (await session.send('DOM.getFlattenedDocument', {
depth: -1,
pierce: true,
})) as Protocol.DOM.GetFlattenedDocumentResponse
```
**Parameters:**
- `depth: -1` - Get entire subtree
- `pierce: true` - **Traverses iframes and shadow roots**
@@ -77,12 +82,14 @@ const { nodes: domNodes } = await session.send('DOM.getFlattenedDocument', {
**Command:** `Accessibility.getFullAXTree`
**Location:** Line 791
```typescript
const { nodes: axNodes } = await session.send('Accessibility.getFullAXTree')
const { nodes: axNodes } = await session.send('Accessibility.getFullAXTree')
as Protocol.Accessibility.GetFullAXTreeResponse
```
**Parameters:** None specified (uses defaults)
- Default: Returns AX tree for root frame
- **Has optional `frameId` parameter** (not currently used)
@@ -93,10 +100,12 @@ const { nodes: axNodes } = await session.send('Accessibility.getFullAXTree')
### Current Implementation
**DOM Level:** ✅ **FULL SUPPORT**
- `DOM.getFlattenedDocument` with `pierce: true` traverses all iframes and shadow roots
- All DOM nodes from all frames are included in the flattened document
**Accessibility Level:** ⚠️ **LIMITED**
- `Accessibility.getFullAXTree` is called **without `frameId` parameter**
- According to CDP spec, when `frameId` is omitted, **only the root frame is used**
- Cross-origin iframes may have additional restrictions
@@ -104,6 +113,7 @@ const { nodes: axNodes } = await session.send('Accessibility.getFullAXTree')
### CDP Spec Details
From `Accessibility.pdl`:
```
experimental command getFullAXTree
parameters
@@ -126,11 +136,13 @@ experimental command getFullAXTree
### Evidence from Code
**Scope handling (Line 760-789):**
- Uses `data-pw-scope` attribute to scope snapshots to a locator
- Builds `allowedBackendIds` set from DOM tree traversal
- Filters AX nodes based on `backendDOMNodeId` membership in this set
**Node mapping:**
- Each AX node has `backendDOMNodeId` property linking to DOM node
- DOM nodes fetched with `pierce: true` include iframe contents
- But AX tree without `frameId` may not cover all frames
@@ -138,6 +150,7 @@ experimental command getFullAXTree
## Key Data Structures
### AriaSnapshotNode
```typescript
type AriaSnapshotNode = {
role: string
@@ -151,12 +164,13 @@ type AriaSnapshotNode = {
```
### AriaRef
```typescript
interface AriaRef {
role: string
name: string
ref: string // Full ref (testid or e1, e2, e3...)
shortRef: string // Short ref (e1, e2, e3...)
ref: string // Full ref (testid or e1, e2, e3...)
shortRef: string // Short ref (e1, e2, e3...)
backendNodeId?: Protocol.DOM.BackendNodeId
}
```
@@ -184,12 +198,28 @@ Generates Playwright-compatible locators:
**Location:** Lines 123-143
Only these roles get refs in interactive mode:
```typescript
const INTERACTIVE_ROLES = new Set([
'button', 'link', 'textbox', 'combobox', 'searchbox',
'checkbox', 'radio', 'slider', 'spinbutton', 'switch',
'menuitem', 'menuitemcheckbox', 'menuitemradio',
'option', 'tab', 'treeitem', 'img', 'video', 'audio',
'button',
'link',
'textbox',
'combobox',
'searchbox',
'checkbox',
'radio',
'slider',
'spinbutton',
'switch',
'menuitem',
'menuitemcheckbox',
'menuitemradio',
'option',
'tab',
'treeitem',
'img',
'video',
'audio',
])
```
@@ -214,7 +244,7 @@ const accessibilitySnapshot = async (options: {
wsUrl: getCdpUrl(this.cdpConfig),
interactiveOnly,
})
// Store refs for refToLocator() function
// Handle search filtering
// Handle diff mode
@@ -240,10 +270,12 @@ Takes a screenshot with Vimium-style labels overlaid on interactive elements:
**File:** `playwriter/src/aria-snapshot.test.ts`
Tests against real websites:
- Hacker News
- GitHub
**Coverage:**
- Snapshot generation
- Interactive-only mode
- Locator format validation
@@ -277,12 +309,14 @@ Tests against real websites:
To support iframes properly:
1. **Enumerate frames:**
```typescript
const { frameTree } = await session.send('Page.getFrameTree')
// Recursively collect all frame IDs
```
2. **Get AX tree per frame:**
```typescript
for (const frameId of frameIds) {
const { nodes } = await session.send('Accessibility.getFullAXTree', { frameId })
@@ -303,12 +337,14 @@ To support iframes properly:
## Summary
**File Paths:**
- Main implementation: `playwriter/src/aria-snapshot.ts`
- Executor integration: `playwriter/src/executor.ts` (lines 533-619)
- CDP session: `playwriter/src/cdp-session.ts`
- Tests: `playwriter/src/aria-snapshot.test.ts`
**CDP Commands:**
- `DOM.enable` - Enable DOM domain
- `DOM.getFlattenedDocument({ depth: -1, pierce: true })` - Get all DOM nodes including iframes
- `Accessibility.enable` - Enable accessibility domain
@@ -316,6 +352,7 @@ To support iframes properly:
- `DOM.getBoxModel({ backendNodeId })` - Get element positions for labels
**Frame Handling:**
- ✅ DOM tree includes iframe content (`pierce: true`)
- ⚠️ Accessibility tree likely **only root frame** (no `frameId` parameter)
- ❌ No frame enumeration or per-frame AX tree fetching
@@ -323,6 +360,7 @@ To support iframes properly:
**Key Insight:**
The current implementation may miss interactive elements inside iframes because:
1. `Accessibility.getFullAXTree()` without `frameId` only returns root frame
2. No iteration over child frames to collect their AX trees
3. Cross-origin iframes would be blocked anyway for security
slop/playwright-accessibility-findings.md
+100 -90
View File
@@ -6,6 +6,7 @@ description: Analysis of how Playwright implements accessibility snapshots and h
## Overview
This document contains findings from exploring the Playwright source code to understand:
1. How Playwright implements accessibility snapshots (ariaSnapshot)
2. Whether Playwright supports getting accessibility tree for iframes/child frames
3. How Playwright handles frame locators for accessibility
@@ -27,10 +28,10 @@ This document contains findings from exploring the Playwright source code to und
// Server-side (packages/playwright-core/src/server/frames.ts:1371)
async ariaSnapshot(progress: Progress, selector: string): Promise<string> {
return await this._retryWithProgressIfNotConnected(
progress,
selector,
true,
true,
progress,
selector,
true,
true,
handle => progress.race(handle.ariaSnapshot())
);
}
@@ -38,7 +39,7 @@ async ariaSnapshot(progress: Progress, selector: string): Promise<string> {
// Element handle (packages/playwright-core/src/server/dom.ts:757)
async ariaSnapshot(): Promise<string> {
return await this.evaluateInUtility(
([injected, element]) => injected.ariaSnapshot(element, { mode: 'expect' }),
([injected, element]) => injected.ariaSnapshot(element, { mode: 'expect' }),
{}
);
}
@@ -48,7 +49,7 @@ ariaSnapshot(node: Node, options: AriaTreeOptions): string {
return this.incrementalAriaSnapshot(node, options).full;
}
incrementalAriaSnapshot(node: Node, options: AriaTreeOptions & { track?: string }):
incrementalAriaSnapshot(node: Node, options: AriaTreeOptions & { track?: string }):
{ full: string, incremental?: string, iframeRefs: string[] } {
if (node.nodeType !== Node.ELEMENT_NODE)
throw this.createStacklessError('Can only capture aria snapshot of Element nodes.');
@@ -69,26 +70,27 @@ From `packages/injected/src/ariaSnapshot.ts:217-232`:
```typescript
function toAriaNode(element: Element, options: InternalOptions): aria.AriaNode | null {
const active = element.ownerDocument.activeElement === element;
const active = element.ownerDocument.activeElement === element
if (element.nodeName === 'IFRAME') {
const ariaNode: aria.AriaNode = {
role: 'iframe',
name: '',
children: [], // ⚠️ Empty children - no content from iframe
children: [], // ⚠️ Empty children - no content from iframe
props: {},
box: computeBox(element),
receivesPointerEvents: true,
active
};
setAriaNodeElement(ariaNode, element);
computeAriaRef(ariaNode, options);
return ariaNode;
active,
}
setAriaNodeElement(ariaNode, element)
computeAriaRef(ariaNode, options)
return ariaNode
}
// ...
}
```
**Key observations:**
- IFrame elements are detected and added to the tree with `role: 'iframe'`
- The `children` array is ALWAYS empty for iframes
- IFrame refs are tracked separately in `snapshot.iframeRefs` array
@@ -99,11 +101,11 @@ function toAriaNode(element: Element, options: InternalOptions): aria.AriaNode |
```typescript
export type AriaSnapshot = {
root: aria.AriaNode;
elements: Map<string, Element>; // ref -> Element mapping
refs: Map<Element, string>; // Element -> ref mapping
iframeRefs: string[]; // List of iframe ref IDs
};
root: aria.AriaNode
elements: Map<string, Element> // ref -> Element mapping
refs: Map<Element, string> // Element -> ref mapping
iframeRefs: string[] // List of iframe ref IDs
}
```
The `iframeRefs` array contains references to iframe elements but NOT their content.
@@ -115,34 +117,37 @@ The CDP protocol DOES support frame-specific accessibility queries:
```typescript
// From protocol.d.ts
export type getFullAXTreeParameters = {
depth?: number;
frameId?: Page.FrameId; // ⚠️ Supports frame-specific queries!
depth?: number
frameId?: Page.FrameId // ⚠️ Supports frame-specific queries!
}
export type getPartialAXTreeParameters = {
nodeId?: DOM.NodeId;
backendNodeId?: DOM.BackendNodeId;
objectId?: Runtime.RemoteObjectId;
fetchRelatives?: boolean;
nodeId?: DOM.NodeId
backendNodeId?: DOM.BackendNodeId
objectId?: Runtime.RemoteObjectId
fetchRelatives?: boolean
}
```
**However, Playwright does NOT use these CDP commands anywhere in the codebase.**
Search results show:
- CDP types defined in `protocol.d.ts`
- CDP types defined in `protocol.d.ts`
- No actual usage in Chromium implementation files
- Firefox uses a custom `Accessibility.getFullAXTree` via Juggler protocol
### 5. Why Playwright Uses DOM Traversal Instead of CDP
**Advantages of DOM traversal approach:**
1. **Cross-browser compatibility** - Works in Firefox, WebKit, Chromium
2. **Full control** - Can customize what gets included/excluded
3. **Performance** - No serialization overhead for large trees
4. **Flexibility** - Can implement custom filtering (visibility, aria roles, etc.)
**Disadvantages:**
1. **Cannot access iframe content** - Browser security prevents cross-origin access
2. **Must execute in each frame separately** - No single command for entire page tree
3. **Slower for deep trees** - Must traverse DOM node-by-node
@@ -158,19 +163,21 @@ Based on code analysis:
3. **By design**: The `generateAriaTree` function explicitly skips iframe children
**To get iframe content accessibility tree, you would need to:**
1. Switch to the iframe's frame context
2. Call `ariaSnapshot()` again on that frame
3. Manually combine the results
Example:
```typescript
// Get main frame snapshot
const mainSnapshot = await page.locator('body').ariaSnapshot();
const mainSnapshot = await page.locator('body').ariaSnapshot()
// Get iframe content
const frameElement = await page.frameLocator('iframe');
const frame = await frameElement.owner();
const frameSnapshot = await frame.contentFrame().locator('body').ariaSnapshot();
const frameElement = await page.frameLocator('iframe')
const frame = await frameElement.owner()
const frameSnapshot = await frame.contentFrame().locator('body').ariaSnapshot()
// Results are separate - no automatic merging
```
@@ -184,14 +191,16 @@ const frameSnapshot = await frame.contentFrame().locator('body').ariaSnapshot();
3. Merge the results into a single tree
**CDP Command:**
```typescript
await session.send('Accessibility.getFullAXTree', {
await session.send('Accessibility.getFullAXTree', {
frameId: 'frame-id-here',
depth: -1 // unlimited depth
});
depth: -1, // unlimited depth
})
```
This would return the full accessibility tree including iframe content, but:
- Only works in Chromium (not Firefox/WebKit)
- Returns CDP's AXNode format, not Playwright's AriaNode format
- Would need conversion logic
@@ -205,34 +214,34 @@ Get accessibility tree for each frame separately:
```typescript
// Pseudo-code for MCP implementation
async function getAccessibilitySnapshot({ sessionId, includeFrames = false }) {
const page = getPage(sessionId);
const page = getPage(sessionId)
// Get main frame snapshot
const mainSnapshot = await page.evaluate(() => {
return injected.ariaSnapshot(document.body, { mode: 'ai' });
});
return injected.ariaSnapshot(document.body, { mode: 'ai' })
})
if (!includeFrames) {
return mainSnapshot;
return mainSnapshot
}
// Get all iframe snapshots
const frames = page.frames();
const frames = page.frames()
const frameSnapshots = await Promise.all(
frames.slice(1).map(async (frame) => {
return {
frameId: frame.name() || frame.url(),
snapshot: await frame.evaluate(() => {
return injected.ariaSnapshot(document.body, { mode: 'ai' });
})
};
})
);
return injected.ariaSnapshot(document.body, { mode: 'ai' })
}),
}
}),
)
return {
main: mainSnapshot,
frames: frameSnapshots
};
frames: frameSnapshots,
}
}
```
@@ -242,24 +251,24 @@ Use CDP `Accessibility.getFullAXTree` for each frame:
```typescript
async function getFullAccessibilityTree({ sessionId }) {
const page = getPage(sessionId);
const frames = page.frames();
const page = getPage(sessionId)
const frames = page.frames()
const trees = await Promise.all(
frames.map(async (frame) => {
const session = await frame._client; // Get CDP session
const session = await frame._client // Get CDP session
const { nodes } = await session.send('Accessibility.getFullAXTree', {
frameId: frame._id
});
frameId: frame._id,
})
return {
frameId: frame.url(),
nodes
};
})
);
nodes,
}
}),
)
// Convert CDP AXNode[] to Playwright AriaNode format
return convertCDPtoAria(trees);
return convertCDPtoAria(trees)
}
```
@@ -274,56 +283,57 @@ async function getFullAccessibilityTree({ sessionId }) {
```typescript
async function getRecursiveSnapshot({ sessionId, maxDepth = 3 }) {
const page = getPage(sessionId);
const page = getPage(sessionId)
async function getFrameSnapshot(frame, depth = 0) {
if (depth >= maxDepth) return null;
if (depth >= maxDepth) return null
// Get snapshot for this frame
const result = await frame.evaluate(() => {
return injected.incrementalAriaSnapshot(document.body, {
return injected.incrementalAriaSnapshot(document.body, {
mode: 'ai',
refPrefix: `f${depth}_`
});
});
refPrefix: `f${depth}_`,
})
})
// Find iframe elements
const iframeElements = await frame.$$('iframe');
const iframeElements = await frame.$$('iframe')
// Get snapshots for child frames
const childSnapshots = await Promise.all(
iframeElements.map(async (iframeEl) => {
const childFrame = await iframeEl.contentFrame();
if (!childFrame) return null;
const childFrame = await iframeEl.contentFrame()
if (!childFrame) return null
return {
iframeSrc: await iframeEl.getAttribute('src'),
content: await getFrameSnapshot(childFrame, depth + 1)
};
})
);
content: await getFrameSnapshot(childFrame, depth + 1),
}
}),
)
return {
snapshot: result.full,
iframes: childSnapshots.filter(Boolean)
};
iframes: childSnapshots.filter(Boolean),
}
}
return await getFrameSnapshot(page.mainFrame());
return await getFrameSnapshot(page.mainFrame())
}
```
## Summary
| Feature | Playwright Support | Notes |
|---------|-------------------|-------|
| Accessibility snapshot for current frame | ✅ Yes | Via injected script DOM traversal |
| Accessibility snapshot for iframe content | ❌ No | Iframes detected but content not included |
| CDP Accessibility commands | ❌ Not used | Available but Playwright doesn't use them |
| Cross-browser support | ✅ Yes | Works in all browsers via DOM traversal |
| Frame-specific queries via CDP | ⚠️ Available | `frameId` parameter exists but unused |
| Multi-frame snapshot | ⚠️ Manual | Must query each frame separately |
| Feature | Playwright Support | Notes |
| ----------------------------------------- | ------------------ | ----------------------------------------- |
| Accessibility snapshot for current frame | ✅ Yes | Via injected script DOM traversal |
| Accessibility snapshot for iframe content | ❌ No | Iframes detected but content not included |
| CDP Accessibility commands | ❌ Not used | Available but Playwright doesn't use them |
| Cross-browser support | ✅ Yes | Works in all browsers via DOM traversal |
| Frame-specific queries via CDP | ⚠️ Available | `frameId` parameter exists but unused |
| Multi-frame snapshot | ⚠️ Manual | Must query each frame separately |
**Bottom line:** Playwright's `ariaSnapshot()` works on a single frame at a time. To get iframe content, you must:
1. Get the iframe element
2. Access its `contentFrame()`
3. Call `ariaSnapshot()` on that frame