Bug 260189 - AX: Voice Over cannot read the state changes when users interacts with some HTML element in PWA Mac
Summary: AX: Voice Over cannot read the state changes when users interacts with some H...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Accessibility (show other bugs)
Version: WebKit Nightly Build
Hardware: Mac (Intel) macOS 13
: P2 Major
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-08-14 19:26 PDT by xiaofeihan
Modified: 2023-08-14 19:26 PDT (History)
2 users (show)

See Also:


Attachments
accessibility inspector on PWA app (299.80 KB, image/png)
2023-08-14 19:26 PDT, xiaofeihan
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description xiaofeihan 2023-08-14 19:26:40 PDT
Created attachment 467274 [details]
accessibility inspector on PWA app

Recently, I found there are some problems with Voice Over on PWA app: Voice Over cannot read the state changes when users interacts with some HTML element.
I am not sure this bug is a chromium bug or Voice Over bug. I have reported the bug(https://bugs.chromium.org/p/chromium/issues/detail?id=1470130&q=pwa%20voiceover&can=2) to chromium, where I wrote the reproduce steps and video. 
 
I am working on the bug, but I don’t have much of an idea. So report here to see if you have some insight on this issue.
 
# How to reproduce the bug
1. Install PWA-test (pwa-a11-test.netlify.app) PWA app
2. Open the PWA app
3. Turn on the VoiceOver
4. Pressing `Space` on the checkbox element
(Note: I only use checkbox element as a example, other HTML element also have the similar problem, such as Pressing Left/Right on slider element)
Expected: Voice Over read the value changes, just like the behavior shown in the browser.
Actual: Voice Over do nothing.
 
# Chromium code analysis
The bug only exist in PWA scenario, it’s fine for browser scenario.
 
FYI, PWA Mac app has its own process(called app shim process), which spawn browser process(think it as Chrome app) and  then communicate between them. For the accessibility implementation for PWA App,  chromium use  NSAccessibilityRemoteUIElement, a private Apple API, to make app shim process have the all accessibility ability of browser process. I doubt the bug has anything to do with NSAccessibilityRemoteUIElement, but I am not familiar with the undocumented API.
 
See the chromium code(https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_manager_mac.mm;l=171), when pressing the `Space` on checkbox, chromium will call  NSAccessibilityPostNotification with NSAccessibilityValueChangedNotification. Voice Over can’t read the value change for PWA Mac app, but can read for browser scenario.
 
I use Xcode accessibility inspector tool to see the notification posted from PWA App. I found the notification was sent successfully, but Voice Over can not read it! So I doubt voice over is doing some check to disallow read it in this case?
 
Do you have any thoughts on this? Appreciate it if there is any comments or response!!
Comment 1 Radar WebKit Bug Importer 2023-08-14 19:26:49 PDT
<rdar://problem/113885286>