Bug 264642

Summary: AX: Cache accessibilityText to serve Title, Description, Help Text attributes off of the main thread
Product: WebKit Reporter: Joshua Hoffman <jhoffman23>
Component: AccessibilityAssignee: Joshua Hoffman <jhoffman23>
Status: RESOLVED FIXED    
Severity: Normal CC: aboxhall, andresg_22, apinheiro, cfleizach, dmazzoni, ews-watchlist, jcraig, samuel_white, tyler_w, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: All   
OS: All   
Attachments:
Description Flags
Patch
ews-feeder: commit-queue-
Patch
none
Patch
none
Patch
none
Patch none

Joshua Hoffman
Reported 2023-11-10 15:48:11 PST
We currently go to the main thread the first time Title/Description/Help text is requested for an element. We should be serving these on of the secondary thread and caching them upfront.
Attachments
Patch (28.66 KB, patch)
2023-11-10 16:23 PST, Joshua Hoffman
ews-feeder: commit-queue-
Patch (28.66 KB, patch)
2023-11-10 16:47 PST, Joshua Hoffman
no flags
Patch (27.45 KB, patch)
2023-11-15 09:26 PST, Joshua Hoffman
no flags
Patch (27.63 KB, patch)
2023-11-15 15:16 PST, Joshua Hoffman
no flags
Patch (27.62 KB, patch)
2023-11-15 21:13 PST, Joshua Hoffman
no flags
Radar WebKit Bug Importer
Comment 1 2023-11-10 15:48:24 PST
Joshua Hoffman
Comment 2 2023-11-10 16:23:18 PST
Joshua Hoffman
Comment 3 2023-11-10 16:47:43 PST
Tyler Wilcock
Comment 4 2023-11-13 10:17:22 PST
Comment on attachment 468567 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review > Source/WebCore/accessibility/AXCoreObject.h:678 > + AccessibilityText(const String& t, const AccessibilityTextSource& s) > + : text(t) > + , textSource(s) I know you're only copy-pasting this, but maybe we can change it to follow the style guidelines and give these proper names, e.g. `text` and `source` > Source/WebCore/accessibility/AccessibilityObject.h:121 > + bool isFileUploadButton() const override; Can this be final? > Source/WebCore/accessibility/AccessibilityObject.h:190 > + bool isMeter() const override; Can this be final? > Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:72 > + Vector<AccessibilityText> texts; > + object->accessibilityText(texts); This may be a good opportunity to make AXCoreObject::accessibilityText return a Vector<AccessibilityText>, rather than returning void and taking in a Vector<AccessibilityText>&. Although that's not related to the intention of your patch, so if you wanted to skip it that's OK too. > Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75 > + auto axTextValue = texts.map([] (const auto& text) -> AccessibilityText { > + return { text.text.isolatedCopy(), text.textSource }; > + }); This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question?
Joshua Hoffman
Comment 5 2023-11-13 10:24:20 PST
Comment on attachment 468567 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review >> Source/WebCore/accessibility/AXCoreObject.h:678 >> + , textSource(s) > > I know you're only copy-pasting this, but maybe we can change it to follow the style guidelines and give these proper names, e.g. `text` and `source` Yeah, I'll go ahead and update those. >> Source/WebCore/accessibility/AccessibilityObject.h:190 >> + bool isMeter() const override; > > Can this be final? yes to both of these. >> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75 >> + }); > > This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question? I think that's a great call out! I will audit what text sources we actually need for those.
Joshua Hoffman
Comment 6 2023-11-15 09:07:25 PST
Comment on attachment 468567 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=468567&action=review >> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:72 >> + object->accessibilityText(texts); > > This may be a good opportunity to make AXCoreObject::accessibilityText return a Vector<AccessibilityText>, rather than returning void and taking in a Vector<AccessibilityText>&. Although that's not related to the intention of your patch, so if you wanted to skip it that's OK too. Yeah we should definitely do this. I'll break this into a follow-up patch since it will be a significant change (we'll also want to update titleElementText, alternativeText, visibleText, and helpText to return a vector too). >>> Source/WebCore/accessibility/isolatedtree/mac/AXIsolatedObjectMac.mm:75 >>> + }); >> >> This is an unresearched idea...but is there some text sources we can filter out before caching AccessibilityText to save memory? We should be caching the minimum to correctly serve title, description, and help text. e.g., if an object computes text with AccessibilityTextSource::Subtitle, is that needed in any way to compute any of the three properties in question? > > I think that's a great call out! I will audit what text sources we actually need for those. Unfortunately we use all of the different sources for calculating Title/Description/Help, so we can't do any filtering here.
Joshua Hoffman
Comment 7 2023-11-15 09:26:01 PST
Joshua Hoffman
Comment 8 2023-11-15 15:16:53 PST
Joshua Hoffman
Comment 9 2023-11-15 21:13:43 PST
EWS
Comment 10 2023-11-16 19:03:13 PST
Committed 270860@main (ef43e42c5970): <https://commits.webkit.org/270860@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 468616 [details].
Note You need to log in before you can comment on or make changes to this bug.