Bug 256629 - AX: Use AXDidSpellCheck to allow assistive technologies to lazily resolve spell-checking
Summary: AX: Use AXDidSpellCheck to allow assistive technologies to lazily resolve spe...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Accessibility (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tyler Wilcock
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-05-10 23:11 PDT by Tyler Wilcock
Modified: 2023-05-17 19:00 PDT (History)
10 users (show)

See Also:


Attachments
Patch (30.35 KB, patch)
2023-05-11 09:53 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff
Patch (31.27 KB, patch)
2023-05-11 10:24 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff
Patch (32.34 KB, patch)
2023-05-11 12:36 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff
Patch (19.91 KB, patch)
2023-05-16 11:05 PDT, Tyler Wilcock
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (21.28 KB, patch)
2023-05-16 11:25 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff
Patch (21.35 KB, patch)
2023-05-16 11:38 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff
Patch (22.14 KB, patch)
2023-05-16 13:43 PDT, Tyler Wilcock
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tyler Wilcock 2023-05-10 23:11:58 PDT
Eager spell-checking can cause performance issues.
Comment 1 Radar WebKit Bug Importer 2023-05-10 23:12:10 PDT
<rdar://problem/109191055>
Comment 2 Tyler Wilcock 2023-05-11 09:53:24 PDT
Created attachment 466319 [details]
Patch
Comment 3 Tyler Wilcock 2023-05-11 10:24:52 PDT
Created attachment 466320 [details]
Patch
Comment 4 chris fleizach 2023-05-11 11:09:12 PDT
Comment on attachment 466320 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=466320&action=review

> Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:2105
> +        if (didSpellCheck) {

we should probably check that this is [didSpellCheck boolValue] ?

> LayoutTests/accessibility/mac/attributed-string/attributed-string-does-not-includes-misspelled-for-non-editable-expected.txt:10
> +PASS attrString.indexOf('AXDidSpellCheck = 0') != -1 is true

do we still want to have a test that does perform spell checking?
Comment 5 Tyler Wilcock 2023-05-11 12:13:51 PDT
(In reply to chris fleizach from comment #4)
> Comment on attachment 466320 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=466320&action=review
> 
> > Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:2105
> > +        if (didSpellCheck) {
> 
> we should probably check that this is [didSpellCheck boolValue] ?
There are two steps here:

  1. Does objectForKey:@"AXDidSpellCheck" return non-nil, indicating that attribute is present?
  2. If non-nil, what is the boolValue?

The `if (didSpellCheck)` you highlighted performs the first step. If that evaluates to true, it means the property is set, and the debug output in the next line does the boolValue.

Checking BOOL misspelled in the lines above follows a similar pattern.

> > LayoutTests/accessibility/mac/attributed-string/attributed-string-does-not-includes-misspelled-for-non-editable-expected.txt:10
> > +PASS attrString.indexOf('AXDidSpellCheck = 0') != -1 is true
> 
> do we still want to have a test that does perform spell checking?
I think probably not, because it means we'd have to keep eager spell-checking codepaths around simply for the sake of testing them even though we won't actually behave that way when queried by AX clients. I'm open to discussion though if you think there's a good reason to keep such tests around. We could always bring them back if we decide we want to eagerly spell-check sometimes.
Comment 6 Tyler Wilcock 2023-05-11 12:36:14 PDT
Created attachment 466322 [details]
Patch
Comment 7 chris fleizach 2023-05-11 12:38:14 PDT
(In reply to Tyler Wilcock from comment #5)
> (In reply to chris fleizach from comment #4)
> > Comment on attachment 466320 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=466320&action=review
> > 
> > > Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:2105
> > > +        if (didSpellCheck) {
> > 
> > we should probably check that this is [didSpellCheck boolValue] ?
> There are two steps here:
> 
>   1. Does objectForKey:@"AXDidSpellCheck" return non-nil, indicating that
> attribute is present?
>   2. If non-nil, what is the boolValue?
> 
> The `if (didSpellCheck)` you highlighted performs the first step. If that
> evaluates to true, it means the property is set, and the debug output in the
> next line does the boolValue.
> 
> Checking BOOL misspelled in the lines above follows a similar pattern.
> 
> > > LayoutTests/accessibility/mac/attributed-string/attributed-string-does-not-includes-misspelled-for-non-editable-expected.txt:10
> > > +PASS attrString.indexOf('AXDidSpellCheck = 0') != -1 is true
> > 
> > do we still want to have a test that does perform spell checking?
> I think probably not, because it means we'd have to keep eager
> spell-checking codepaths around simply for the sake of testing them even
> though we won't actually behave that way when queried by AX clients. I'm
> open to discussion though if you think there's a good reason to keep such
> tests around. We could always bring them back if we decide we want to
> eagerly spell-check sometimes.



We have other clients besides VO though and they won't likely update to handle this so I feel we need to support both paths. We only know this will work in VO
Comment 8 Andres Gonzalez 2023-05-11 18:27:14 PDT
(In reply to chris fleizach from comment #7)
> (In reply to Tyler Wilcock from comment #5)
> > (In reply to chris fleizach from comment #4)
> > > Comment on attachment 466320 [details]
> > > Patch
> > > 
> > > View in context:
> > > https://bugs.webkit.org/attachment.cgi?id=466320&action=review
> > > 
> > > > Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:2105
> > > > +        if (didSpellCheck) {
> > > 
> > > we should probably check that this is [didSpellCheck boolValue] ?
> > There are two steps here:
> > 
> >   1. Does objectForKey:@"AXDidSpellCheck" return non-nil, indicating that
> > attribute is present?
> >   2. If non-nil, what is the boolValue?
> > 
> > The `if (didSpellCheck)` you highlighted performs the first step. If that
> > evaluates to true, it means the property is set, and the debug output in the
> > next line does the boolValue.
> > 
> > Checking BOOL misspelled in the lines above follows a similar pattern.
> > 
> > > > LayoutTests/accessibility/mac/attributed-string/attributed-string-does-not-includes-misspelled-for-non-editable-expected.txt:10
> > > > +PASS attrString.indexOf('AXDidSpellCheck = 0') != -1 is true
> > > 
> > > do we still want to have a test that does perform spell checking?
> > I think probably not, because it means we'd have to keep eager
> > spell-checking codepaths around simply for the sake of testing them even
> > though we won't actually behave that way when queried by AX clients. I'm
> > open to discussion though if you think there's a good reason to keep such
> > tests around. We could always bring them back if we decide we want to
> > eagerly spell-check sometimes.
> 
> 
> 
> We have other clients besides VO though and they won't likely update to
> handle this so I feel we need to support both paths. We only know this will
> work in VO

Here is a proposal. The Mac API already has to entry points: AXAttributedStringForTextMarkerRangeAttribute and AXAttributedStringForTextMarkerRangeWithOptionsAttribute. In ITM we are always doing eager caching of spelling and removing it if the client doesn't want it through the latter API. What we need to do instead is:
1. Always cache without spelling.
2. Only do the spelling if requested through AXAttributedStringForTextMarkerRangeWithOptionsAttribute. In this case, hit the main thread and be slow.
3. Make sure that VO never uses AXAttributedStringForTextMarkerRangeWithOptionsAttribute to request for spelling.


Here is a proposal: The Mac API already has to entry points: AXAttributedStringForTextMarkerRangeAttribute and AXAttributedStringForTextMarkerRangeWithOptionsAttribute. In ITM we were always doing eager caching of the spelling.
Comment 9 Tyler Wilcock 2023-05-16 11:05:28 PDT
Created attachment 466365 [details]
Patch
Comment 10 Tyler Wilcock 2023-05-16 11:25:33 PDT
Created attachment 466366 [details]
Patch
Comment 11 chris fleizach 2023-05-16 11:29:35 PDT
Comment on attachment 466365 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=466365&action=review

> Source/WebCore/accessibility/mac/AXObjectCacheMac.mm:658
> +bool AXObjectCache::shouldSpellCheckEagerly()

this feels like it should be the default mode.. aka it should just be called

shouldSpellCheck()

with the assumption that means spell checking will happen

the opposite would then be

"deferSpellCheckingToClient()"
Comment 12 Tyler Wilcock 2023-05-16 11:38:43 PDT
Created attachment 466367 [details]
Patch
Comment 13 Tyler Wilcock 2023-05-16 11:39:11 PDT
(In reply to chris fleizach from comment #11)
> Comment on attachment 466365 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=466365&action=review
> 
> > Source/WebCore/accessibility/mac/AXObjectCacheMac.mm:658
> > +bool AXObjectCache::shouldSpellCheckEagerly()
> 
> this feels like it should be the default mode.. aka it should just be called
> 
> shouldSpellCheck()
> 
> with the assumption that means spell checking will happen
> 
> the opposite would then be
> 
> "deferSpellCheckingToClient()"
You're right, that naming is better. Updated in the latest patch.
Comment 14 Tyler Wilcock 2023-05-16 13:43:10 PDT
Created attachment 466371 [details]
Patch
Comment 15 EWS 2023-05-17 19:00:03 PDT
Committed 264188@main (017e7fd5e52a): <https://commits.webkit.org/264188@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 466371 [details].