Bug 262595 - REGRESSION(Sonoma): [ Sonoma ] 2 lookup menu-related tests are constant text failures.
Summary: REGRESSION(Sonoma): [ Sonoma ] 2 lookup menu-related tests are constant text ...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-10-03 18:11 PDT by Ben Schwartz
Modified: 2024-07-02 14:58 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Schwartz 2023-10-03 18:11:28 PDT
The following tests are constantly failing with text diffs on macOS Sonoma:

editing/mac/dictionary-lookup/dictionary-lookup.html
editing/selection/context-menu-text-selection-lookup.html

HISTORY:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=editing%2Fmac%2Fdictionary-lookup%2Fdictionary-lookup.html&test=editing%2Fselection%2Fcontext-menu-text-selection-lookup.html

TEXT DIFFs:


 Some text to look up.
 
 Lookup string for normal text: 'look'.
-Lookup string for title-case phrase: 'New York'.
+Lookup string for title-case phrase: 'New'.
 Lookup string for unselectable text: 'null'.
_____

 This test checks that triggering the context menu selects/doesn't select using surround text as context (as platform-appropriate).
 
 New York
+FAILURE: We expected New York, but we got New.
 SUCCESS
 SUCCESS
-SUCCESS


DIFF URLs:

https://build.webkit.org/results/Apple-Sonoma-Release-WK1-Tests/268810@main%20(91)/editing/mac/dictionary-lookup/dictionary-lookup-pretty-diff.html
https://build.webkit.org/results/Apple-Sonoma-Release-WK1-Tests/268810@main%20(91)/editing/selection/context-menu-text-selection-lookup-pretty-diff.html

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Sonoma (14.0, 23A344/arm64) ToT using the following command:

run-webkit-tests --verbose --clobber-old-results editing/mac/dictionary-lookup/dictionary-lookup.html editing/selection/context-menu-text-selection-lookup.html

REGRESSION:

This test regressed with the bring-up of Sonoma testers (see bug #262489).

NOTES:

This seems to be an issue with how tests on Sonoma are run when the menus are open, which could be an OS regression.
Comment 1 Radar WebKit Bug Importer 2023-10-03 18:11:52 PDT
<rdar://problem/116437531>
Comment 2 EWS 2023-10-03 19:36:21 PDT
Test gardening commit 268820@main (f399a5ee439d): <https://commits.webkit.org/268820@main>

Reviewed commits have been landed. Closing PR #18602 and removing active labels.
Comment 3 EWS 2024-07-02 13:26:31 PDT
Test gardening commit 280590@main (e604d0614fb7): <https://commits.webkit.org/280590@main>

Reviewed commits have been landed. Closing PR #30403 and removing active labels.
Comment 4 Robert Jenner 2024-07-02 14:58:59 PDT
(In reply to EWS from comment #3)
> Test gardening commit 280590@main (e604d0614fb7):
> <https://commits.webkit.org/280590@main>
> 
> Reviewed commits have been landed. Closing PR #30403 and removing active
> labels.

I modified the existing expectations because these tests haven't been fixed, and it appears to impact both Sonoma and Sequoia.