Bug 259157

Summary: REGRESSION(264419@main): [ iOS ] 264419@Main introduced multiple flaky ImageOnlyFailures
Product: WebKit Reporter: Robert Jenner <jenner>
Component: WebKit Process ModelAssignee: Nobody <webkit-unassigned>
Status: REOPENED ---    
Severity: Normal CC: simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Image Diff files none

Description Robert Jenner 2023-07-12 13:02:49 PDT 
imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed.html

is a flaky ImageOnlyFailure on iOS 16. 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Felements%2Fglobal-attributes%2Fdir_auto-textarea-script-mixed.html

The Diff shows that it appears to off by .01%
https://build.webkit.org/results/Apple-iOS-16-Simulator-Release-GPUProcess-WK2-Tests/265978@main%20(1887)/imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed-diff.png

I have attached the diff files to this bug.
Comment 1 Robert Jenner 2023-07-12 13:03:30 PDT 
Created attachment 467027 [details]
Image Diff files
Comment 2 Radar WebKit Bug Importer 2023-07-12 13:03:55 PDT 
<rdar://problem/112153465>
Comment 3 Robert Jenner 2023-07-12 13:06:31 PDT 
I was able to reproduce the failure at iOS Release ToT running the test as follows:

run-webkit-tests --ios-simulator imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed.html --child-process=1 --iterations 10

It doesn't fail every time, but it does consistently fail running the test as I did above.
Comment 4 Robert Jenner 2023-07-12 13:17:27 PDT 
Even though this is flaky, I was able to bisect to a regression point. 

This test will consistently fail at 264419@main, but won't fail at 264418@main. I ran the test the exact same way, up-to 1000 iterations, and it did not fail once. Where as at and after 264419@main it will fail roughly every 4 out of 10 times.
Comment 5 EWS 2023-07-12 13:30:12 PDT 
Committed 266003@main (1866fb9a9339): <https://commits.webkit.org/266003@main>

Reviewed commits have been landed. Closing PR #15792 and removing active labels.
Comment 6 Robert Jenner 2023-07-12 15:25:48 PDT 
I have found three more tests that become flaky ImageOnly Failures starting at 264419@main. They all passed when being run in 1000 iterations each at 264418@main, but became reproducible flaky failures starting at 264418@main. The additional tests found are:


fast/block/transparent-outline-with-and-without-border-radius.html
fast/forms/basic-textareas-quirks-simple-lines.html
imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-mixed.html

Observed at:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&test=fast%2Fblock%2Ftransparent-outline-with-and-without-border-radius.html&test=fast%2Fforms%2Fbasic-textareas-quirks-simple-lines.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Felements%2Fglobal-attributes%2Fdir_auto-textarea-mixed.html
Comment 7 EWS 2023-07-12 15:47:29 PDT 
Test gardening commit 266014@main (738b85f95555): <https://commits.webkit.org/266014@main>

Reviewed commits have been landed. Closing PR #15799 and removing active labels.
Comment 8 EWS 2023-07-12 17:36:01 PDT 
Test gardening commit 265870.59@safari-7616-branch (1a4523f844fb): <https://commits.webkit.org/265870.59@safari-7616-branch>

Reviewed commits have been landed. Closing PR #697 and removing active labels.
Comment 9 Alexey Proskuryakov 2023-07-18 18:43:26 PDT 
Re-opening, as only TestExpectations were landed.