RESOLVED FIXED256602
[ iOS ] fast/mediastream/MediaStream-video-element-video-tracks-disabled.html is a flaky ImageOnlyFailure 
https://bugs.webkit.org/show_bug.cgi?id=256602
Summary [ iOS ] fast/mediastream/MediaStream-video-element-video-tracks-disabled.html...
Robert Jenner
Reported 2023-05-10 11:38:01 PDT
fast/mediastream/MediaStream-video-element-video-tracks-disabled.html is a flaky ImageOnlyFailue on iOS 16. HISTORY: https://results.webkit.org/?suite=layout-tests&test=fast%2Fmediastream%2FMediaStream-video-element-video-tracks-disabled.html It appears to be a recent flaky regression. It became a near constant Timeout at 263498@main, and then at 263750@main it stopped timing out, and was just a flaky Image failure, and has continued since. This is also occurring on EWS observed at: https://ews-build.webkit.org/#/builders/15/builds/6238 DIFF URL: https://build.webkit.org/results/Apple-iOS-16-Simulator-Release-WK2-Tests/263908@main%20(2559)/fast/mediastream/MediaStream-video-element-video-tracks-disabled-diffs.html Image Diffs will be attached below
Attachments
Test ImageDiffs (38.87 KB, application/zip)
2023-05-10 11:38 PDT, Robert Jenner 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Robert Jenner
Comment 1 2023-05-10 11:38:40 PDT
Created attachment 466309 [details] Test ImageDiffs
Radar WebKit Bug Importer
Comment 2 2023-05-10 11:44:19 PDT
<rdar://problem/109163885>
Robert Jenner
Comment 3 2023-05-10 11:45:26 PDT
I was able to reproduce this failure by running the test as follows: run-webkit-tests --iOS-simulator fast/mediastream/MediaStream-video-element-video-tracks-disabled.html --iterations 10 The following was the result of the local repo case: failed unexpectedly (reference mismatch maxDifference=255; totalPixels=76800)
Robert Jenner
Comment 4 2023-05-10 11:55:39 PDT
(In reply to Robert Jenner from comment #3) > I was able to reproduce this failure by running the test as follows: > > > run-webkit-tests --iOS-simulator > fast/mediastream/MediaStream-video-element-video-tracks-disabled.html > --iterations 10 > > > The following was the result of the local repo case: > > failed unexpectedly (reference mismatch maxDifference=255; totalPixels=76800) Specifically I reproduced it on iOS16 Release ToT.
EWS
Comment 5 2023-05-10 12:03:22 PDT
Test gardening commit 263916@main (a26726c7688a): <https://commits.webkit.org/263916@main> Reviewed commits have been landed. Closing PR #13708 and removing active labels.
Robert Jenner
Comment 6 2023-05-10 14:04:26 PDT
So this test has some interesting recent developments: The test appears to have been last modified here: https://commits.webkit.org/263497@main That probably caused it to consistently timeout. However, there were signs that it did pass on that test run. It started to consistently timeout at the next test run: https://commits.webkit.org/263498@main It continued to consistently timeout until: https://commits.webkit.org/263750@main At that point the timing out stopped, but it turned into a flaky ImageOnlyFailure at that point.
youenn fablet
Comment 7 2023-05-12 00:03:25 PDT
Pull request: https://github.com/WebKit/WebKit/pull/13794
EWS
Comment 8 2023-05-12 07:49:01 PDT
Committed 264012@main (907131a79e54): <https://commits.webkit.org/264012@main> Reviewed commits have been landed. Closing PR #13794 and removing active labels.
Note You need to log in before you can comment on or make changes to this bug.