Bug 256602

Summary:

[ iOS ] fast/mediastream/MediaStream-video-element-video-tracks-disabled.html is a flaky ImageOnlyFailure

Product:

WebKit

Reporter:

Robert Jenner <jenner>

Component:

Media

Assignee:

youenn fablet <youennf>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

eric.carlson, jer.noble, philn, webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Attachments:

Description	Flags
Test ImageDiffs	none

Description Robert Jenner 2023-05-10 11:38:01 PDT

fast/mediastream/MediaStream-video-element-video-tracks-disabled.html

is a flaky ImageOnlyFailue on iOS 16. 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fmediastream%2FMediaStream-video-element-video-tracks-disabled.html

It appears to be a recent flaky regression. It became a near constant Timeout at 263498@main, and then at 263750@main it stopped timing out, and was just a flaky Image failure, and has continued since. This is also occurring on EWS observed at:

https://ews-build.webkit.org/#/builders/15/builds/6238


DIFF URL:
https://build.webkit.org/results/Apple-iOS-16-Simulator-Release-WK2-Tests/263908@main%20(2559)/fast/mediastream/MediaStream-video-element-video-tracks-disabled-diffs.html

Image Diffs will be attached below

Comment 1 Robert Jenner 2023-05-10 11:38:40 PDT

Created attachment 466309 [details]
Test ImageDiffs

Comment 2 Radar WebKit Bug Importer 2023-05-10 11:44:19 PDT

<rdar://problem/109163885>

Comment 3 Robert Jenner 2023-05-10 11:45:26 PDT

I was able to reproduce this failure by running the test as follows:


run-webkit-tests --iOS-simulator 
fast/mediastream/MediaStream-video-element-video-tracks-disabled.html --iterations 10


The following was the result of the local repo case: 

failed unexpectedly (reference mismatch maxDifference=255; totalPixels=76800)

Comment 4 Robert Jenner 2023-05-10 11:55:39 PDT

(In reply to Robert Jenner from comment #3)
> I was able to reproduce this failure by running the test as follows:
> 
> 
> run-webkit-tests --iOS-simulator 
> fast/mediastream/MediaStream-video-element-video-tracks-disabled.html
> --iterations 10
> 
> 
> The following was the result of the local repo case: 
> 
> failed unexpectedly (reference mismatch maxDifference=255; totalPixels=76800)

Specifically I reproduced it on iOS16 Release ToT.

Comment 5 EWS 2023-05-10 12:03:22 PDT

Test gardening commit 263916@main (a26726c7688a): <https://commits.webkit.org/263916@main>

Reviewed commits have been landed. Closing PR #13708 and removing active labels.

Comment 6 Robert Jenner 2023-05-10 14:04:26 PDT

So this test has some interesting recent developments: 


The test appears to have been last modified here:
https://commits.webkit.org/263497@main

That probably caused it to consistently timeout. However, there were signs that it did pass on that test run. It started to consistently timeout at the next test run:
https://commits.webkit.org/263498@main

It continued to consistently timeout until:
https://commits.webkit.org/263750@main


At that point the timing out stopped, but it turned into a flaky ImageOnlyFailure at that point.

Comment 7 youenn fablet 2023-05-12 00:03:25 PDT

Pull request: https://github.com/WebKit/WebKit/pull/13794

Comment 8 EWS 2023-05-12 07:49:01 PDT

Committed 264012@main (907131a79e54): <https://commits.webkit.org/264012@main>

Reviewed commits have been landed. Closing PR #13794 and removing active labels.