Bug 255631

Summary:	REGRESSION(262632@main?): [ iOS ] 2X imported/w3c/web-platform-tests/screen-orientation (layout-tests) are consistent text failures
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	Media	Assignee:	Marcos Caceres <marcosc>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	dino, marcosc, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=255931

Description Robert Jenner 2023-04-18 16:36:34 PDT

imported/w3c/web-platform-tests/screen-orientation/nested-documents.html
imported/w3c/web-platform-tests/screen-orientation/unlock.html

are both consistent text failures on iOS. 


Note, they are not constant failures, but they appear to fail more than they pass. Which is why I'm not off the bat calling these flaky. 


HISTORY:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fscreen-orientation%2Fnested-documents.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fscreen-orientation%2Funlock.html

These tests both appear to have the same history as well. They started failing at the same. 



DIFF URLS:
https://build.webkit.org/results/Apple-iOS-16-Simulator-Debug-WK2-Tests/263081@main%20(966)/imported/w3c/web-platform-tests/screen-orientation/nested-documents-pretty-diff.html
https://build.webkit.org/results/Apple-iOS-16-Simulator-Debug-WK2-Tests/263081@main%20(966)/imported/w3c/web-platform-tests/screen-orientation/unlock-pretty-diff.html

TEXT DIFF for "imported/w3c/web-platform-tests/screen-orientation/nested-documents.html":

+CONSOLE MESSAGE: Unhandled Promise Rejection: SecurityError: Locking the screen orientation is only allowed when in fullscreen
 
 
 Harness Error (FAIL), message = Test named 'Requesting orientation lock from one document cancels the lock request from another document' specified 1 'cleanup' function, and 1 failed.
 
-FAIL Requesting orientation lock from one document cancels the lock request from another document promise_test: Unhandled rejection with value: object "TypeError: Type error"
+FAIL Requesting orientation lock from one document cancels the lock request from another document assert_unreached: Should have rejected: undefined Reached unreachable code
 NOTRUN The orientation lock from one document affects lock requests from other documents




TEXT DIFF for "imported/w3c/web-platform-tests/screen-orientation/unlock.html":
-Harness Error (FAIL), message = Test named 'unlock() aborts a pending lock request across documents' specified 1 'cleanup' function, and 1 failed.
 
 PASS unlock() doesn't throw when there is no lock
 PASS unlock() returns a void value
 PASS unlock() doesn't throw when there is no lock with fullscreen
 PASS unlock() aborts a pending lock request
-FAIL unlock() aborts a pending lock request across documents promise_test: Unhandled rejection with value: object "TypeError: Type error"
+FAIL unlock() aborts a pending lock request across documents promise_rejects_dom: function "function () { throw e }" threw object "SecurityError: Locking the screen orientation is only allowed when in fullscreen" that is not a DOMException AbortError: property "code" is equal to 18, expected 20

Comment 1 Radar WebKit Bug Importer 2023-04-18 16:37:03 PDT

<rdar://problem/108231120>

Comment 2 Robert Jenner 2023-04-18 18:08:02 PDT

I was able to reproduce these failures at iOS 16 Release ToT running as follows:

run-webkit-tests --iOS-simulator <test_goes_here> --iterations 10


It will reproduce almost every single time. Adding 10 iterations guarantees successful reproduction every time.

Comment 3 Robert Jenner 2023-04-18 18:30:22 PDT

I'm uncertain what the regression point is for this particular bug. 


It appears that these tests were mostly skipped prior to 262632@main because the history indicates that's the first time that these tests ran completely on all iOS queues for quite some time. Interestingly it was ONLY running on iOS 16 Simulator wk2 Release on iPhone 12 with x86_64 for sometime, and it was flaky failing. But it was less consistent. 


They stopped running on that queue, and started running on all other iOS queues on 262632@main. However, I don't see any test expectations added or removed at that commit, or any of the 15 subsequent commits before it. So It's a mystery to me. In any case I've marked expectations for these tests below.

Comment 4 EWS 2023-04-18 18:41:54 PDT

Test gardening commit 263112@main (d289e0001d2a): <https://commits.webkit.org/263112@main>

Reviewed commits have been landed. Closing PR #12897 and removing active labels.

Comment 5 Marcos Caceres 2023-04-19 06:43:36 PDT

Pull request: https://github.com/WebKit/WebKit/pull/12909

Comment 6 EWS 2023-04-21 01:58:14 PDT

Committed 263224@main (40235be133be): <https://commits.webkit.org/263224@main>

Reviewed commits have been landed. Closing PR #12909 and removing active labels.