imported/w3c/web-platform-tests/html/dom/idlharness.https.html is a near constant failure. It is failing on iOS 16 and is a flaky time out / failure on macOS Debug. HISTORY: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Fidlharness.https.html%3Fexclude%3D(Document%7CWindow%7CHTML.*) DIFF TEXT: @@ -1275,9 +1275,9 @@ PASS Worklet interface object length PASS Worklet interface object name PASS Worklet interface: existence and properties of interface prototype object -FAIL Worklet interface: existence and properties of interface prototype object's "constructor" property assert_own_property: Worklet.prototype does not have own property "constructor" expected property "constructor" missing +PASS Worklet interface: existence and properties of interface prototype object's "constructor" property PASS Worklet interface: existence and properties of interface prototype object's @@unscopables property -FAIL Worklet interface: operation addModule(USVString, optional WorkletOptions) assert_own_property: interface prototype object missing non-static operation expected property "addModule" missing +PASS Worklet interface: operation addModule(USVString, optional WorkletOptions) PASS Storage interface: existence and properties of interface object PASS Storage interface object length PASS Storage interface object name
I was able to replicate this failure on iOS 16 ToT as follows: run-webkit-tests --iOS-simulator imported/w3c/web-platform-tests/html/dom/idlharness.https.html
<rdar://problem/107393338>
There are many problems with this test, but AFAICT the diff posted above just means that iOS has incorrect expectations for it. Other problems that I see is that the test is super slow in debug, frequently timing out, and diffs of at least one more kind: -PASS MediaError must be primary interface of errorVideo.error -PASS Stringification of errorVideo.error -PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_ABORTED" with the proper type -PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_NETWORK" with the proper type -PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_DECODE" with the proper type -PASS MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_SRC_NOT_SUPPORTED" with the proper type -PASS MediaError interface: errorVideo.error must inherit property "code" with the proper type -PASS MediaError interface: errorVideo.error must inherit property "message" with the proper type +FAIL Stringification of errorVideo.error assert_class_string: class string of errorVideo.error expected "[object MediaError]" but got "[object Null]" +FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_ABORTED" with the proper type assert_inherits: provided value is not an object +FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_NETWORK" with the proper type assert_inherits: provided value is not an object +FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_DECODE" with the proper type assert_inherits: provided value is not an object +FAIL MediaError interface: errorVideo.error must inherit property "MEDIA_ERR_SRC_NOT_SUPPORTED" with the proper type assert_inherits: provided value is not an object +FAIL MediaError interface: errorVideo.error must inherit property "code" with the proper type assert_inherits: provided value is not an object +FAIL MediaError interface: errorVideo.error must inherit property "message" with the proper type assert_inherits: provided value is not an object
Test gardening commit 262595@main (4798b5317d9d): <https://commits.webkit.org/262595@main> Reviewed commits have been landed. Closing PR #12396 and removing active labels.
Test gardening commit 259548.602@safari-7615-branch (1fd08d3625dd): <https://commits.webkit.org/259548.602@safari-7615-branch> Reviewed commits have been landed. Closing PR #537 and removing active labels.
Looking at the test results, on iOS it might require a specific expectation for iPad vs. iPhone due to MSE support. For Debug in general, idlharness test should be marked as slow.
Pull request: https://github.com/WebKit/WebKit/pull/19164
Committed 269461@main (2e494e5298f7): <https://commits.webkit.org/269461@main> Reviewed commits have been landed. Closing PR #19164 and removing active labels.