|
Description
Ben Schwartz
2023-10-02 10:35:07 PDT
Correction: http/tests/appcache/fail-on-update-2.html is timing out on all versions of macOS and iOS, not just iOS17/Sonoma. It appears the reported tests had these expectations set previously: LayoutTests/platform/ios-device/TestExpectations:27:fast/forms/input-text-word-wrap.html [ Failure ] LayoutTests/platform/ios-device/TestExpectations:28:fast/forms/search-rtl.html [ Failure ] LayoutTests/platform/ios-device/TestExpectations:29:fast/forms/search-vertical-alignment.html [ Failure ] LayoutTests/platform/ios-device/TestExpectations:31:fast/forms/visual-hebrew-text-field.html [ Failure ] LayoutTests/platform/ios-simulator-wk2/TestExpectations:169:tables/mozilla/bugs/bug2479-2.html [ Failure Pass ] LayoutTests/platform/ios-simulator/TestExpectations:14:fast/forms/input-text-word-wrap.html [ Skip ] LayoutTests/platform/ios-simulator/TestExpectations:15:fast/forms/search-vertical-alignment.html [ Skip ] LayoutTests/platform/ios-simulator/TestExpectations:17:fast/forms/visual-hebrew-text-field.html [ Skip ] LayoutTests/platform/ios-wk2/TestExpectations:1060:fast/forms/input-appearance-spinbutton.html [ Failure ] LayoutTests/platform/ios-wk2/TestExpectations:1107:webkit.org/b/225665 http/tests/appcache/fail-on-update-2.html [ DumpJSConsoleLogInStdErr Timeout Pass ] LayoutTests/platform/ios-wk2/TestExpectations:2355:webkit.org/b/252866 [ Debug ] fast/text/glyph-display-lists/glyph-display-list-colr-unshared.html [ Pass Failure ] LayoutTests/platform/ios-wk2/TestExpectations:451:http/tests/navigation/javascriptlink-frames.html LayoutTests/platform/ios-wk2/TestExpectations:49:fast/repaint/placeholder-after-caps-lock-hidden.html [ Pass ] LayoutTests/platform/ios-wk2/TestExpectations:842:fast/text/drawBidiText.html [ Failure Pass ] LayoutTests/platform/ios/TestExpectations:1666:fast/forms/input-baseline.html [ Failure ] LayoutTests/platform/ios/TestExpectations:1678:fast/forms/placeholder-position.html [ Failure ] LayoutTests/platform/ios/TestExpectations:1681:fast/forms/search/search-size-with-decorations.html [ Failure ] LayoutTests/platform/ios/TestExpectations:2608:fast/table/003.html [ Skip ] LayoutTests/platform/ios/TestExpectations:3356:http/wpt/mediarecorder/pause-recording.html [ Pass Crash ] LayoutTests/platform/ios/TestExpectations:3400:http/tests/media/video-webm-stall.html [ Skip ] LayoutTests/platform/ios/TestExpectations:419:editing/selection/context-menu-text-selection-lookup.html [ Skip ] LayoutTests/platform/ios/TestExpectations:4527:fast/repaint/placeholder-after-caps-lock-hidden.html [ Timeout ] LayoutTests/platform/ios/TestExpectations:4535:fast/css/rtl-ordering.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4538:fast/forms/basic-inputs.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4539:fast/forms/input-appearance-height.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4540:fast/forms/input-value.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4541:fast/forms/listbox-bidi-align.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4542:fast/forms/placeholder-pseudo-style.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4543:fast/forms/select-visual-hebrew.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4544:fast/forms/time/time-input-rendering-basic.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4564:imported/w3c/web-platform-tests/fetch/range/non-matching-range-response.html [ Failure ] LayoutTests/platform/ios/TestExpectations:4575:tables/mozilla_expected_failures/bugs/bug92647-1.html [ Failure ] LayoutTests/platform/ios/TestExpectations:642:fast/forms/input-appearance-spinbutton-up.html [ Skip ] LayoutTests/platform/ios/TestExpectations:683:fast/forms/search-rtl.html [ Skip ] LayoutTests/platform/ios/TestExpectations:780:fast/text/textIteratorNilRenderer.html [ Skip ] LayoutTests/platform/mac-monterey/TestExpectations:9:http/tests/appcache/fail-on-update-2.html [ DumpJSConsoleLogInStdErr Timeout Failure ] LayoutTests/platform/mac-ventura-wk2-lbse-text/TestExpectations:251:svg/custom/inline-svg-in-xhtml.xml [ Failure ] LayoutTests/platform/mac-ventura-wk2/TestExpectations:24:imported/w3c/web-platform-tests/fetch/range/non-matching-range-response.html [ Pass Failure ] LayoutTests/platform/mac-ventura/TestExpectations:16:fast/forms/auto-fill-button/input-credit-card-auto-fill-button.html [ Failure ] LayoutTests/platform/mac-ventura/TestExpectations:21:http/tests/appcache/fail-on-update-2.html [ DumpJSConsoleLogInStdErr Timeout Failure ] LayoutTests/platform/mac-wk1/TestExpectations:1004:http/tests/media/video-webm-stall.html LayoutTests/platform/mac-wk2/TestExpectations:125:fast/repaint/placeholder-after-caps-lock-hidden.html [ Pass ] LayoutTests/platform/mac-wk2/TestExpectations:1370:webkit.org/b/225665 [ Release arm64 ] http/tests/appcache/fail-on-update-2.html [ DumpJSConsoleLogInStdErr Slow ] LayoutTests/platform/mac-wk2/TestExpectations:1695:[ Monterey+ ] http/tests/media/video-webm-stall.html [ Failure ] LayoutTests/platform/mac-wk2/TestExpectations:1700:[ Monterey Ventura x86_64 ] http/wpt/mediarecorder/pause-recording.html [ Pass Failure ] # change back to [ Monterey+ x86_64 ] when webkit.org/b/262425 is resolved. LayoutTests/platform/mac/TestExpectations:1980:[ BigSur ] http/tests/media/video-webm-stall.html [ Skip ] LayoutTests/platform/mac/TestExpectations:2746:[ Monterey ] fast/block/margin-collapse/103.html [ Pass Failure ImageOnlyFailure ] LayoutTests/platform/mac/TestExpectations:2753:[ Monterey ] imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/mrow-fallback.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:2754:[ Monterey ] imported/w3c/web-platform-tests/mathml/presentation-markup/tables/table-002.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:2755:[ Monterey ] imported/w3c/web-platform-tests/mathml/presentation-markup/tables/table-003.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:2756:[ Monterey ] imported/w3c/web-platform-tests/mathml/relations/css-styling/floats/not-floating-001.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:2757:[ Monterey ] imported/w3c/web-platform-tests/mathml/relations/css-styling/ignored-properties-001.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:2758:[ Monterey ] imported/w3c/web-platform-tests/mathml/relations/css-styling/width-height-001.html [ Pass Failure ] LayoutTests/platform/mac/TestExpectations:3034:http/tests/appcache/fail-on-update-2.html [ Pass Timeout ] LayoutTests/platform/mac/TestExpectations:61:fast/forms/search/search-padding-cancel-results-buttons.html [ Pass ] I will work around these when I file expectations. --- SONOMA UPDATE --- After the changes landed at 268861@main, most of the macOS Sonoma expectations and baselines should be up-to-date. RESOLVED: Below are the tests were left out of the changes in 268861@main due to already being resolved. Bug #262210 (rdar://116139561) -- rebaseline landed in 268612@main: - http/tests/appcache/fail-on-update-2.html Bug #262425 (rdar://116267156) -- fixes landed in 268775@main and 268795@main: - http/wpt/mediarecorder/MediaRecorder-dataavailable.html - http/wpt/mediarecorder/MediaRecorder-multiple-start-stop.html - http/wpt/mediarecorder/MediaRecorder-requestData.html - http/wpt/mediarecorder/pause-recording.html - http/wpt/mediarecorder/record-96KHz-sources.html Bug #262427 (rdar://116268173) -- rebaselines landed in 268823@main: - fast/forms/basic-inputs.html - fast/forms/input-disabled-color.html - fast/forms/number/number-appearance-spinbutton-disabled-readonly.html - fast/forms/placeholder-pseudo-style.html OPEN: Below are the tests that still have outstanding bug reports. Bug #262595 (rdar://116437531): - editing/mac/dictionary-lookup/dictionary-lookup.html - editing/selection/context-menu-text-selection-lookup.html Bug #262604 (rdar://116443613): - http/tests/media/video-webm-stall.html Bug #262663 (rdar://116490690): - imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/failures_X25519.https.any.html - imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/failures_X25519.https.any.worker.html - imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_X25519.https.any.html - imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_X25519.https.any.worker.html - imported/w3c/web-platform-tests/WebCryptoAPI/import_export/okp_importKey.https.any.html - imported/w3c/web-platform-tests/WebCryptoAPI/import_export/okp_importKey.https.any.worker.html - imported/w3c/web-platform-tests/WebCryptoAPI/import_export/okp_importKey_failures_X25519.https.any.html - imported/w3c/web-platform-tests/WebCryptoAPI/import_export/okp_importKey_failures_X25519.https.any.worker.html Important note: in my previous comment, I listed a few bugs that are waiting on resolutions. These tests may require re-baselines, but we will only know after a resolution is reached. Since I upgraded to macOS Sonoma, the following two SVG tests are constant text failures for me: svg/custom/inline-svg-in-xhtml.xml svg/hixie/mixed/003.xml Font related render tree text differences.. I tried to add Sonoma specific results here (https://ews-build.webkit.org/#/builders/50/builds/491), but the "macOS-AppleSilicon-Sonoma-Debug-WK2-Tests-EWS" results are actually identical to the ones obtained with Ventura/Monterey and not different, as on my machine. I was investigating the issue and came across this ticket, where you mentioned also these two tests as failing on Sonoma. Why don't we observe that on the aforementioned bot? I'm slightly confused. |