Bug 263527

Summary: REGRESSION(267278@main): Multiple imported audio encoder/decoder layout tests are constantly failing.
Product: WebKit Reporter: Ben Schwartz <ben_schwartz>
Component: MediaAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: mattwoodrow, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=261777
https://bugs.webkit.org/show_bug.cgi?id=261107
https://bugs.webkit.org/show_bug.cgi?id=259874

Description Ben Schwartz 2023-10-23 09:41:11 PDT 
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html

This test, previously constantly crashing on wk2 debug and constantly failing on wk2 release, is now constantly failing on all Apple configurations.

HISTORY:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebcodecs%2Faudio-encoder.https.any.html

TEXT DIFF:

+CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code
+CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code
+CONSOLE MESSAGE: NotSupportedError: AudioData creation failed
 
-PASS Simple audio encoding
-PASS Test reset during flush
-PASS Encode audio with negative timestamp
-FAIL Channel number variation: 1 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Channel number variation: 2 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 3000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 13000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 23000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 33000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 43000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 53000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 63000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 73000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 83000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 93000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Encoding and decoding assert_true: expected true got false
-PASS Emit decoder config and extra data.
-PASS encodeQueueSize test
-PASS Test encoding Opus with additional parameters: Empty Opus config
-PASS Test encoding Opus with additional parameters: Opus with frameDuration
-PASS Test encoding Opus with additional parameters: Opus with complexity
-PASS Test encoding Opus with additional parameters: Opus with useinbandfec
-PASS Test encoding Opus with additional parameters: Opus with usedtx
-PASS Test encoding Opus with additional parameters: Opus mixed parameters
+Harness Error (FAIL), message = NotSupportedError: AudioData creation failed
 
+FAIL Simple audio encoding promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed"
+FAIL Test reset during flush assert_unreached: unexpected error Reached unreachable code
+FAIL Encode audio with negative timestamp promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed"
+

DIFF URL:

https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK2-Tests/269648@main%20(5586)/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-pretty-diff.html

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Ventura (13.6, 22G120/arm64) ToT using the following command:

run-webkit-tests --debug --clobber-old-results --verbose imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html

REGRESSION:

I was able to bisect a regression point at 267963@main (issue did not reproduce at 267962@main). Changes at that point are directly related to this issue, and are likely what caused it.
Comment 1 Radar WebKit Bug Importer 2023-10-23 09:41:42 PDT 
<rdar://problem/117357248>
Comment 2 Ben Schwartz 2023-10-23 10:07:47 PDT 
It appears this issue already has test expectations for multiple failing layout tests with no related bug. I'm going to file a gardening commit that has a link back to this bug for reference.


The following tests are constantly failing (with extremely flaky crashes) on all Apple platforms (regression at 267278@main):

imported/w3c/web-platform-tests/webcodecs/audio-data-serialization.any.html
imported/w3c/web-platform-tests/webcodecs/audio-data.any.html
imported/w3c/web-platform-tests/webcodecs/audio-data.any.worker.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.worker.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?adts_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp3
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp4_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?opus
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_alaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_mulaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?adts_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp3
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp4_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?opus
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_alaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_mulaw


The following tests, previously constantly failing on all Apple platforms, regressed at 267403@main and are constantly passing after 267743@main:

imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.worker.html


The following tests, previously constantly failing on all Apple platforms, regressed at 267963@main and are constantly passing after 268781@main:

imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker.html


The following tests, marked as constant failures, appear never to have been failing:

imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.worker.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.worker.html