Bug 263527 - REGRESSION(267278@main): Multiple imported audio encoder/decoder layout tests are constantly failing.
Summary: REGRESSION(267278@main): Multiple imported audio encoder/decoder layout tests...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-10-23 09:41 PDT by Ben Schwartz
Modified: 2023-10-23 10:09 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Schwartz 2023-10-23 09:41:11 PDT
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html

This test, previously constantly crashing on wk2 debug and constantly failing on wk2 release, is now constantly failing on all Apple configurations.

HISTORY:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebcodecs%2Faudio-encoder.https.any.html

TEXT DIFF:

+CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code
+CONSOLE MESSAGE: Error: assert_unreached: error: NotSupportedError: Not supported Reached unreachable code
+CONSOLE MESSAGE: NotSupportedError: AudioData creation failed
 
-PASS Simple audio encoding
-PASS Test reset during flush
-PASS Encode audio with negative timestamp
-FAIL Channel number variation: 1 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Channel number variation: 2 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 3000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 13000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 23000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 33000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 43000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 53000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 63000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 73000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 83000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Sample rate variation: 93000 assert_unreached: Should have rejected: undefined Reached unreachable code
-FAIL Encoding and decoding assert_true: expected true got false
-PASS Emit decoder config and extra data.
-PASS encodeQueueSize test
-PASS Test encoding Opus with additional parameters: Empty Opus config
-PASS Test encoding Opus with additional parameters: Opus with frameDuration
-PASS Test encoding Opus with additional parameters: Opus with complexity
-PASS Test encoding Opus with additional parameters: Opus with useinbandfec
-PASS Test encoding Opus with additional parameters: Opus with usedtx
-PASS Test encoding Opus with additional parameters: Opus mixed parameters
+Harness Error (FAIL), message = NotSupportedError: AudioData creation failed
 
+FAIL Simple audio encoding promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed"
+FAIL Test reset during flush assert_unreached: unexpected error Reached unreachable code
+FAIL Encode audio with negative timestamp promise_test: Unhandled rejection with value: object "NotSupportedError: AudioData creation failed"
+

DIFF URL:

https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK2-Tests/269648@main%20(5586)/imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any-pretty-diff.html

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Ventura (13.6, 22G120/arm64) ToT using the following command:

run-webkit-tests --debug --clobber-old-results --verbose imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html

REGRESSION:

I was able to bisect a regression point at 267963@main (issue did not reproduce at 267962@main). Changes at that point are directly related to this issue, and are likely what caused it.
Comment 1 Radar WebKit Bug Importer 2023-10-23 09:41:42 PDT
<rdar://problem/117357248>
Comment 2 Ben Schwartz 2023-10-23 10:07:47 PDT
It appears this issue already has test expectations for multiple failing layout tests with no related bug. I'm going to file a gardening commit that has a link back to this bug for reference.


The following tests are constantly failing (with extremely flaky crashes) on all Apple platforms (regression at 267278@main):

imported/w3c/web-platform-tests/webcodecs/audio-data-serialization.any.html
imported/w3c/web-platform-tests/webcodecs/audio-data.any.html
imported/w3c/web-platform-tests/webcodecs/audio-data.any.worker.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.crossOriginIsolated.https.any.worker.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder-codec-specific.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder.https.any.html
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?adts_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp3
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?mp4_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?opus
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_alaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.html?pcm_mulaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?adts_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp3
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?mp4_aac
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?opus
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_alaw
imported/w3c/web-platform-tests/webcodecs/audioDecoder-codec-specific.https.any.worker.html?pcm_mulaw


The following tests, previously constantly failing on all Apple platforms, regressed at 267403@main and are constantly passing after 267743@main:

imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-decoder.https.any.worker.html


The following tests, previously constantly failing on all Apple platforms, regressed at 267963@main and are constantly passing after 268781@main:

imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.html
imported/w3c/web-platform-tests/webcodecs/audio-encoder-config.https.any.worker.html


The following tests, marked as constant failures, appear never to have been failing:

imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.any.worker.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.html
imported/w3c/web-platform-tests/webcodecs/encoded-audio-chunk.crossOriginIsolated.https.any.worker.html