Bug 263901 - [Cocoa] WebCodecs H264 decoder should reorder frames according presentation time
Summary: [Cocoa] WebCodecs H264 decoder should reorder frames according presentation time
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-10-30 10:15 PDT by youenn fablet
Modified: 2024-02-08 07:37 PST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2023-10-30 10:15:17 PDT
[Cocoa] WebCodecs H264 decoder should reorder frames according presentation time
Comment 1 Radar WebKit Bug Importer 2023-10-30 10:15:56 PDT
<rdar://problem/117692801>
Comment 2 youenn fablet 2023-10-30 10:24:08 PDT
Pull request: https://github.com/WebKit/WebKit/pull/19725
Comment 3 Marcello Bastea-Forte 2023-10-30 11:13:03 PDT
I can reproduce this by modifying the w3c demo project to log timestamps and check for out-of-order:
https://github.com/w3c/webcodecs/pull/734/files

I get out-of-order output callbacks for the test h264 file already in that repo (and a bunch of my own test files).

Repro'ed on macOS 14.0 (23A344):
- Safari Version 17.0 (19616.1.27.211.1)
- Safari Technology Preview Release 181 (Safari 17.4, WebKit 19618.1.3.1)
Comment 4 Marcello Bastea-Forte 2023-10-30 11:17:23 PDT
Full repro steps:
- pull the code:
  - either from https://github.com/marcello3d/webcodecs/tree/marcello/frame-out-of-order-logging
  - apply the patch yourself to https://github.com/w3c/webcodecs https://patch-diff.githubusercontent.com/raw/w3c/webcodecs/pull/734.patch
- run the server with `node server` in the samples folder
- open http://localhost:8888/ in safari
- open developer console to see javascript logs
- leave settings at 2D & h264
- click start

Expected:
- no error logs, everything decodes in ascending order

Actual:
- alternating decode and decode out of order logs:

[Log] Decoded frame 66666 µs (worker.js, line 78)
[Log] Decoded frame 100000 µs (worker.js, line 78)
[Log] Decoded frame 133333 µs (worker.js, line 78)
[Log] Decoded frame 200000 µs (worker.js, line 78)
[Error] Decoded frame 166666 µs out of order, last frame was 200000 µs
	output (worker.js:76)
[Log] Decoded frame 233333 µs (worker.js, line 78)
[Log] Decoded frame 366666 µs (worker.js, line 78)
[Error] Decoded frame 300000 µs out of order, last frame was 366666 µs
	output (worker.js:76)
[Error] Decoded frame 266666 µs out of order, last frame was 300000 µs
	output (worker.js:76)
[Log] Decoded frame 333333 µs (worker.js, line 78)
[Log] Decoded frame 500000 µs (worker.js, line 78)
[Error] Decoded frame 433333 µs out of order, last frame was 500000 µs
	output (worker.js:76)

...etc
Comment 5 Marcello Bastea-Forte 2023-10-30 11:22:13 PDT
I don't know if this should be a separate ticket, but the same issue happens with HEVC/h265 on Safari Technology Preview Release 181 (Safari 17.4, WebKit 19618.1.3.1)

Repro is the same except choose h265 before clicking start. VP8, VP9, and AV1 all run successfully with no out-of-order errors
Comment 6 EWS 2023-11-20 23:52:58 PST
Committed 271004@main (cf345a7967b3): <https://commits.webkit.org/271004@main>

Reviewed commits have been landed. Closing PR #19725 and removing active labels.
Comment 7 Marcello Bastea-Forte 2023-12-13 10:04:05 PST
How do I run the webkit nightly build? 

I just tried Safari Technical Preview Release 184 (Safari 17.4, WebKit 19618.1.7.1.1) and still see the bug, but not sure if this fix is in there?
Comment 8 Marcello Bastea-Forte 2023-12-24 09:52:47 PST
Also still broken on Release 185 (Safari 17.4, WebKit 19618.1.9.8)
Comment 9 Marcello Bastea-Forte 2024-01-16 12:45:05 PST
Also still broken on Safari Technical Preview Release 186 (Safari 17.4, WebKit 19618.1.10.1.1) on macOS Sonoma 14.1.2 (23B92)
Comment 10 Marcello Bastea-Forte 2024-01-30 15:48:01 PST
Still broken on Safari Technical Preview Release 187 (Safari 17.4, WebKit 19619.0.1.2) on Sonoma 14.1.2 (23B92)
Comment 11 youenn fablet 2024-02-08 07:37:48 PST
I filed https://bugs.webkit.org/show_bug.cgi?id=268987 to keep track of this, it seems additional work is needed.