Bug 212260

Summary: requestAnimationFrame conflict with WebGL video playback
Product: WebKit Reporter: Dustin Kerstein <dustin.kerstein>
Component: MediaAssignee: Nobody <webkit-unassigned>
Status: NEW    
Severity: Major CC: dino, jer.noble, justin_fan, kbr, kkinnunen, sabouhallawa, simon.fraser, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Safari 13   
Hardware: All   
OS: All   
Bug Depends on: 205734    
Bug Blocks: 223918    
Attachments:
Description Flags
Replication video none

Dustin Kerstein
Reported 2020-05-22 05:41:59 PDT
Created attachment 400040 [details] Replication video Check out https://jsfiddle.net/9sfr8dj3 using Safari. This one is a little quirky. When the issue happens (about 20% of the time on my 2018 Macbook Pro), texture updates will freeze and the entire tab becomes sluggish (rAF drops to ~5fps on my device). Note that if you are unable to replicate at first using the JSFiddle, just keep hitting the 'Play Again' button until it does replicate. Some additional notes: 1. Happens with both video.src as URL video.src as and Blob (try toggling 'useBlob' in the JSFiddle) 2. Using setInterval/setTimeout avoids this issue (try toggling 'useRAF' in the JSFidddle) 3. Having the <video> tag visible on the page avoids this issue (try toggling 'showBothCanvasAndVideoTag' in the JSFidddle) Let me know if there's any further debug I can get.
Attachments
Replication video (6.76 MB, video/quicktime)
2020-05-22 05:41 PDT, Dustin Kerstein 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Dustin Kerstein
Comment 1 2020-05-22 05:59:38 PDT
FYI, this can also be replicated without Three.js (using WebGL directly) - https://jsfiddle.net/bm67yk9x/
Dustin Kerstein
Comment 2 2020-05-22 07:19:49 PDT
Another note - Sometimes having the Javascript developer console open can make replication harder for some reason, so I'd recommend not opening it for initial replication.
Radar WebKit Bug Importer
Comment 3 2020-05-25 12:15:37 PDT
<rdar://problem/63603790>
Simon Fraser (smfr)
Comment 4 2020-05-25 12:49:36 PDT
I think this is more about media playback.
Kenneth Russell
Comment 5 2020-05-26 10:49:12 PDT
This is reproducible in current Safari 13.1 (15609.1.20.111.8), but appears fixed in Safari Technology Preview Release 106 (Safari 13.2, WebKit 15610.1.12.2). Bug 205734 fixed a problem which happened intermittently in WebGL video-to-texture uploads, and could either be related, or be the root cause of this issue.
Dustin Kerstein
Comment 6 2020-05-26 12:15:28 PDT
Kenneth, I just gave https://jsfiddle.net/9sfr8dj3 a try in release 106 of the Technical Preview and am able to replicate similar, though slightly different, behavior. Instead of the texture updates entirely freezing, the texture updates slow to the pace of which video.currentTime is updating. But when the issue happens (around 1/5 times for me) the playback speed also slows to a crawl (and still nearly freezes the tab). Are you able to replicate that behavior in the Technical Preview?
Kenneth Russell
Comment 7 2020-05-26 12:26:24 PDT
https://jsfiddle.net/9sfr8dj3 , when "Play Again" is clicked rapidly, occasionally demonstrates some slow updating on my machine. The pure WebGL example you provided, https://jsfiddle.net/bm67yk9x/ , works pretty reliably in Safari Technology Preview, even clicking "Play Again" rapidly. I think this may be a library-level problem in Three's VideoTexture implementation. Could you debug into that and see whether it's failing to terminate earlier instances promptly, whether requestAnimationFrame callbacks are piling up somewhere, or similar?
Dustin Kerstein
Comment 8 2020-05-26 12:34:49 PDT
I'm able to replicate the same behavior with the WebGL example + Technical Preview (the newer behavior where the textures update slowly rather than freeze entirely). And here is a slightly modified Three.js example that doesn't use VideoTexture and instead manually marks needsUpdate in the render loop - https://jsfiddle.net/hxvsjnfm - Both examples replicate for me in the Technical Preview.
Kenneth Russell
Comment 9 2020-05-26 12:58:42 PDT
Let's please focus on the test cases that don't involve Three.js. Any browser engineering team needs the smallest possible test, and Three brings in a lot of additional code. https://jsfiddle.net/bm67yk9x/ , when "Play Again" is clicked rapidly before waiting to see the rendered result, does occasionally demonstrate the slow updating and apparent high CPU consumption in Safari Technology Preview - confirmed.
Dustin Kerstein
Comment 10 2020-05-26 13:24:41 PDT
K, thanks for the feedback. Let me know if there's anything else I can get you. FYI, I can also replicate on the WebGL example even when waiting for the video to play several loops (and then hitting Play Again). Ie. I don't have to rapidly click Play Again before it starts displaying frames to replicate.
Note You need to log in before you can comment on or make changes to this bug.