Bug 256366

Summary:

Safari tab freeze and gpu process crash when calling canvas function drawimage/getimagedata/setimagedata from high resolution getUserMedia stream with several background tabs

Product:

WebKit

Reporter:

Shawn Lee <shawnlee>

Component:

Canvas

Assignee:

youenn fablet <youennf>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

dino, hagaib, jer.noble, kkinnunen, sabouhallawa, shawnlee, simon.fraser, webkit-bug-importer, youennf

Priority:

Keywords:

InRadar

Version:

Safari 16

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=258977

Attachments:

Description	Flags
Screen cast	none
CVPixelBufferPool in memory graph after page is navigated away	none

Shawn Lee

Reported 2023-05-05 06:57:46 PDT

Demo Page: https://3qnl86.csb.app/ Tested On iphone 13/ipad pro 11 2018/iphone 14 pro max. ios 16.4/16.5 Open the page in safari, click the "start" button to start streaming and processing video. Notice that it works at first tab. Click the "open new tab" button. The start the video again on new tab. Repeat the steps few times and eventually the last page will freeze. Open the freezed page with web inspector, you can still run javascript on the freezed page, but the view will not update, even if the elements are removed. This happens on only on high resolution media stream ie 4032x3024. The error log from console app error 21:56:12.028856+0800 MobileSafari 0x10904c380 - GPUProcessProxy::didClose: error 21:56:12.028966+0800 MobileSafari 0x10904c380 - GPUProcessProxy::gpuProcessExited: reason=Crash default 21:56:12.029247+0800 MobileSafari 0x7eec0aa88 - WebProcessPool::gpuProcessDidExit: PID=1220, reason=Crash error 21:56:12.029436+0800 MobileSafari 0x10b001430 - [PID=1217] WebProcessProxy::gpuProcessExited: reason=Crash

Attachments
Screen cast (40.43 MB, video/mp4) 2023-05-07 23:26 PDT, Shawn Lee	no flags	Details
CVPixelBufferPool in memory graph after page is navigated away (115.77 KB, image/png) 2023-08-25 16:32 PDT, Shawn Lee	no flags	Details
View All Add attachment proposed patch, testcase, etc.

Shawn Lee

Comment 1 2023-05-07 23:26:28 PDT

Created attachment 466270 [details] Screen cast screencast

Radar WebKit Bug Importer

Comment 2 2023-05-08 19:48:46 PDT

<rdar://problem/109074255>

Simon Fraser (smfr)

Comment 3 2023-05-08 19:48:52 PDT

Did this happen prior to iOS 16.4?

Shawn Lee

Comment 4 2023-05-08 20:37:48 PDT

Some(In reply to Simon Fraser (smfr) from comment #3) > Did this happen prior to iOS 16.4? The demo is not tested before 16.4 as I can not downgrade the phone. For our website that depends on this operation, it do happens sometimes before but not as often as 16.4 It becomes much easier to trigger the issue after I update he device to 16.4, When there several tabs already open in safari.

Shawn Lee

Comment 5 2023-05-12 06:33:13 PDT

https://websdk.demos.scandit.com/ A real world application where this issue occurred. Select 4k camera on the top menu. It will freeze if you have many tabs in background.

Shawn Lee

Comment 6 2023-07-07 04:49:02 PDT

The issue has become more severe in iOS 17 beta, rendering our website completely unusable.

Kimmo Kinnunen

Comment 7 2023-07-07 05:07:51 PDT

Thank you for the report. I can reproduce. It appears GPU process fails to attribute the images and runs out of memory

Shawn Lee

Comment 8 2023-08-21 19:12:00 PDT

https://lcg3ks.csb.app/ Looks like there is memory leak when drawing getUserMedia video from to canvas or create bitmap from getUserMedia if you do not close the tab. I have create another demo. First open instruments to monitor the memory usage of com.apple.webkit.gpu. Then click "Start drawImage from camera", it will start play video and call drawImage. Click stop to stop video and remove canvas and video frame the page. Click replace location to ensure there are no page caches. Run console.takeHeapSnapshot in inspector to trigger gc. observer that the memory usage of com.apple.webkit.gpu. is increased to 100mb. If you close the tab, the memory is freed. If you have more tabs, the memory get accumulated, and eventually the system kills the gpu process and safari can not recovery for the foreground tab.

Shawn Lee

Comment 9 2023-08-21 19:13:43 PDT

Also notice that he leaks do not happens if the video is comes from file instead of camera. createImageBitmap also trigger the leaks.

Shawn Lee

Comment 10 2023-08-21 23:14:42 PDT

Just tested the leak can be reproduced on desktop safari 16.4 using continuity camera.

Simon Fraser (smfr)

Comment 11 2023-08-22 11:00:31 PDT

Please test iOS 17 beta, where this issue should be resolved. It was fixed via bug 258977. *** This bug has been marked as a duplicate of bug 258977 ***

Shawn Lee

Comment 12 2023-08-22 19:02:03 PDT

I have test with ios 17 beta 7 and latest safari Technology preview 177. The leaks is still exist. I think it is not a duplicate of bug 258977. It seems that the bug 258977 only happens when switch player engine. While my demo only play video from getUserMedia. I have also try to update the demo to not to set url to empty when stop the video. The leak still exists https://3t65yt.csb.app/

Shawn Lee

Comment 13 2023-08-22 19:08:52 PDT

Aloo, when testing with my demo, you can create qrcodes with some query parameters. Like https://3t65yt.csb.app/?1 https://3t65yt.csb.app/?2 https://3t65yt.csb.app/?3 https://3t65yt.csb.app/?4 And open the link from camera app. That can ensure new content process being created for each tab. From my observation, the leak happens once for each content process.

Shawn Lee

Comment 14 2023-08-22 19:21:59 PDT

Also notice that the leak happens only when drawImage is called. Without drawImage the memory of gpu process shrink back to original after video is stopped.

Shawn Lee

Comment 15 2023-08-25 16:32:36 PDT

Created attachment 467443 [details] CVPixelBufferPool in memory graph after page is navigated away The CVPixelBufferPool is retained even after the page is navigated away.

Shawn Lee

Comment 16 2023-08-26 21:45:06 PDT

The CVPixelBufferPool is retained by cached reference of PixelBufferConformerCV, if I set it to nullptr after use, the memory usage of com.apple.webkit.gpu grows much slower. https://github.com/WebKit/WebKit/blob/2a288d2a3ca713875fe7474e8c70f3e304edbd74/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp#L164-L170

youenn fablet

Comment 17 2023-08-28 10:09:03 PDT

<rdar://114570539>

youenn fablet

Comment 18 2023-09-15 01:43:22 PDT

Pull request: https://github.com/apple/WebKit/pull/773

youenn fablet

Comment 19 2023-09-15 01:47:14 PDT

Pull request: https://github.com/WebKit/WebKit/pull/17808

EWS

Comment 20 2023-09-26 02:40:28 PDT

Committed 268444@main (9f4b18808c83): <https://commits.webkit.org/268444@main> Reviewed commits have been landed. Closing PR #17808 and removing active labels.

Note You need to log in before you can comment on or make changes to this bug.