Bug 262479

Summary: Picture-in-picture is broken if srcObject is video stream (Safari 17 on macOS 14)
Product: WebKit Reporter: Stefan Benicke <stefan.benicke>
Component: MediaAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, drmoerkerke, jer.noble, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Safari 17   
Hardware: Unspecified   
OS: macOS 14   
Attachments:
Description Flags
Screenshot broken pip on Safari 17 macOS 14
none
HTML/Javascript demo file
none
Screenshot of pip working in Safari 17 on macOS Ventura none

Description Stefan Benicke 2023-10-02 05:36:55 PDT
Created attachment 468021 [details]
Screenshot broken pip on Safari 17 macOS 14

Picture-in-picture on video elements is broken if srcObject is set to a video stream.

What happens is that the video element shows the picture-in-picture display, but no pip-window. Furthermore the Javascript API document.pictureInPictureElement is null and document.exitPictureInPicture() doesn't work and the video element keeps showing the black picture-in-picture state display.

As a quick workaround, I've found out that it works if:

```js
video.controls = true;
video.srcObject = stream;
video.controls = false;
```

Best regards,
Stefan
Comment 1 Stefan Benicke 2023-10-02 05:39:35 PDT
Created attachment 468022 [details]
HTML/Javascript demo file

Use "Start Peer Connection" and then "Start Pip" on one of the videos with the camera stream to see the error.

Reload the page.

Use "Start Peer Connection with video controls" and then "Start Pip" on one of the videos with the camera stream to see it working.
Comment 2 Alexey Proskuryakov 2023-10-03 10:00:55 PDT
Thank you for the report. Could you please clarify if this worked in earlier Safari versions for you? We can also test ourselves, but it's better to have your confirmation too.
Comment 3 Stefan Benicke 2023-10-04 02:00:58 PDT
Created attachment 468059 [details]
Screenshot of pip working in Safari 17 on macOS Ventura

Here's a screenshot to proof that this issue is related to the operating system. In Safari 17 on macOS 13.5.2 (Ventura) the Picture-in-picture is working correctly.
Comment 4 Radar WebKit Bug Importer 2023-10-04 08:23:40 PDT
<rdar://problem/116465668>
Comment 5 youenn fablet 2023-10-16 07:53:27 PDT
Pull request: https://github.com/WebKit/WebKit/pull/19113
Comment 6 EWS 2023-10-17 01:22:12 PDT
Committed 269403@main (36e3488b2736): <https://commits.webkit.org/269403@main>

Reviewed commits have been landed. Closing PR #19113 and removing active labels.
Comment 7 Danny Moerkerke 2023-11-17 03:29:28 PST
This still only works with the workaround from Stefan Benicke in Safari 17.1 and doesn't work at all in Safari on iOS 17.1.1 with a stream as srcObject.
Comment 8 youenn fablet 2023-11-17 03:33:55 PST
Have you tried iOS 17.2 beta?
Comment 9 Danny Moerkerke 2024-01-31 00:20:55 PST
This still doesn't work on iOS 17.3 when the srcObject is a video stream like the webcam. 

It throws this error: NotSupportedError: The video element does not support the Picture-in-Picture mode.

On macOS Sonoma 14.3 starting Picture in Picture works but stopping Picture in Picture  renders a video element with a black picture.
Comment 10 youenn fablet 2024-02-05 00:21:17 PST
(In reply to Danny Moerkerke from comment #9)
> This still doesn't work on iOS 17.3 when the srcObject is a video stream
> like the webcam. 
> 
> It throws this error: NotSupportedError: The video element does not support
> the Picture-in-Picture mode.

This is an expected limitation on iOS right now
I filed https://bugs.webkit.org/show_bug.cgi?id=268732.

> On macOS Sonoma 14.3 starting Picture in Picture works but stopping Picture
> in Picture  renders a video element with a black picture.

I am not exactly sure why it does not work in 14.3.
I tried on Sonoma 14.4 and it works fine for me.
Ditto with Safari Tech Preview.
Can you try either STP or Sonoma latest beta?
Comment 11 Danny Moerkerke 2024-02-05 01:29:21 PST
To reproduce:

- Visit https://whatpwacando.today/media

- click the + in the bottom left corner of the demo
- start web cam
- click PiP button, PiP starts
- exit PiP mode, video is displayed but only part of the top left, seems like a aspect ratio bug
Comment 12 youenn fablet 2024-02-05 08:56:12 PST
(In reply to Danny Moerkerke from comment #11)
> To reproduce:
> 
> - Visit https://whatpwacando.today/media
> 
> - click the + in the bottom left corner of the demo
> - start web cam
> - click PiP button, PiP starts
> - exit PiP mode, video is displayed but only part of the top left, seems
> like a aspect ratio bug

I can reproduce the rendering bug when exiting Pip mode.
The bug gets fixed quickly but the transition is not smooth and should probably be fixed.
Comment 13 youenn fablet 2024-02-05 08:58:44 PST
The rendering bug is similar to rendering bugs triggered by UI side compositing.
I filed https://bugs.webkit.org/show_bug.cgi?id=268750 to keep track of this.