Bug 261064

Summary:

Text recognition results appear on wrong position when the parents elements containing <video> element is brought fullscreen

Product:

WebKit

Reporter:

Ethan Wong <bunnnywong>

Component:

Media

Assignee:

Nobody <webkit-unassigned>

Status:

NEW

Severity:

Normal

CC:

webkit-bug-importer, wenson_hsieh

Priority:

Keywords:

InRadar

Version:

Safari 16

Hardware:

Mac (Apple Silicon)

OS:

macOS 13

Attachments:

Description	Flags
A page that produces this issue	none
A screen recording of this issue	none

Ethan Wong

Reported 2023-09-02 09:43:55 PDT

Created attachment 467525 [details] A page that produces this issue Text recognition results may appear on wrong position when the parent element containing <video> elements is brought fullscreen. See the attached example and screen recording for this issue.

Attachments
A page that produces this issue (544 bytes, text/html) 2023-09-02 09:43 PDT, Ethan Wong	no flags	Details
A screen recording of this issue (3.50 MB, video/mp4) 2023-09-02 09:46 PDT, Ethan Wong	no flags	Details
View All Add attachment proposed patch, testcase, etc.

Ethan Wong

Comment 1 2023-09-02 09:46:48 PDT

Created attachment 467526 [details] A screen recording of this issue

Wenson Hsieh

Comment 2 2023-09-03 15:01:18 PDT

Oh, I think the issue is that we don't correctly account for margin when positioning the native VisionKit overlay view, in the case of fullscreen video.

Wenson Hsieh

Comment 3 2023-09-03 16:54:29 PDT

Part of a fix so far involves extracting `inlineVideoFrame()` out of `VideoFullscreenManager.mm`, and using it in `WebPage::beginTextRecognitionForVideoInElementFullScreen` when determining the video rect in root view coordinates. Unfortunately, even after this, the live text button ends up appearing over the built-in video controls... needs further investigation to figure out how to avoid this subsequent issue.

Radar WebKit Bug Importer

Comment 4 2023-09-09 09:44:15 PDT

<rdar://problem/115220914>

Note You need to log in before you can comment on or make changes to this bug.