Bug 258467

Summary:

Camera RAW files picked via file input are returned as PNG on change

Product:

WebKit

Reporter:

sblochwehbas

Component:

Forms

Assignee:

Aditya Keerthi <akeerthi>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

akeerthi, cdumez, heycam, webkit-bug-importer, wenson_hsieh

Priority:

Keywords:

InRadar

Version:

Safari Technology Preview

Hardware:

Mac (Apple Silicon)

OS:

macOS 13

See Also:

https://bugs.webkit.org/show_bug.cgi?id=260098

Attachments:

Description	Flags
Open HTML file and select a .raf or .cr3 file; the file is provided with png filename	none
Open html file, click file input, select raf or cr3 file. Notice that png file name is printed to screen.	none

Description sblochwehbas 2023-06-23 10:16:40 PDT

Our app accepts a number of file formats, including many RAW image types. The images are accepted via an <input type="file" />.

Currently in Safari 16.5, STP 172 and WK build 265418@main we see that images imported with file extensions .raf or .cr3 are passed to our input's onchange handler as a .png.

Because the file is given to us as a .png we misidentify its content. For .raf files this means that we fail to render it - our WASM backend fails to interpret the data correctly and returns an error. For .cr3 files our wasm backend interprets the file as .png and skips some important steps in the user journey for adding raw files.

The full set of file extensions that we accept currently is as follows:

['psd', 'psb', 'png', 'jpeg', 'jpg', 'tiff', 'tif', 'psdc', 'heic', 'heif', 'webp', 'crw', 'nef', 'raf', 'orf', 'mrw', 'dcr', 'mos', 'raw', 'pef', 'srf', 'dng', 'x3f', 'cr2', 'erf', 'sr2', 'kdc', 'mfw', 'mef', 'arw', 'nrw', 'rw2', 'rwl', 'iiq', '3fr', 'fff', 'srw', 'gpr', 'dxo', 'arq', 'cr3']

I haven't tested each of these exhaustively to check Safari's behaviour but I can do so if necessary - ideally we don't want any conversion performed before we receive the file's name, type and body regardless of its format.

Comment 1 sblochwehbas 2023-06-23 10:47:40 PDT

Created attachment 466816 [details]
Open html file, click file input, select raf or cr3 file. Notice that png file name is printed to screen.

Comment 2 Aditya Keerthi 2023-06-23 12:41:12 PDT

WebKit has logic to automatically transcode images of an unaccepted format, if an accepted format is specified.

In the case of RAW images (.raf), `NSURLFileTypeMappings` does not return a MIME type for the extension, and WebKit incorrect excludes the format from the accepted list. However, the file is recognized by ImageIO as an image type, so WebKit converts it to a PNG.

I also tried using the UniformTypeIdentifiers framework instead of `NSURLFileTypeMappings`, however the extension is still not associated with a valid MIME type.

This is definitely a bug that should be fixed. I think our options are:

1. Reach out to the other frameworks to understand why the type is not recognized. However, even if they add support it wouldn't fix the issue on older OSes.
2. Maintain an additional list of types in WebKit. We sort of already do this for some other formats.
3. Add an exception for transcoding if there is an extension match (don't just rely on MIME types).

(2) is probably the safest approach. (3) might also be nice/correct, but could have some compat fallout. For (1), we should at least learn why the types are missing.

Comment 3 Radar WebKit Bug Importer 2023-06-23 13:23:24 PDT

<rdar://problem/111231838>

Comment 4 Cameron McCormack (:heycam) 2023-06-26 20:11:17 PDT

If we go for (2), we should also confirm that ImageIO recognizes these RAW files on older OS versions.

Comment 5 Aditya Keerthi 2023-07-12 17:08:18 PDT

Pull request: https://github.com/WebKit/WebKit/pull/15801

Comment 6 EWS 2023-07-13 15:04:56 PDT

Committed 266049@main (00c3f53f2de4): <https://commits.webkit.org/266049@main>

Reviewed commits have been landed. Closing PR #15801 and removing active labels.