Bug 256036 - process-css-properties.py fails for some host locales when trying to decode JSON as GBK
Summary: process-css-properties.py fails for some host locales when trying to decode J...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebCore Misc. (show other bugs)
Version: WebKit Local Build
Hardware: PC Windows 10
: P2 Normal
Assignee: Fujii Hironori
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-04-27 03:05 PDT by javalitterboy
Modified: 2023-04-30 13:23 PDT (History)
5 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description javalitterboy 2023-04-27 03:05:19 PDT 
Fix JSON parsing failure during Windows platform compilation.
Windows 10 x64     VS2022

perl Tools/Scripts/build-webkit --wincairo --release --no-ninja --generate-project-only

Generate WebCore.


10>Traceback (most recent call last):
10>  File "E:/GitHub/WebKit/Source/WebCore/css/process-css-properties.py", line 6142, in <module>
10>    main()
10>  File "E:/GitHub/WebKit/Source/WebCore/css/process-css-properties.py", line 6109, in main
10>    properties_json = json.load(properties_file)
10>  File "C:\Users\14183\AppData\Local\Programs\Python\Python38\lib\json\__init__.py", line 293, in load
10>    return loads(fp.read(),
10>UnicodeDecodeError: 'gbk' codec can't decode byte 0xaf in position 400278: illegal multibyte sequence
Comment 1 Alexey Proskuryakov 2023-04-27 17:38:23 PDT 
We should specify an encoding (probably UTF-8 or ASCII) when opening the file, so that nothing locale specific happens during the build.

Regressed with Python rewrite in https://commits.webkit.org/256414@main.
Comment 2 Fujii Hironori 2023-04-27 17:54:43 PDT 
Could you try removing "§" in CSSProperties.json? I think this is the simplest solution if it fixes.
https://github.com/WebKit/WebKit/blob/b1cc64631025e09fd181bffbb11e292604ab5ad1/Source/WebCore/css/CSSProperties.json#L10467 

Other solutions are set env var PYTHONUTF8=1 or "-X utf8" command switch.
Comment 3 Fujii Hironori 2023-04-27 18:19:24 PDT 
I didn't read comment#1. It's better idea. I'm going to create a PR.
Comment 4 Fujii Hironori 2023-04-27 18:22:05 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/13256
Comment 5 Fujii Hironori 2023-04-27 18:23:09 PDT 
Could you test the patch? Do we need more changes?
Comment 6 EWS 2023-04-30 13:22:16 PDT 
Committed 263543@main (aad6f77b647e): <https://commits.webkit.org/263543@main>

Reviewed commits have been landed. Closing PR #13256 and removing active labels.
Comment 7 Radar WebKit Bug Importer 2023-04-30 13:23:18 PDT 
<rdar://problem/108717925>