Bug 264816

Summary: [update-test-expectations-from-bugzilla] Handle variants
Product: WebKit Reporter: Sam Sneddon [:gsnedders] <gsnedders>
Component: Tools / TestsAssignee: Sam Sneddon [:gsnedders] <gsnedders>
Status: RESOLVED FIXED    
Severity: Normal CC: jbedard, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Sam Sneddon [:gsnedders] 2023-11-14 09:32:42 PST
Merging https://github.com/WebKit/WebKit/pull/20412 and https://github.com/WebKit/WebKit/pull/18037, and trying to run the script leads to:

% ./Tools/Scripts/update-test-expectations-from-bugzilla -d github-pr 
Updating results from bot mac-wk2 (generic)
mac-wk2 archive: https://ews-build.s3-us-west-2.amazonaws.com/macOS-AppleSilicon-Sonoma-Debug-WK2-Tests-EWS/da9a3797-1341.zip
Traceback (most recent call last):
  File "/Volumes/gsnedders/projects/Safari/OS2/./Tools/Scripts/update-test-expectations-from-bugzilla", line 35, in <module>
    sys.exit(test_expectation_updater.main(sys.argv[1:], sys.stdout, sys.stderr))
  File "/Volumes/gsnedders/projects/Safari/OS2/Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py", line 211, in main
    updater.do_update()
  File "/Volumes/gsnedders/projects/Safari/OS2/Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py", line 171, in do_update
    self._update_for_generic_bot(platform_name)
  File "/Volumes/gsnedders/projects/Safari/OS2/Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py", line 119, in _update_for_generic_bot
    for test_name, expected_content in self._tests_to_update(platform_name).items():
  File "/Volumes/gsnedders/projects/Safari/OS2/Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py", line 113, in _tests_to_update
    return {result: zip_file.read(TestResultWriter.actual_filename(result, self.filesystem)) for result in results_to_update}
  File "/Volumes/gsnedders/projects/Safari/OS2/Tools/Scripts/webkitpy/common/net/bugzilla/test_expectation_updater.py", line 113, in <dictcomp>
    return {result: zip_file.read(TestResultWriter.actual_filename(result, self.filesystem)) for result in results_to_update}
  File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/zipfile.py", line 1463, in read
    with self.open(name, "r", pwd) as fp:
  File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/zipfile.py", line 1502, in open
    zinfo = self.getinfo(name)
  File "/AppleInternal/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/zipfile.py", line 1429, in getinfo
    raise KeyError(
KeyError: "There is no item named 'imported/w3c/web-platform-tests/editing/other/delete-in-child-of-head.tentative-actual.txt' in the archive"

Looking at https://ews-build.s3-us-west-2.amazonaws.com/macOS-AppleSilicon-Sonoma-Debug-WK2-Tests-EWS/da9a3797-1341/results.html shows that `imported/w3c/web-platform-tests/editing/other/delete-in-child-of-head.tentative.html?designMode=on&method=backspace` failed, and a shows a diff with `imported/w3c/web-platform-tests/editing/other/delete-in-child-of-head.tentative_designMode=on&method=backspace-actual.txt`.

This is almost certainly an existing issue, and not directly related to the GitHub support.
Comment 1 Radar WebKit Bug Importer 2023-11-14 09:32:52 PST
<rdar://problem/118402058>
Comment 2 Sam Sneddon [:gsnedders] 2023-11-15 08:17:40 PST
Pull request: https://github.com/WebKit/WebKit/pull/20539
Comment 3 EWS 2023-11-15 09:53:51 PST
Committed 270773@main (03a4b1575bc8): <https://commits.webkit.org/270773@main>

Reviewed commits have been landed. Closing PR #20539 and removing active labels.