Bug 253537 - [buildbot] Move SFTP upload code from generate-bundle script to a new script for uploads and document it
Summary: [buildbot] Move SFTP upload code from generate-bundle script to a new script ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Carlos Alberto Lopez Perez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-03-07 15:07 PST by Carlos Alberto Lopez Perez
Modified: 2023-03-09 10:31 PST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Alberto Lopez Perez 2023-03-07 15:07:53 PST
The current way we are uploading the built-products from the workers to the master is really slow and is causing problems (see bug 252864).

On the GTK/WPE Packaging bots, on the step `generate-bundle` we upload the bundle created via SFTP to the production server. See: https://build.webkit.org/#/builders/30

The SSH server accepting the uploads is configured in a restricted way: the clients connecting to it are only allowed to run the `sftp-internal` command and are restricted inside a chroot so they can only modify their own files.

As an extra security measure the server can be configured to move the files that the bots upload out of that directory as soon as the upload is finished.

After talking with Aakash we agreed that it would be a good idea if I can factorize this code out of this script and make it useful also for uploading the built-products to the post-commit and EWS servers

Let's try it.
Comment 1 Carlos Alberto Lopez Perez 2023-03-07 15:20:07 PST
Pull request: https://github.com/WebKit/WebKit/pull/11193
Comment 2 EWS 2023-03-09 10:30:15 PST
Committed 261448@main (c3827e6cc0dc): <https://commits.webkit.org/261448@main>

Reviewed commits have been landed. Closing PR #11193 and removing active labels.
Comment 3 Radar WebKit Bug Importer 2023-03-09 10:31:17 PST
<rdar://problem/106508921>