Bug 253537

Summary:	[buildbot] Move SFTP upload code from generate-bundle script to a new script for uploads and document it
Product:	WebKit	Reporter:	Carlos Alberto Lopez Perez <clopez>
Component:	Tools / Tests	Assignee:	Carlos Alberto Lopez Perez <clopez>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	aakash_jain, jbedard, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=252864

Description Carlos Alberto Lopez Perez 2023-03-07 15:07:53 PST

The current way we are uploading the built-products from the workers to the master is really slow and is causing problems (see bug 252864).

On the GTK/WPE Packaging bots, on the step `generate-bundle` we upload the bundle created via SFTP to the production server. See: https://build.webkit.org/#/builders/30

The SSH server accepting the uploads is configured in a restricted way: the clients connecting to it are only allowed to run the `sftp-internal` command and are restricted inside a chroot so they can only modify their own files.

As an extra security measure the server can be configured to move the files that the bots upload out of that directory as soon as the upload is finished.

After talking with Aakash we agreed that it would be a good idea if I can factorize this code out of this script and make it useful also for uploading the built-products to the post-commit and EWS servers

Let's try it.

Comment 1 Carlos Alberto Lopez Perez 2023-03-07 15:20:07 PST

Pull request: https://github.com/WebKit/WebKit/pull/11193

Comment 2 EWS 2023-03-09 10:30:15 PST

Committed 261448@main (c3827e6cc0dc): <https://commits.webkit.org/261448@main>

Reviewed commits have been landed. Closing PR #11193 and removing active labels.

Comment 3 Radar WebKit Bug Importer 2023-03-09 10:31:17 PST

<rdar://problem/106508921>