Bug 262213 - [commits.webkit.org] Disable crawling on commits.webkit.org
Summary: [commits.webkit.org] Disable crawling on commits.webkit.org
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-09-27 14:26 PDT by Jonathan Bedard
Modified: 2023-09-27 14:48 PDT (History)
1 user (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Bedard 2023-09-27 14:26:04 PDT 
There isn't a good reason for spiders to be crawling commits.webkit.org, and those crawlers can cause considerable performance issues. Disable these spiders by default.
Comment 1 Radar WebKit Bug Importer 2023-09-27 14:26:24 PDT 
<rdar://problem/116141929>
Comment 2 Jonathan Bedard 2023-09-27 14:28:22 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/18305
Comment 3 EWS 2023-09-27 14:48:51 PDT 
Committed 268539@main (8a00858352ad): <https://commits.webkit.org/268539@main>

Reviewed commits have been landed. Closing PR #18305 and removing active labels.