Bug 262213

Summary:	[commits.webkit.org] Disable crawling on commits.webkit.org
Product:	WebKit	Reporter:	Jonathan Bedard <jbedard>
Component:	Tools / Tests	Assignee:	Jonathan Bedard <jbedard>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Jonathan Bedard 2023-09-27 14:26:04 PDT

There isn't a good reason for spiders to be crawling commits.webkit.org, and those crawlers can cause considerable performance issues. Disable these spiders by default.

Comment 1 Radar WebKit Bug Importer 2023-09-27 14:26:24 PDT

<rdar://problem/116141929>

Comment 2 Jonathan Bedard 2023-09-27 14:28:22 PDT

Pull request: https://github.com/WebKit/WebKit/pull/18305

Comment 3 EWS 2023-09-27 14:48:51 PDT

Committed 268539@main (8a00858352ad): <https://commits.webkit.org/268539@main>

Reviewed commits have been landed. Closing PR #18305 and removing active labels.