Bug 253257

Summary:	NEW TEST [ EWS ]: TestWebKitAPI.WKNavigation.HTTPSOnlyWithSameSiteBypass is a constant failure
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	New Bugs	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	m_finkel, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=248968

Description Robert Jenner 2023-03-02 10:46:17 PST

TestWebKitAPI.WKNavigation.HTTPSOnlyWithSameSiteBypass

is a constant failing API-test since it was introduced at: https://commits.webkit.org/261022@main

It is NOT failing on Opensource/ Post-commit seen here:
https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.WKNavigation.HTTPSOnlyWithSameSiteBypass


But it is a constant failure on API-Tests-macOS-EWS seen here:
https://ews-build.webkit.org/#/builders/3/builds/98777

and looking backwards down the queue here:
https://ews-build.webkit.org/#/builders/3

It also appears that when this API-test was introduced that it was landed before it ran tests on EWS seen on the below pull-request, that's why this was not caught:
https://github.com/WebKit/WebKit/pull/10773

Comment 1 Radar WebKit Bug Importer 2023-03-02 10:47:06 PST

<rdar://problem/106152850>

Comment 2 Robert Jenner 2023-03-02 10:48:32 PST

Here is the failure:
    TestWebKitAPI.WKNavigation.HTTPSOnlyWithSameSiteBypass
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1494
        Value of: !(error)
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1530
        Expected equality of these values:
          errorCode
            Which is: -1202
          0
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1531
        Value of: finishedSuccessfully
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1540
        Expected equality of these values:
          @"http://www2.site.example/secure"
            Which is: "http://www2.site.example/secure"
          (NSString *)value
            Which is: "http://site.example/secure"
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1544
        Expected equality of these values:
          @"http://www2.site.example/secure"
            Which is: "http://www2.site.example/secure"
          [webView _mainFrameURL].absoluteString
            Which is: "http://site.example/secure"
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1583
        Expected equality of these values:
          @"http://www2.site.example/secure"
            Which is: "http://www2.site.example/secure"
          (NSString *)value
            Which is: "http://site.example/secure"
        
        
        /Volumes/Data/worker/macOS-BigSur-Release-Build-EWS/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/Navigation.mm:1587
        Expected equality of these values:
          @"http://www2.site.example/secure"
            Which is: "http://www2.site.example/secure"
          [webView _mainFrameURL].absoluteString
            Which is: "http://site.example/secure"

Comment 3 Matthew Finkel 2023-03-02 11:08:39 PST

Pull request: https://github.com/WebKit/WebKit/pull/10952

Comment 4 EWS 2023-03-02 11:30:00 PST

Committed 261086@main (56b772315e29): <https://commits.webkit.org/261086@main>

Reviewed commits have been landed. Closing PR #10952 and removing active labels.