XCTest-Gherkin: UI test crashing on 11.4+ when updating from 11.1

258 views Asked by At

I'm trying to update a project from Xcode 11.1 to 11.6. When a test step fails, instead of reporting a failure, an exception is thrown (see below).

There are changes coming to test error reporting with Xcode 12, but that shouldn't affect 11.6. The issue could either be in XCTest-Gherkin or a "bug" in XCTest itself. Not sure if this exception is new, so it might be just not handled somewhere.

Stack trace:

  *** Terminating app due to uncaught exception '_XCTestCaseInterruptionException', reason: 'Interrupting test'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff23e3de6e __exceptionPreprocess + 350
    1   libobjc.A.dylib                     0x00007fff512539b2 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff23e3dcac +[NSException raise:format:] + 188
    3   XCTest                              0x000000010f011362 -[XCTestCase(Failures) _recordFailureWithDescription:inFile:atLine:expected:] + 487
    4   XCTest                              0x000000010efb60ed -[XCTestCase recordFailureWithDescription:inFile:atLine:expected:] + 66
    5   XCTest_Gherkin                      0x0000000113e3a2d1 $s14XCTest_Gherkin0B5StateC8testCase_22didFailWithDescription6inFile6atLineySo0aE0C_S2SSgSitF + 1313
    6   XCTest_Gherkin                      0x0000000113e3a5dd $s14XCTest_Gherkin0B5StateC8testCase_22didFailWithDescription6inFile6atLineySo0aE0C_S2SSgSitFTo + 285
    7   XCTest                              0x000000010efce9a0 -[XCTestObservationCenter _testCaseDidFail:withDescription:inFile:atLine:] + 578
    8   XCTest                              0x000000010efab10c -[XCTestCaseRun recordFailureWithDescription:inFile:atLine:expected:] + 161
    9   XCTest                              0x000000010f01123a -[XCTestCase(Failures) _recordFailureWithDescription:inFile:atLine:expected:] + 191
    10  XCTest                              0x000000010efb60ed -[XCTestCase recordFailureWithDescription:inFile:atLine:expected:] + 66
    11  XCTest                              0x000000010f01093b -[XCTestCase(Failures) _dequeueFailures] + 859
    12  XCTest                              0x000000010f010dfb -[XCTestCase(Failures) _enqueueFailureWithDescription:inFile:atLine:expected:interruptTest:breakWhenDequeued:] + 803
    13  XCTest                              0x000000010efc504c _XCTFailureHandler + 516
    14  XCTest                              0x000000010efc55e7 _XCTPreformattedFailureHandler + 155
    15  libXCTestSwiftSupport.dylib         0x000000011e81d56b $s6XCTest13XCTAssertTrue__4file4lineySbyKXK_SSyXKs12StaticStringVSutF + 1179
    16  App-UITests-Runner                  0x0000000111227547 $s13appUITest28ScreenStepsC06defineF0yyFySicfU4_ + 199
    17  App-UITests-Runner                  0x000000011122767f $sSiIegy_SiIegn_TR + 15
    18  XCTest_Gherkin                      0x0000000113e30efb $s14XCTest_Gherkin11StepDefinerC4step_4file4line1fySS_SSSiyxctAA26MatchedStringRepresentableRzlFyAA0C7MatchesVySSGcfU_ + 1003
    19  XCTest_Gherkin                      0x0000000113e30fcc $s14XCTest_Gherkin11StepDefinerC4step_4file4line1fySS_SSSiyxctAA26MatchedStringRepresentableRzlFyAA0C7MatchesVySSGcfU_TA + 60
    20  XCTest_Gherkin                      0x0000000113e42914 $sSo10XCTestCaseC0A8_GherkinE11performStep_7keyword4file4lineySS_S2SSitFySo11XCTActivity_pXEfU0_ + 900
    21  XCTest_Gherkin                      0x0000000113e42b43 $sSo10XCTestCaseC0A8_GherkinE11performStep_7keyword4file4lineySS_S2SSitFySo11XCTActivity_pXEfU0_TA + 51
    22  XCTest_Gherkin                      0x0000000113e42b5f $sSo11XCTActivity_ps5Error_pIggzo_SoAA_pytsAB_pIeggrzo_TR + 15
    23  XCTest_Gherkin                      0x0000000113e42bb4 $sSo11XCTActivity_ps5Error_pIggzo_SoAA_pytsAB_pIeggrzo_TRTA + 20
    24  libXCTestSwiftSupport.dylib         0x000000011e819e48 $sSo10XCTContextC6XCTestE11runActivity5named5blockxSS_xSo11XCTActivity_pKXEtKlFZxyKXEfU_ + 152
    25  libXCTestSwiftSupport.dylib         0x000000011e819ea1 $sSo10XCTContextC6XCTestE11runActivity5named5blockxSS_xSo11XCTActivity_pKXEtKlFZxyKXEfU_TA + 33
    26  libswiftObjectiveC.dylib            0x000000011f303c6e $s10ObjectiveC15autoreleasepool8invokingxxyKXE_tKlF + 46
    27  libXCTestSwiftSupport.dylib         0x000000011e819d45 $sSo10XCTContextC6XCTestE11runActivity5named5blockxSS_xSo11XCTActivity_pKXEtKlFZ + 197
    28  XCTest_Gherkin                      0x0000000113e41f26 $sSo10XCTestCaseC0A8_GherkinE11performStep_7keyword4file4lineySS_S2SSitF + 6390
    29  XCTest_Gherkin                      0x0000000113e42c82 $sSo10XCTestCaseC0A8_GherkinE11performStep_7keyword4file4lineySS_S2SSitFTo + 194
    30  XCTest_Gherkin                      0x0000000113e3dfde $sSo10XCTestCaseC0A8_GherkinE4Then_4file4lineySS_SSSitF + 254
    31  XCTest_Gherkin                      0x0000000113e3e0aa $sSo10XCTestCaseC0A8_GherkinE4Then_4file4lineySS_SSSitFTo + 138
    32  App-UITests-Runner                  0x0000000111639002 $s13appUITest23AppFeatureC66test_caseyyF + 2722
    33  App-UITests-Runner                  0x000000011163910b $s13appUITest23AppFeatureC66test_caseyyFTo + 43
    34  CoreFoundation                      0x00007fff23e44dec __invoking___ + 140
    35  CoreFoundation                      0x00007fff23e41fd1 -[NSInvocation invoke] + 321
    36  XCTest                              0x000000010efb6fa7 __24-[XCTestCase invokeTest]_block_invoke_2 + 52
    37  XCTest                              0x000000010efb6f53 __24-[XCTestCase invokeTest]_block_invoke.206 + 320
    38  XCTest                              0x000000010f011d32 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
    39  XCTest                              0x000000010f011c44 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
    40  XCTest                              0x000000010efb6966 -[XCTestCase invokeTest] + 1183
    41  XCTest                              0x000000010efb8299 __26-[XCTestCase performTest:]_block_invoke_2 + 43
    42  XCTest                              0x000000010f011d32 +[XCTestCase(Failures) performFailableBlock:testCase:testCaseRun:shouldInterruptTest:] + 69
    43  XCTest                              0x000000010f011c44 -[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 115
    44  XCTest                              0x000000010efb81d0 __26-[XCTestCase performTest:]_block_invoke.359 + 86
    45  XCTest                              0x000000010f02497d +[XCTContext runInContextForTestCase:block:] + 211
    46  XCTest                              0x000000010efb7a84 -[XCTestCase performTest:] + 566
    47  XCTest                              0x000000010effe2fe -[XCTest runTest] + 57
    48  XCTest                              0x000000010efb1cc0 __27-[XCTestSuite performTest:]_block_invoke + 354
    49  XCTest                              0x000000010efb1412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
    50  XCTest                              0x000000010f02497d +[XCTContext runInContextForTestCase:block:] + 211
    51  XCTest                              0x000000010efb13c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
    52  XCTest                              0x000000010efb172e -[XCTestSuite performTest:] + 348
    53  XCTest                              0x000000010effe2fe -[XCTest runTest] + 57
    54  XCTest                              0x000000010efb1cc0 __27-[XCTestSuite performTest:]_block_invoke + 354
    55  XCTest                              0x000000010efb1412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
    56  XCTest                              0x000000010f02497d +[XCTContext runInContextForTestCase:block:] + 211
    57  XCTest                              0x000000010efb13c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
    58  XCTest                              0x000000010efb172e -[XCTestSuite performTest:] + 348
    59  XCTest                              0x000000010effe2fe -[XCTest runTest] + 57
    60  XCTest                              0x000000010efb1cc0 __27-[XCTestSuite performTest:]_block_invoke + 354
    61  XCTest                              0x000000010efb1412 __59-[XCTestSuite _performProtectedSectionForTest:testSection:]_block_invoke + 24
    62  XCTest                              0x000000010f02497d +[XCTContext runInContextForTestCase:block:] + 211
    63  XCTest                              0x000000010efb13c9 -[XCTestSuite _performProtectedSectionForTest:testSection:] + 148
    64  XCTest                              0x000000010efb172e -[XCTestSuite performTest:] + 348
    65  XCTest                              0x000000010effe2fe -[XCTest runTest] + 57
    66  XCTest                              0x000000010f033e84 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
    67  XCTest                              0x000000010f033f71 __44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.100 + 96
    68  XCTest                              0x000000010efcc6b6 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 682
    69  XCTest                              0x000000010f033c0f -[XCTTestRunSession runTestsAndReturnError:] + 615
    70  XCTest                              0x000000010ef956b4 -[XCTestDriver runTestsAndReturnError:] + 456
    71  XCTest                              0x000000010f0205bc _XCTestMain + 2496
    72  App-UITests-Runner                  0x000000010eda2e1a -[_XCTRunnerAppDelegate applicationWillResignActive:] + 0
    73  App-UITests-Runner                  0x000000010eda2d18 _XCTRunnerRunTests + 0
    74  CoreFoundation                      0x00007fff23da1abc __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    75  CoreFoundation                      0x00007fff23da11b3 __CFRunLoopDoBlocks + 195
    76  CoreFoundation                      0x00007fff23d9bfa3 __CFRunLoopRun + 995
    77  CoreFoundation                      0x00007fff23d9b8a4 CFRunLoopRunSpecific + 404
    78  GraphicsServices                    0x00007fff38c39bbe GSEventRunModal + 139
    79  UIKitCore                           0x00007fff49325968 UIApplicationMain + 1605
    80  App-UITests-Runner                  0x000000010eda2fd5 main + 183
    81  libdyld.dylib                       0x00007fff520ce1fd start + 1
)
libc++abi.dylib: terminating with uncaught exception of type _XCTestCaseInterruptionException
1

There are 1 answers

0
Dan On

The workaround I found for Xcode 11.6 is to remove continueAfterFailure = false. This issue has been resolved in the Xcode 12 betas.