Skip to content

Conversation

@stmontgomery
Copy link

This PR adopts Swift Testing in place of XCTest in this package repo's tests.

Modifications

I approached these changes as follows:

  • First, I did a very straightforward, mechanical 1-1 replacement of the XCTest API usages for analogous Swift Testing APIs. I landed that as the first commit, and kept the old XCTest file around temporarily for reference as I continued development.
  • Then, I modified the names of test functions and suite types to use Raw Identifiers for better readability.
  • Next, I evaluated each @Test function (there were only a handful) looking for ways to make them more idiomatic Swift Testing-style tests:
    • Some, I decomposed into multiple smaller tests and grouped into a new sub-suite.
    • Others, I transformed into parameterized test functions.
    • In making these changes, I tried my best to preserve existing code comments and the general flow/structure.
  • Finally, once the conversion was complete, I deleted the old XCTest file.

Testing performed

I ran these at-desk from Xcode 26.1 Beta and confirmed all modified tests are passing.

Fixes #27

…t of changes which are strictly necessary to convert XCTest API usages into their Swift Testing counterparts
…efactor where they will be needed at other scopes
…focused test functions in a sub-suite
…ore focused test functions in a new sub-suite named `Malformed proposals` (after the snapshot it references)
…this for `Good dates` test function but it has many more arguments and its performance in Xcode is somewhat poor currently
@stmontgomery
Copy link
Author

@dempseyatgithub, how does this look? Offline, we discussed a concern about whether it was safe to adopt Swift 6.2 features in this repo yet, but if this Swift CI Jenkins job is what performs that build, it seems like this should be safe since it does not build the package's tests.

Either way, I'm interested to hear your feedback and hopefully get this landed! Thanks.

Copy link
Collaborator

@dempseyatgithub dempseyatgithub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I requested / suggested some changes.

If any of them seem way out of line for idiomatic Swift Testing, please let me know. Obviously, you have had much more experience with it than I have, this is my first serious foray.

The two preferences I have (noted with suggestions in the review) are to explicitly annotate suites with @Suite and to name them using title case so they appear more like a heading.

One other question/piece of feedback. I find that sometimes when a test is gathering arguments with a lot of set-up (like Warning and errors() in this file) the actual test - including seeing the name of the test, gets lost in the noise of gathering the arguments.

Do you have any suggestions to aid in that? Is it a poor stylistic choice to always use a test description with @Test as opposed to using a raw identifier? (I realize that leads to the 'name the thing twice' issue)

Thank you for your patience and for doing the migration to Swift Testing.

And also thank you for answering my questions. I am really excited about Swift Testing (I just rewatched the 2024 WWDC Videos before doing this review, and got enthused all over again - such a beautiful API design!)

try expected.write(to: FileUtilities.expandedAndStandardizedURL(for: path).appending(path: "\(filePrefix)expected.json"))
try actual.write(to: FileUtilities.expandedAndStandardizedURL(for: path).appending(path: "\(filePrefix)actual.json"))
}
struct `Extraction tests` {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer suite names in title case, so it reads more like a header in reports.

I realize suites are implied by a type containing tests, but is there a drawback to explicitly marking with @Suite on the line above?

Suggested change
struct `Extraction tests` {
@Suite
struct `Extraction Tests` {

}
struct `Extraction tests` {

struct `All proposals` {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
struct `All proposals` {
@Suite
struct `All Proposals` {


// Uncomment to write full JSON files out to compare in a diff tool
// try writeJSONFilesToPath(expected: expectedJSONData, actual: actualJSONData, path: "~/Desktop")
@Test func `Serialization`() throws {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the raw identifier is only a single word, the test shows up as 'Serialized()' in reports with the parenthesis.

If I understand correctly, could either add a test description to @Test or add another word to the method name.

Maybe 'Expected Serialization`


for (actualResult, expectedResult) in zip(extractionMetadata.proposals, expectedResults.proposals) {
XCTAssertEqual(actualResult, expectedResult)
struct `Malformed proposals` {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original had a single line to double check that the test array counts are equal, essentially a lightweight validation of the test data.

XCTAssertEqual(extractionMetadata.proposals.count, expectedResults.proposals.count)

I don't think that warrants an entire test on its own.

Can't that just be part of gathering the arguments in Warnings and errors ()?


for (actualResult, expectedResult) in zip(extractionMetadata.proposals, expectedResults.proposals) {
XCTAssertEqual(actualResult, expectedResult)
struct `Malformed proposals` {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the array count check is folded into Warnings and errors() then this may not need to be a separate suite.

Suggested change
struct `Malformed proposals` {
@Suite
struct `Malformed Proposals` {

@Test(arguments: try await {
let extractionJob = try await Self.extractionJob
let expectedResults = try #require(extractionJob.expectedResults, "No expected results found for extraction job with source '\(extractionJob.source)'")
let extractionMetadata = try await EvolutionMetadataExtractor.extractEvolutionMetadata(for: extractionJob)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put the comment from line 86-88 above line 99. (Similar to its original position.)

@Test(arguments: try await {
let extractionJob = try await Self.extractionJob
let expectedResults = try #require(extractionJob.expectedResults, "No expected results found for extraction job with source '\(extractionJob.source)'")
let extractionMetadata = try await EvolutionMetadataExtractor.extractEvolutionMetadata(for: extractionJob)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include this here and remove the Extraction metadata() test.

I changed the #expect to a try #require() since the test should fail if the test data is incorrect (mismatched number of elements).

// This test zips the extraction results with the expected results
// If the two arrays don't have the same count, the test data itself has an error
try #require(extractionMetadata.proposals.count == expectedResults.proposals.count)


@testable import EvolutionMetadataModel
@testable import EvolutionMetadataExtraction

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Proposal type could use a custom test description.

I don't know what the best practice is for adding test descriptions. (As part of the original definition of the type, as part of the test bundle, etc.)

In the case, it is a small extension and likely will not be joined by other test descriptions, so defining at the top doesn't add much clutter, but makes it obvious where that description is coming from.

Would appreciate any feedback you have in terms of canonical Swift Testing as to where this should live, but Proposal definitely needs a custom test description.

Suggested change
extension Proposal: CustomTestStringConvertible {
public var testDescription: String { id.isEmpty ? "No ID" : id }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move to Swift Testing when Swift 6 is available in the deployment environment

2 participants