-
Notifications
You must be signed in to change notification settings - Fork 13
Adopt Swift Testing in metadata extractor tests #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…t of changes which are strictly necessary to convert XCTest API usages into their Swift Testing counterparts
…able and idiomatic
…efactor where they will be needed at other scopes
…focused test functions in a sub-suite
…ore focused test functions in a new sub-suite named `Malformed proposals` (after the snapshot it references)
…this for `Good dates` test function but it has many more arguments and its performance in Xcode is somewhat poor currently
|
@dempseyatgithub, how does this look? Offline, we discussed a concern about whether it was safe to adopt Swift 6.2 features in this repo yet, but if this Swift CI Jenkins job is what performs that build, it seems like this should be safe since it does not build the package's tests. Either way, I'm interested to hear your feedback and hopefully get this landed! Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I requested / suggested some changes.
If any of them seem way out of line for idiomatic Swift Testing, please let me know. Obviously, you have had much more experience with it than I have, this is my first serious foray.
The two preferences I have (noted with suggestions in the review) are to explicitly annotate suites with @Suite and to name them using title case so they appear more like a heading.
One other question/piece of feedback. I find that sometimes when a test is gathering arguments with a lot of set-up (like Warning and errors() in this file) the actual test - including seeing the name of the test, gets lost in the noise of gathering the arguments.
Do you have any suggestions to aid in that? Is it a poor stylistic choice to always use a test description with @Test as opposed to using a raw identifier? (I realize that leads to the 'name the thing twice' issue)
Thank you for your patience and for doing the migration to Swift Testing.
And also thank you for answering my questions. I am really excited about Swift Testing (I just rewatched the 2024 WWDC Videos before doing this review, and got enthused all over again - such a beautiful API design!)
| try expected.write(to: FileUtilities.expandedAndStandardizedURL(for: path).appending(path: "\(filePrefix)expected.json")) | ||
| try actual.write(to: FileUtilities.expandedAndStandardizedURL(for: path).appending(path: "\(filePrefix)actual.json")) | ||
| } | ||
| struct `Extraction tests` { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would prefer suite names in title case, so it reads more like a header in reports.
I realize suites are implied by a type containing tests, but is there a drawback to explicitly marking with @Suite on the line above?
| struct `Extraction tests` { | |
| @Suite | |
| struct `Extraction Tests` { |
| } | ||
| struct `Extraction tests` { | ||
|
|
||
| struct `All proposals` { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| struct `All proposals` { | |
| @Suite | |
| struct `All Proposals` { |
|
|
||
| // Uncomment to write full JSON files out to compare in a diff tool | ||
| // try writeJSONFilesToPath(expected: expectedJSONData, actual: actualJSONData, path: "~/Desktop") | ||
| @Test func `Serialization`() throws { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the raw identifier is only a single word, the test shows up as 'Serialized()' in reports with the parenthesis.
If I understand correctly, could either add a test description to @Test or add another word to the method name.
Maybe 'Expected Serialization`
|
|
||
| for (actualResult, expectedResult) in zip(extractionMetadata.proposals, expectedResults.proposals) { | ||
| XCTAssertEqual(actualResult, expectedResult) | ||
| struct `Malformed proposals` { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The original had a single line to double check that the test array counts are equal, essentially a lightweight validation of the test data.
XCTAssertEqual(extractionMetadata.proposals.count, expectedResults.proposals.count)
I don't think that warrants an entire test on its own.
Can't that just be part of gathering the arguments in Warnings and errors ()?
|
|
||
| for (actualResult, expectedResult) in zip(extractionMetadata.proposals, expectedResults.proposals) { | ||
| XCTAssertEqual(actualResult, expectedResult) | ||
| struct `Malformed proposals` { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the array count check is folded into Warnings and errors() then this may not need to be a separate suite.
| struct `Malformed proposals` { | |
| @Suite | |
| struct `Malformed Proposals` { |
| @Test(arguments: try await { | ||
| let extractionJob = try await Self.extractionJob | ||
| let expectedResults = try #require(extractionJob.expectedResults, "No expected results found for extraction job with source '\(extractionJob.source)'") | ||
| let extractionMetadata = try await EvolutionMetadataExtractor.extractEvolutionMetadata(for: extractionJob) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please put the comment from line 86-88 above line 99. (Similar to its original position.)
| @Test(arguments: try await { | ||
| let extractionJob = try await Self.extractionJob | ||
| let expectedResults = try #require(extractionJob.expectedResults, "No expected results found for extraction job with source '\(extractionJob.source)'") | ||
| let extractionMetadata = try await EvolutionMetadataExtractor.extractEvolutionMetadata(for: extractionJob) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please include this here and remove the Extraction metadata() test.
I changed the #expect to a try #require() since the test should fail if the test data is incorrect (mismatched number of elements).
// This test zips the extraction results with the expected results
// If the two arrays don't have the same count, the test data itself has an error
try #require(extractionMetadata.proposals.count == expectedResults.proposals.count)
|
|
||
| @testable import EvolutionMetadataModel | ||
| @testable import EvolutionMetadataExtraction | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Proposal type could use a custom test description.
I don't know what the best practice is for adding test descriptions. (As part of the original definition of the type, as part of the test bundle, etc.)
In the case, it is a small extension and likely will not be joined by other test descriptions, so defining at the top doesn't add much clutter, but makes it obvious where that description is coming from.
Would appreciate any feedback you have in terms of canonical Swift Testing as to where this should live, but Proposal definitely needs a custom test description.
| extension Proposal: CustomTestStringConvertible { | |
| public var testDescription: String { id.isEmpty ? "No ID" : id } | |
| } |
This PR adopts Swift Testing in place of XCTest in this package repo's tests.
Modifications
I approached these changes as follows:
@Testfunction (there were only a handful) looking for ways to make them more idiomatic Swift Testing-style tests:Testing performed
I ran these at-desk from Xcode 26.1 Beta and confirmed all modified tests are passing.
Fixes #27