This episode is free for everyone.So, this is pretty incredible. We have now explored our new assertMacro testing tool for testing macros, and shown that it fixes basically every shortcoming of Apple’s native testing tool:
We get to capture the expanded macro directly inline in the test without resorting to copying-and-pasting.
We can update macro tests by just flipping an
isRecordingboolean to true and running the test suite again.
And we can even assert on diagnostics directly inline in the the source code. No need to mentally try to visualize where column 20 in line 7 is. It’s just rendered right at the offending character of the input source code.
So, this is all looking pretty great, but so far we have only given this tool a spin for the most basic kind of macro, which is the #stringify macro that comes by default with the macro template in SPM.
Let’s quickly see how the tool fares when testing larger, more complicated macros.
I’ve got our MacroTesting project open right here, and in it we have a quite extensive test suite so that we could really put the library through its paces. To do this we have copied over nearly every example macro that Apple created during the proposal period of macros, which is in the following repo: http://github.com/DougGregor/swift-macro-examples
This repo has some fun macros, like a macro that can create callback versions of async functions, or vice-versa, and a macro for creating URLs from strings that are known at compile time to be correct, as well as some silly ones just to show off the potential, such as a macro that prevents you from using the addition operator.
And even a few tests were included in this project, but not that many. It looks like only the @MetaEnum macro and @NewType macros have tests. It’s hard to know why more tests weren’t written, but it certainly seems possible that it’s because they can be quite annoying to write using the bare assertMacroExpansion helper.
For example, the test for @MetaEnum has the following:
func testBasic() throws {
let sf: SourceFileSyntax = """
@MetaEnum enum Cell {
case integer(Int)
case text(String)
case boolean(Bool)
case null
}
"""
let context = BasicMacroExpansionContext(
sourceFiles: [
sf: .init(
moduleName: "MyModule",
fullFilePath: "test.swift"
)
]
)
let transformed = sf.expand(
macros: testMacros, in: context
)
XCTAssertEqual(transformed.description, """
enum Cell {
case integer(Int)
case text(String)
case boolean(Bool)
case null
enum Meta {
case integer
case text
case boolean
case null
init(_ __macro_local_6parentfMu_: Cell) {
switch __macro_local_6parentfMu_ {
case .integer:
self = .integer
case .text:
self = .text
case .boolean:
self = .boolean
case .null:
self = .null
}
}
}
}
""")
}
This shows that the apply the @MetaEnum macro to an enum causes a Meta enum type to be nested inside your type that has a case for each of your cases, but no associated value. It also provides a way to create a meta value from one of your values.
But, if you look closely you will see this weird thing:
init(_ __macro_local_6parentfMu_: Cell) {
…
}
The SwiftSyntax library has a tool for generating a unique identifier so that you can make sure to not accidentally use an identifier that conflicts with the user’s code. We don’t think this is a concern here, but maybe they are just demonstrating its use in this macro, or maybe there is something we don’t see.
So, this shows a new pain point of writing tests for macros. Sometimes the expanded macro code can contain a unique identifier that we can’t even predict until we run the macro for the first time. And then we have to copy-and-paste that into the expected expanded source.
And then a little down the test file you will see how they tested the diagnostic that runs to make sure the type the macro is applied to is an enum:
func testNonEnum() throws {
let sf: SourceFileSyntax = """
@MetaEnum struct Cell {
let integer: Int
let text: String
let boolean: Bool
}
"""
let context = BasicMacroExpansionContext(
sourceFiles: [
sf: .init(
moduleName: "MyModule",
fullFilePath: "test.swift"
)
]
)
let transformed = sf.expand(
macros: testMacros, in: context
)
XCTAssertEqual(transformed.description, """
struct Cell {
let integer: Int
let text: String
let boolean: Bool
}
""")
XCTAssertEqual(context.diagnostics.count, 1)
let diag = try XCTUnwrap(context.diagnostics.first)
XCTAssertEqual(
diag.message,
"""
'@MetaEnum' can only be attached to an enum, not a \
struct
"""
)
XCTAssertEqual(diag.diagMessage.severity, .error)
}
This too is a big of a pain, and isn’t asserting on everything it could. Because it can be annoying to assert on the full details of a diagnostic, a shortcut has been taken to only assert on the message and severity of the diagnostic. So we aren’t capturing the line and column where the error took place.
Let’s see what these tests look like over in our swift-macro-testing library.
For one thing, the basic test for the expansion looks like this:
func testMetaEnum() {
assertMacro {
#"""
@MetaEnum enum Value {
case integer(Int)
case text(String)
case boolean(Bool)
case null
}
"""#
} matches: {
"""
enum Value {
case integer(Int)
case text(String)
case boolean(Bool)
case null
enum Meta {
case integer
case text
case boolean
case null
init(_ __macro_local_6parentfMu_: Value) {
switch __macro_local_6parentfMu_ {
case .integer:
self = .integer
case .text:
self = .text
case .boolean:
self = .boolean
case .null:
self = .null
}
}
}
}
"""
}
}
…which is very similar to what was over in Apple’s project. But we can completely delete the last trailing closure:
func testMetaEnum() {
assertMacro {
#"""
@MetaEnum enum Value {
case integer(Int)
case text(String)
case boolean(Bool)
case null
}
"""#
}
}
…run the test again, and it automatically fills in the freshest macro expansion.
And even better, the diagnostic for when the macro is applied to a non-enum looks much better:
func testNonEnum() {
assertMacro {
"""
@MetaEnum struct Cell {
let integer: Int
let text: String
let boolean: Bool
}
"""
} matches: {
"""
@MetaEnum struct Cell {
┬────────
╰─ 🛑 '@MetaEnum' can only be attached to an enum,
not a struct
let integer: Int
let text: String
let boolean: Bool
}
"""
}
}
It shows the right directly on the line and column where one would see the error in Xcode. And again we can delete this trailing closure and run again to have it automatically fill back in with the freshest expansion.
But now that macro tests are so easy write we should feel empowered to test all types of little edge cases and nuances of the macro. By doing so we can make sure to give users of our macro library a good experience, and not leave them with hidden or cryptic failures.
For example, did you know that Swift technically supports overloaded case names in enums? We’re not sure how often it is used, but it is possible, and so you should probably always write a test for that use case whenever you have a macro that operates on enum cases.
Luckily for us it’s extremely easy to write that test now:
func testOverloadedCaseName() {
assertMacro {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
}
}
And this is perfectly valid Swift code. The following compiles just fine:
enum Foo {
case bar(int: Int)
case bar(string: String)
}
let fooInt = Foo.bar(int: 1)
let fooString = Foo.bar(string: "Hello")
Running this test expands to the following:
func testDuplicateCaseName() {
assertMacro {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
enum Foo {
case bar(int: Int)
case bar(string: String)
enum Meta {
case bar
case bar
init(_ __macro_local_6parentfMu_: Foo) {
switch __macro_local_6parentfMu_ {
case .bar:
self = .bar
case .bar:
self = .bar
}
}
}
}
"""
}
}
And already we can see the problem.
Because the nested Meta enum only looks at the case names, and not the label names for the case associated values, we are getting duplicated cases:
enum Meta {
case bar
case bar
…
}
And that is going to be a compilation error, which we can see if we copy and paste the expanded source:
enum Foo {
…
enum Meta {
case bar
case bar 🛑
…
}
Invalid redeclaration of ‘bar’
And that is technically fine. Macros can generate invalid Swift code, and it will just result in a compilation error once Swift has finished expanding the macro and starts to compile the full source code. However, there is a pretty big ergonomics gap between diagnostics the macro surfaces itself and the compilation errors one encounters from an invalid macro expansion.
To see this, let’s hop back over to Apple’s macro project, and paste this enum into their main.swift file where they showcase the various macros:
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
We instantly get a compilation error.
And just to be sure, let’s remove the @MetaEnum:
enum Foo {
case bar(int: Int)
case bar(string: String)
}
…and now it compiles fine. So this kind of enum is totally fine, but the macro cannot handle it.
And the failure we get is pretty mystifying unfortunately. It doesn’t show an error on the line with the enum, and instead we have to go to the “Issue navigator” to see the problem. And only there do we see that it has to do with invalid generated code.
This demonstrates a very important principle for when building your own macros. It is far, far better for your macro implementation to emit diagnostics in as many situations as possible rather than letting the macro to generate invalid Swift code and letting the compiler choke on the error. If you allow Swift to diagnose your problem then the compilation errors become hard to find and very cryptic.
So, let’s improve this macro! We are going to have the macro detect when overloaded case names are used, and when it does detect that it will early-out and throw an error. This should give us much better failure messages in Xcode.
So, how can we do that?
Well, let’s start by putting a breakpoint somewhere in the macro code and run the test so that we can see what we have available to us in the Swift syntax tree. If we look at the MetaEnumMacro:
extension MetaEnumMacro: MemberMacro {
public static func expansion(
of node: AttributeSyntax,
providingMembersOf declaration: some DeclGroupSyntax,
in context: some MacroExpansionContext
) throws -> [DeclSyntax] {
let macro = try MetaEnumMacro(
node: node,
declaration: declaration,
context: context
)
return [ macro.makeMetaEnum() ]
}
}
We will see that the majority of its logic must be contained in the initializer and makeMetaEnum method.
If we then hop to the initializer we will see that there already is a bit of validation logic in here to make sure that the macro is applied only to enums:
init(
node: AttributeSyntax,
declaration: some DeclGroupSyntax,
context: some MacroExpansionContext
) throws {
guard
let enumDecl = declaration.as(EnumDeclSyntax.self)
else {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(node))
])
}
…
}
Let’s put a breakpoint in right after this guard.
And let’s run the test we just wrote.
Let’s see what is in this enumDecl variable that is bound after from the guard:
(lldb) po enumDecl
EnumDeclSyntax
├─attributes: AttributeListSyntax
│ ╰─[0]: AttributeSyntax
│ ├─atSign: atSign
│ ╰─attributeName: IdentifierTypeSyntax
│ ╰─name: identifier("MetaEnum")
├─enumKeyword: keyword(SwiftSyntax.Keyword.enum)
├─name: identifier("Foo")
╰─memberBlock: MemberBlockSyntax
├─leftBrace: leftBrace
├─members: MemberBlockItemListSyntax
│ ├─[0]: MemberBlockItemSyntax
│ │ ╰─decl: EnumCaseDeclSyntax
│ │ ├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
│ │ ╰─elements: EnumCaseElementListSyntax
│ │ ╰─[0]: EnumCaseElementSyntax
│ │ ├─name: identifier("bar")
│ │ ╰─parameterClause: EnumCaseParameterClauseSyntax
│ │ ├─leftParen: leftParen
│ │ ├─parameters: EnumCaseParameterListSyntax
│ │ │ ╰─[0]: EnumCaseParameterSyntax
│ │ │ ├─firstName: identifier("int")
│ │ │ ├─colon: colon
│ │ │ ╰─type: IdentifierTypeSyntax
│ │ │ ╰─name: identifier("Int")
│ │ ╰─rightParen: rightParen
│ ╰─[1]: MemberBlockItemSyntax
│ ╰─decl: EnumCaseDeclSyntax
│ ├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
│ ╰─elements: EnumCaseElementListSyntax
│ ╰─[0]: EnumCaseElementSyntax
│ ├─name: identifier("bar")
│ ╰─parameterClause: EnumCaseParameterClauseSyntax
│ ├─leftParen: leftParen
│ ├─parameters: EnumCaseParameterListSyntax
│ │ ╰─[0]: EnumCaseParameterSyntax
│ │ ├─firstName: identifier("string")
│ │ ├─colon: colon
│ │ ╰─type: IdentifierTypeSyntax
│ │ ╰─name: identifier("String")
│ ╰─rightParen: rightParen
╰─rightBrace: rightBrace
We can now see all the details of the enum in its abstract syntax tree format. We can use this print out to figure out how to traverse inside the syntax tree and extract out the information we need. In particular, we want to extract out the names of the cases of the enum:
├─name: identifier("bar")
…
├─name: identifier("bar")</pre>
…and check if there are any duplicates.
First layer we need to traverse into is the memberBlock of the variable:
(lldb) po enumDecl.memberBlock
MemberBlockSyntax
├─leftBrace: leftBrace
├─members: MemberBlockItemListSyntax
│ ├─[0]: MemberBlockItemSyntax
│ │ ╰─decl: EnumCaseDeclSyntax
│ │ ├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
│ │ ╰─elements: EnumCaseElementListSyntax
│ │ ╰─[0]: EnumCaseElementSyntax
│ │ ├─name: identifier("bar")
│ │ ╰─parameterClause: EnumCaseParameterClauseSyntax
│ │ ├─leftParen: leftParen
│ │ ├─parameters: EnumCaseParameterListSyntax
│ │ │ ╰─[0]: EnumCaseParameterSyntax
│ │ │ ├─firstName: identifier("int")
│ │ │ ├─colon: colon
│ │ │ ╰─type: IdentifierTypeSyntax
│ │ │ ╰─name: identifier("Int")
│ │ ╰─rightParen: rightParen
│ ╰─[1]: MemberBlockItemSyntax
│ ╰─decl: EnumCaseDeclSyntax
│ ├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
│ ╰─elements: EnumCaseElementListSyntax
│ ╰─[0]: EnumCaseElementSyntax
│ ├─name: identifier("bar")
│ ╰─parameterClause: EnumCaseParameterClauseSyntax
│ ├─leftParen: leftParen
│ ├─parameters: EnumCaseParameterListSyntax
│ │ ╰─[0]: EnumCaseParameterSyntax
│ │ ├─firstName: identifier("string")
│ │ ├─colon: colon
│ │ ╰─type: IdentifierTypeSyntax
│ │ ╰─name: identifier("String")
│ ╰─rightParen: rightParen
╰─rightBrace: rightBrace
And then in here we can further traverse into the members of the member block:
(lldb) po enumDecl.memberBlock.members
MemberBlockItemListSyntax
├─[0]: MemberBlockItemSyntax
│ ╰─decl: EnumCaseDeclSyntax
│ ├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
│ ╰─elements: EnumCaseElementListSyntax
│ ╰─[0]: EnumCaseElementSyntax
│ ├─name: identifier("bar")
│ ╰─parameterClause: EnumCaseParameterClauseSyntax
│ ├─leftParen: leftParen
│ ├─parameters: EnumCaseParameterListSyntax
│ │ ╰─[0]: EnumCaseParameterSyntax
│ │ ├─firstName: identifier("int")
│ │ ├─colon: colon
│ │ ╰─type: IdentifierTypeSyntax
│ │ ╰─name: identifier("Int")
│ ╰─rightParen: rightParen
╰─[1]: MemberBlockItemSyntax
╰─decl: EnumCaseDeclSyntax
├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
╰─elements: EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("string")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("String")
╰─rightParen: rightParen
Now we have an array of each case in the enum. We can map over the members to extract out the decl field from each member:
(lldb) po enumDecl.memberBlock.members.map { $0.decl }
▿ 2 elements
- 0 : EnumCaseDeclSyntax
├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
╰─elements: EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("int")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("Int")
╰─rightParen: rightParen
- 1 : EnumCaseDeclSyntax
├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
╰─elements: EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("string")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("String")
╰─rightParen: rightParen
Now we have an array of what seems to be EnumCaseDeclSyntax values, but really it’s an array of DeclSyntax values. We need to cast the DeclSyntax values to EnumCaseDeclSyntax, which can be done using the .as method:
(lldb) po enumDecl.memberBlock.members.map { $0.decl.as(EnumCaseDeclSyntax.self) }
▿ 2 elements
▿ 0 : Optional<EnumCaseDeclSyntax>
- some : EnumCaseDeclSyntax
├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
╰─elements: EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("int")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("Int")
╰─rightParen: rightParen
▿ 1 : Optional<EnumCaseDeclSyntax>
- some : EnumCaseDeclSyntax
├─caseKeyword: keyword(SwiftSyntax.Keyword.case)
╰─elements: EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("string")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("String")
╰─rightParen: rightParen</pre>
Which can return an optional, so let’s compactMap the result.
OK, we are getting closer and closer by whittling down this large, complex syntax tree down to the parts we care about. Next we can reach into the elements of the enum case declaration:
(lldb) po enumDecl.memberBlock.members.compactMap { $0.decl.as(EnumCaseDeclSyntax.self)?.elements }
▿ 2 elements
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("int")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("Int")
╰─rightParen: rightParen
▿ 1 : Optional<EnumCaseElementListSyntax>
- some : EnumCaseElementListSyntax
╰─[0]: EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("string")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("String")
╰─rightParen: rightParen</pre>
Technically each EnumCaseDeclSyntax has two elements, but only the first element is of interest. It’s the value that holds the actual name of the case:
(lldb) po enumDecl.memberBlock.members.compactMap { $0.decl.as(EnumCaseDeclSyntax.self)?.elements.first }
▿ 2 elements
▿ 0 : EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("int")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("Int")
╰─rightParen: rightParen
▿ 1 : EnumCaseElementSyntax
├─name: identifier("bar")
╰─parameterClause: EnumCaseParameterClauseSyntax
├─leftParen: leftParen
├─parameters: EnumCaseParameterListSyntax
│ ╰─[0]: EnumCaseParameterSyntax
│ ├─firstName: identifier("string")
│ ├─colon: colon
│ ╰─type: IdentifierTypeSyntax
│ ╰─name: identifier("String")
╰─rightParen: rightParen</pre>
This has all the information we need to perform the validation. So, let’s store this array in a variable:
let enumCaseDecls = enumDecl.memberBlock
.members
.compactMap {
$0.decl.as(EnumCaseDeclSyntax.self)?.elements.first
}
And we can use a set to determine if there are duplicate case names:
let caseNames = Set(enumCaseDecls.map { $0.name.text })
if caseNames.count != enumCaseDecls.count {
// Throw diagnostic
}
In this if branch we can throw an error and that will bubble up to a compilation error when compiling with the macro.
The easiest thing to do is simply define a new error type:
struct OverloadedCaseError: Error {}
And throw the error when we detect duplicate case names:
if seenCaseNames.contains(name) {
throw OverloadedCaseError()
}
If we delete the previously recorded snapshot of the macro expansion and run the test again we will see a new expansion recorded in the test file that shows the diagnostic:
func testOverloadedCaseName() {
assertMacro {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
@MetaEnum enum Foo {
┬────────
╰─ 🛑 OverloadedCaseError()
case bar(int: Int)
case bar(string: String)
}
"""
}
}
So, this is certainly better than an obscured, hidden compilation failure in Xcode, but also we can certainly do better. It be a lot better if the failure pointed to the line where the duplicate case was detected, and I think we can make a much nicer error message.
We can see how to do this by taking inspiration from what the macro does to diagnose the macro being applied to a non-struct:
guard
let enumDecl = declaration.as(EnumDeclSyntax.self)
else {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(node))
])
}
It throws a DiagnosticsError, which is a SwiftSyntax type, and to construct this type you provide an array of Diagnostic value, which is also a SwiftSyntax type. This macro has has a pattern for dealing with diagnostics:
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(node))
The CaseMacroDiagnostic type is an enum that lists each kind of diagnostic that can be emitted. Currently there is only one:
enum CaseMacroDiagnostic {
case notAnEnum(DeclGroupSyntax)
}
It holds onto a DeclGroupSyntax value because it’s later used to customize the error messaging.
So, we should be able to add a new kind of diagnostic to the macro by adding a new case:
enum CaseMacroDiagnostic {
case overloadedCase
case notAnEnum(DeclGroupSyntax)
}
This creates a few compiler errors because we need to exhaustively handle this new case in various switchs. Such as in the message property:
case .overloadedCase:
return """
'@MetaEnum' cannot be applied to enums with \
overloaded case names.
"""
And the diagnosticID:
case .overloadedCase:
return MessageID(
domain: "MetaEnumDiagnostic", id: "overloadedCase"
)
And severity:
case .overloadedCase:
return .error
That’s all it takes and we can now throw a more specific kind of error:
if caseNames.count != enumCaseDecls.count {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.overloadedCase
.diagnose(at: Syntax(node))
])
}
And now if we re-record the macro snapshot the test we will see a much better output:
func testDuplicateCaseName() {
assertMacro(record: true) {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
@MetaEnum enum Foo {
┬────────
╰─ 🛑 '@MetaEnum' cannot be applied to enums with
overloaded case names.
case bar(int: Int)
case bar(string: String)
}
"""
}
}
But it’s still pointing to the macro itself. How can we get it to point at the case?
We can go back to the macro code and refactor the code a bit to get at the particular enum decl case node and use that instead.
var seenCaseNames: Set<String> = []
for enumCaseDecl in enumCaseDecls {
let name = enumCaseDecl.name.text
defer { seenCaseNames.insert(name) }
if seenCaseNames.contains(name) {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.overloadedCase
.diagnose(at: Syntax(enumCaseDecl))
])
}
}
And if we run the test another time it records a new snapshot:
func testDuplicateCaseName() {
assertMacro(record: true) {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
┬──────────────────
╰─ 🛑 '@MetaEnum' cannot be applied to enums
with overloaded case names.
}
"""
}
}
Much much better! But also, maybe we can localize the highlighted code to just the name:
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.overloadedCase
.diagnose(at: Syntax(enumCaseDecl.name))
])
And if we run the test one more time time:
func testDuplicateCaseName() {
assertMacro(record: true) {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
┬──
╰─ 🛑 '@MetaEnum' cannot be applied to enums
with overloaded case names.
}
"""
}
}
We now see a bunch better failure message and it even points to the exact line and columns where the offending case is.
And we can even take this for a spin in Apple’s example macro project. Let’s copy-and-paste our improved MetaEnumMacro into the Apple project.
And now when we build we get an error directly on the problematic line:
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
‘@MetaEnum’ cannot be applied to enums with overloaded case names.
And that’s pretty incredible.
But let’s improve things a bit more. There’s another edge case that our diagnostic doesn’t account for, and that’s due to the fact that enums can specify multiple cases at once. We can get a test in place that demonstrates the problem:
func testOverloadedCaseName_SingleLine() {
assertMacro {
"""
@MetaEnum enum Foo {
case bar(int: Int), bar(string: String)
}
"""
}
}
And we can run the test to expand things:
assertMacro {
"""
@MetaEnum enum Foo {
case bar(int: Int), bar(string: String)
}
"""
} matches: {
"""
enum Foo {
case bar(int: Int), bar(string: String)
enum Meta {
case bar
case bar
init(_ __macro_local_6parentfMu_: Foo) {
switch __macro_local_6parentfMu_ {
case .bar:
self = .bar
case .bar:
self = .bar
}
}
}
}
"""
}
Well we seem to be back to generating invalid code instead of showing an error. We can find the problem in the macro code we wrote:
let enumCaseDecls = enumDecl.memberBlock
.members
.compactMap {
$0.decl.as(EnumCaseDeclSyntax.self)?.elements.first
}
Based on the example we were working with, we simply plucked out the first element of the enum case decl, but elements corresponds to all the cases that may appear for a single case decl. So it’s not correct to simply grab the first element. Instead, we should grab all of them at once, which we can do by flat-mapping over all of the elements:
let enumCaseDecls = enumDecl.memberBlock
.members
.compactMap {
$0.decl.as(EnumCaseDeclSyntax.self)
}
.flatMap(\.elements)
And if we re-run the test, it generates the diagnostics we want:
func testDuplicateCaseName() {
assertMacro(record: true) {
"""
@MetaEnum enum Foo {
case bar(int: Int)
case bar(string: String)
}
"""
} matches: {
"""
@MetaEnum enum Foo {
case bar(int: Int), bar(string: String)
┬──
╰─ 🛑 '@MetaEnum' cannot be
applied to enums with
overloaded case names.
}
"""
}
}
This just shows how complex it can be for us to write proper macros that account for all the weird edge cases of Swift syntax.
While we typically don’t define our enum cases in this way, it doesn’t mean a user of our macros doesn’t define enum cases in this way, so we should still accommodate this syntax. And luckily our MacroTesting library makes it easy to test these edge cases.
And these tests are just so easy to write that it empowers you to fine tune your macros in the most precise way possible. For example, let’s look at the test that shows a diagnostic when you apply @MetaEnum to a non-enum type:
func testNonEnum() {
assertMacro {
"""
@MetaEnum struct Cell {
let integer: Int
let text: String
let boolean: Bool
}
"""
} matches: {
"""
@MetaEnum struct Cell {
┬────────
╰─ 🛑 '@MetaEnum' can only be attached to an enum,
not a struct
let integer: Int
let text: String
let boolean: Bool
}
"""
}
}
There’s something a little strange about this diagnostic. It wasn’t easy to see with how this was tested over in Apple’s swift-macro-testing project, because that assertion looked like this:
XCTAssertEqual(context.diagnostics.count, 1)
let diag = try XCTUnwrap(context.diagnostics.first)
XCTAssertEqual(
diag.message,
"""
'@MetaEnum' can only be attached to an enum, not a \
struct
"""
)
XCTAssertEqual(diag.diagMessage.severity, .error)
There is no mention of the line or column of the diagnostic and so it’s not possible to visualize where it actually shows in Xcode.
But, even though the diagnostic display is quite a bit better, these lines don’t seem quite right to me:
@MetaEnum struct Cell {
┬────────
╰─ 🛑 '@MetaEnum' can only be attached to an enum, not a
struct
The problem is pointing to @MetaEnum, but the root cause of the problem is that we are using a struct and not an enum. What does it take to get the diagnostic to point in the right spot?
The problem is with these lines right here:
guard
let enumDecl = declaration.as(EnumDeclSyntax.self)
else {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(node))
])
}
This checks if the declaration the macro is applied to is an enum, and if not it throws a diagnostic pointing to the node, and in this case the node is the actual @MetaEnum macro attribute.
So now it makes sense why the diagnostic is pointing to a non-optimal place, but can we fix it?
What we need to do is get access to the struct, class or actor token in the declaration. Since the declaration is not an enum it must definitely at least by a struct, class or actor.
We have to do this in a pretty ad-hoc way. We can first check if the declaration is a struct:
if let structDecl = declaration.as(StructDeclSyntax.self) {
…and if it is we will throw a more specific diagnostic pointing to the struct keyword of the struct declaration:
if let structDecl = declaration.as(StructDeclSyntax.self) {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(structDecl.structKeyword))
])
}
And we need to do the same for classes:
} else if let classDecl = declaration.as(
ClassDeclSyntax.self
) {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(classDecl.classKeyword))
])
}
And the same for actors:
} else if let actorDecl = declaration.as(
ActorDeclSyntax.self
) {
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(actorDecl.actorKeyword))
])
}
And it shouldn’t be possible to get past these 3 if statements since every type is either an enum, struct, actor or class, but just to be future forward we can fallback to diagnosing at the node like we did before:
guard
let enumDecl = declaration.as(EnumDeclSyntax.self)
else {
if let structDecl = declaration.as(
StructDeclSyntax.self
) {
…
} else if let classDecl = declaration.as(
ClassDeclSyntax.self
) {
…
} else if let actorDecl = declaration.as(
ActorDeclSyntax.self
) {
…
}
throw DiagnosticsError(diagnostics: [
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(node))
])
}
If we run tests…
And now we see that the diagnostic is pointing to the correct spot:
func testNonEnum() {
assertMacro {
"""
@MetaEnum struct Cell {
let integer: Int
let text: String
let boolean: Bool
}
"""
} matches: {
"""
@MetaEnum struct Cell {
┬─────
╰─ 🛑 '@MetaEnum' can only be attached to an
enum, not a struct
let integer: Int
let text: String
let boolean: Bool
}
"""
}
}
And so that is pretty incredible. The visual test makes it so clear that we are now pointing to the correct spot with our diagnostic.
For example, there are a lot of things we could have done wrong in the positioning of this diagnostic. We could have pointed it at the name of the type instead:
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(structDecl.name))
And that results in an expansion like this:
"""
@MetaEnum struct Cell {
┬───
╰─ 🛑 '@MetaEnum' can only be attached to
an enum, not a struct
let integer: Int
let text: String
let boolean: Bool
}
"""
And that’s a little weird, just as it was weird to point at the macro for the error. We think it makes most sense to point to the struct keyword, and so let’s go back to that:
CaseMacroDiagnostic.notAnEnum(declaration)
.diagnose(at: Syntax(structDecl.structKeyword))
And these kinds of nuances would have been very difficult to understand by only being able to assert against the abstract line and column numbers. The difference between column 10, which is where the struct keyword is, and column 17, which is where the Cell identifier is, isn’t immediately understandable unless you actually count out the columns. And that’s a pain to do.
So, we have now seen how our macro testing library can massively improve writing tests for macros in some real world macros that Apple created to demonstrate the power of the new macro system.
But, let’s explore more real world macros. We took a look around the community to see what kinds of macros people are making and how our testing tool can greatly improve how tests are written. And maybe even our tool can empower us to improve some of these tools in subtle ways because testing them is so easy.
One of the more polished and extensive projects we found out there was a project by Ian Keen called MacroKit. It has a variety of interesting macros, such as one for generating public initializers for types, one for generating mocks for protocols, and a lot more. And best of all, there is a full test suite that demonstrates how all of the macro code is expanded, and Ian even tests his diagnostics, which is great.
So, let’s take a look at this project, and see what it looks like to use our testing tool, and we’ll even make a few improvements to Ian’s macros along the way.
I’ve got Ian’s MacroKit project opened right now, but I am actually on a fork of his repo because we needed to be able to update his project to target the newest version of SwiftSyntax in order to be compatible with the version of SwiftSyntax our MacroTesting library uses.
This is just another example of some of the complications that have arisen with trying to use SwiftSyntax in open source libraries, and is one of the things we brought up in our Swift forum post. We still don’t have any official guidance from Apple to see how we are expected to deal with these issues, but hopefully something will come of it soon.
We can see in the project that the library ships 6 handy macros. The @Default macro is something that can be used to give a field a default value when decoding it from JSON. It’s usage is quite similar to a property wrapper, but property wrappers are actually not powerful enough to implement this feature. So it’s pretty cool to see how macros can pick up the slack.
Then there’s a @GenerateMock macro that will generate an implementation of most any kind of protocol that allows you to spy on how one interacted with the conformance, such as checking if a property getter/setter or method was called. We can even jump to the client target to expand an example of this macro to see how it works.
There’s also a @KeyPathIterable macro that exposes a static variable that holds every key path that can be derived from the type’s properties in its declaration. There’s a @PublicInit macro that can generate a public initializer for most any kind of struct. There’s a @StaticMemberIterable macro that generates a static array on any type that simply holds all the static values held in the type. And finally there’s an @UnkeyedCodable macro that encodes a value into JSON by just dropping all the keys and putting all the data in a flat array.
So, lots of cool stuff in here, but even better there is a complete test suite. Let’s take a look at a few tests in more detail.
Let’s start with the @PublicInit macro. There is a test that shows the macro applied to a struct with a variety of types of properties, such as stored properties, private ones, computed ones, and even a property with its type inferred. The test shows how the macro expands its code, and it even asserts that it a diagnostic is emitted:
func testPublicInit_HappyPath() {
assertMacroExpansion(
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
""",
expandedSource: """
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
public init(
a: String,
b: Int = 42
) {
self.a = a
self.b = b
}
}
""",
diagnostics: [
.init(
message: """
@PublicInit requires stored properties provide \
explicit type annotations
""",
line: 5,
column: 5
)
],
macros: testMacros
)
}
The diagnostic is there because without any explicit type information on the property it is impossible to incorporate the property in the initializer.
However, it’s kind of hard to really see where the diagnostic would be displayed in Xcode. It says line 5 and column 5, but I need to actually count that out in the string to understand it will actually point to the var c portion of the code.
Well, hopefully our testing tool will make this a lot better. Let’s add our library to this package:
.package(
url: """
https://github.com/pointfreeco/swift-macro-testing
""",
from: "0.1.0"
)
And we’ll add MacroTesting to the MacroKitTests target:
.testTarget(
name: "MacroKitTests",
dependencies: [
"MacroKitMacros",
.product(
name: "SwiftSyntaxMacrosTestSupport",
package: "swift-syntax"
),
.product(
name: "MacroTesting",
package: "swift-macro-testing"
)
]
),
And now we can import our library:
import MacroTesting
To recreate this “happy path” test in our library we just have to use the assertMacro helper and specify the source string we want to expand and can completely forget about the string that asserts what should be expanded:
func testPublicInit_HappyPath_Improved() {
assertMacro {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
"""
}
}
And we have to specify which macros we want to expand in this test, which we can do once for the entire test file by overriding invokeTest:
override func invokeTest() {
withMacroTesting(macros: [PublicInitMacro.self]) {
super.invokeTest()
}
}
Now we are ready to run the test, and when we do we see the macro code expanded automatically for us with a very informative diagnostic inlined directly into the source code:
func testPublicInit_HappyPath_Improved() {
assertMacro {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
"""
} matches: {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
╰─ 🛑 @PublicInit requires stored properties
provide explicit type annotations
var b2: Int {
return b + 1
}
}
"""
}
}
This shows us exactly where the diagnostic will be displayed in the source code. No more figuring out what line 5 and column 5 corresponds to.
But let’s take things further. What if this error had a fixit associated that filled in a little placeholder for the type so that it was very clear what it wanted us to do. We can even leverage a nice little feature of Xcode where if you bracket some text in <#…#> it will be rendered in a unique style.
So, what if the fixit simply inserted this little bit of text:
struct Foo {
var a: String
private var b: Int = 42
var c: <#Type#> = true
var b2: Int {
return b + 1
}
}
Making it incredibly clear what we intend the user to do to fix the error.
So, let’s see how we can introduce such a fixit, and then see how we can test it.
If we search for a fragment of the diagnostic message in the project, say “PublicInit requires stored properties”, we will see exactly where the diagnostic is added in the macro code:
for property in structDecl.storedProperties {
if property.type != nil {
included.append(property)
} else {
context.diagnose(
.init(
node: property._syntaxNode,
message: InferenceDiagnostic()
)
)
}
}
Here we are looping over every stored property in the struct and if there is no type annotation we create a diagnostic whose message is supplied by InferenceDiagnostic, which is a simple type defined above:
struct InferenceDiagnostic: DiagnosticMessage {
let diagnosticID = MessageID(
domain: "PublicInitMacro", id: "inference"
)
let severity: DiagnosticSeverity = .error
let message: String = """
@PublicInit requires stored properties provide \
explicit type annotations
"""
}
The context.diagnose method takes a Diagnostic value as an argument, and its initializer has an extra argument called fixIt:
context.diagnose(
.init(
node: property._syntaxNode,
message: InferenceDiagnostic(),
fixIt: <#FixIt#>
)
)
This is a FixIt value that describes a ways to fix the current syntax.
A FixIt can be created by specifying a message and an array of changes to apply when the “Fix” button is tapped in the Xcode interface:
context.diagnose(
.init(
node: property._syntaxNode,
message: InferenceDiagnostic(),
fixIts: [
FixIt(
message: <#FixItMessage#>,
changes: <#[FixIt.Change]#>
)
]
)
)
A message can be provided by creating a new type that conforms to the FixItMessage protocol, which can be done like so:
struct InsertTypeAnnotationFixItMessage: FixItMessage {
var message = "Insert type annotation."
var fixItID = MessageID(
domain: "PublicInitMacro", id: "type-annotation"
)
}
And now we can use that type:
context.diagnose(
.init(
node: property._syntaxNode,
message: InferenceDiagnostic(),
fixIts: [
FixIt(
message: InsertTypeAnnotationFixItMessage(),
changes: <#[FixIt.Change]#>
)
]
)
)
And then in the array of changes we want to return one single change that will modify the property we are analyzing in order to have a type annotation:
context.diagnose(
.init(
node: property._syntaxNode,
message: InferenceDiagnostic(),
fixIts: [
FixIt(
message: InsertTypeAnnotationFixItMessage(),
changes: [
.replace(oldNode: <#Syntax#>, newNode: <#Syntax#>)
]
)
]
)
)
This allows us to provide the current property node that we want to rewrite as well as the new node that describes the rewrite. And when the user taps “Fix” in Xcode this change will be applied.
The old node is the property we already have a handle on:
changes: [
.replace(oldNode: Syntax(property), newNode: <#Syntax#>)
]
And the new node needs to be computed. It can be a little hairy to do this. Let’s start by making a copy of the current property so that we can make mutations to it:
var newProperty = property
Which we will use that as the new node for rewriting:
changes: [
.replace(
oldNode: Syntax(property),
newNode: Syntax(newProperty)
)
]
The type annotation information is held inside the bindings of this property. Currently there is one binding in this property, which we can see by putting in a breakpoint and printing the description of bindings:
(lldb) po newProperty.bindings.description
"c = true"
We want to alter this binding to include the type information.
To do that we can traverse into the first binding by using the startIndex and then further traverse into the typeAnnotation:
newProperty.bindings[newProperty.bindings.startIndex]
.typeAnnotation
And then we can mutate this property to add a type annotation that is a IdentifierTypeSyntax with the name we want:
newProperty.bindings[newProperty.bindings.startIndex]
.typeAnnotation = TypeAnnotationSyntax(
type: IdentifierTypeSyntax(name: "<#Type#>")
)
The identifier <#Type#> is of course not a valid type name at all in Swift, but that’s OK. We want to insert this weird piece of syntax into the source so that it is treated like a placeholder token in Xcode.
And that’s basically all it takes. But how do we test this? Well, let’s run our test suite just to see if anything has changed at all.
And sure enough we get a test failure:
failed - Actual output (+) differed from expected output (-). Difference: …
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
╰─ 🛑 @PublicInit requires stored properties
provide explicit type annotations
+ ✏️ Insert type annotation.
var b2: Int {
return b + 1
}
}
This is showing us that we now have a FixIt diagnostic in our macro expansion, and we are not currently asserting against it. It helpfully shows us what line and column it affects, has a nice little emoji to indicate that it is a fixIt, and it shows us the message that would show in Xcode.
So, let’s re-record this test by deleting the matches trailing closure and running again:
func testPublicInit_HappyPath_Improved() {
assertMacro {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
"""
} matches: {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
╰─ 🛑 @PublicInit requires stored properties
provide explicit type annotations
✏️ Insert type annotation.
var b2: Int {
return b + 1
}
}
"""
}
}
This is absolutely amazing. But it gets ever better. Sure we are testing that we show a fix-it to the user right now, but how do we test the applied fix-it? We had some gnarly logic for diving into a property’s bindings in order to add a type annotation, and so we would want to make sure all of that is correct.
Well, our library has just the tool for this. There is an optional argument that can be provided to assertMacro that will apply any fix-its emitted by a diagnostic. Let’s start a new test that is similar to the current one, but let’s delete the matches trailing closure and let’s provide the applyFixIts option:
func testPublicInit_HappyPath_FixItsApplied() {
assertMacro(applyFixIts: true) {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
"""
}
}
Now when we run this test the expanded macro code is inserted into the file, but there are no more diagnostics:
func testPublicInit_HappyPath_FixItsApplied() {
assertMacro(applyFixIts: true) {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int {
return b + 1
}
}
"""
} matches: {
"""
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c :<#Type#>= true
var b2: Int {
return b + 1
}
}
"""
}
}
Once the fix-it is applied, the diagnostic is no longer important, and so it goes away. And further we see exactly how our type annotation is going to appear in Xcode:
var c :<#Type#>= true
All a user has to do is click on that placeholder and provide the type information.
However, the spacing is a little funky. Technically this is completely valid Swift code, but if we wanted to expand to something a little more reasonable we could go the extra mile.
If we want to trim the whitespace around the c we can simply do this:
newProperty.bindings[newProperty.bindings.startIndex]
.pattern = newProperty.bindings[
newProperty.bindings.startIndex
]
.pattern.trimmed
And if we want to provide spacing around the type placeholder, we can do this:
newProperty.bindings[newProperty.bindings.startIndex]
.typeAnnotation = TypeAnnotationSyntax(
type: IdentifierTypeSyntax(name: " <#Type#> ")
)
Now when we re-run the test a new expansion is recorded with the spacing that is more reasonable:
var c: <#Type#> = true
So this is pretty incredible. We are not only getting test coverage on how fix-it diagnostics will appear in Xcode, but we are even getting test coverage on how those fix-its will be applied in the final source code. And on top of all of that testing these nuanced features is incredible easy. No need to copy-and-paste strings from test failure messages just to get formatting right, and no need to mentally track what line and column numbers correspond to what diagnostics. It’s all just visually shown to us in a very nice format.
We’ve now spent a lot of time implementing the feature and writing a test for it, but we haven’t even seen how it would work in practice. To do that we can jump over to the PublicInit.swift file in the MacroKitClient target, which is a little playground space for using macros in an executable.
Ian already has some sample usage of this macro, and he even calls out a property that would not be allowed since it doesn’t have a type annotation:
import Foundation
import MacroKit
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
//var c = true // @PublicInit requires stored properties provide explicit type annotations
var b2: Int { b + 1 }
}
Well, let’s bring back that property:
import Foundation
import MacroKit
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c = true
var b2: Int { b + 1 }
}
We immediately get a compile error, but it also comes with a handy fix-it:
@PublicInit requires stored properties provide explicit type annotations
Insert type annotation.
If we click the “Fix” button a type placeholder is inserted into the source code:
@PublicInit
public struct Foo {
var a: String
private var b: Int = 42
var c: <#Type#> = true
var b2: Int { b + 1 }
}
Absolutely incredible.
So, this is all looking pretty great. Thanks to our new testing tool we can write very deep and nuanced tests of macros and their diagnostics in a super lightweight way.
And it’s so easy to write tests with this tool that it can empower us to write tests for every little edge case in our macro. This is incredibly important because macros are quite complex, both because it transforms a large abstract syntax tree into a whole new one, but also because the syntax tree itself is complex and requires a lot of work to assert against.
And this is just the beginning of our explorations of macros on Point-Free. Macros are going to fundamentally change many of our popular libraries, such as case paths and the Composable Architecture, but we will have to discuss that another time.
Until next time!
0251-macro-testing-pt2