Unit testing Rust Syn crate

Question

From the Syn documentation:

Syn operates on the token representation provided by the proc-macro2 crate from crates.io rather than using the compiler's built in proc-macro crate directly. This enables code using Syn to execute outside of the context of a procedural macro, such as in unit tests or build.rs

I am trying to enable unit testing for some Syn functions, however I can't get it to work no matter what I have tried. It does not work with the proc_macro2::TokenStream type, but it won't work with the proc_macro::TokenStream because we are not in a proc-macro context.

link to playground

use quote::quote;
use syn;

fn test() {
    // let stream: syn::export::TokenStream = quote!{fn foo() {};}.into(); // doesn't work
    let stream: proc_macro2::TokenStream = quote!{fn foo() {};}.into(); // doesn't work
    // let item = parse_macro_input!(stream as Item); // doesn't work
    let item = syn::parse(stream).unwrap();

}

fn main() {
    test();
}

Any help on how to test syn functions outside of the proc-macro context would be appreciated. I am aware of the trybuild crate, but I would like to be able to unit test the macro's functions first.

Answer 1

It does not work with the proc_macro2::TokenStream type, but it won't work with the proc_macro::TokenStream because we are not in a proc-macro context.

Yes, and that's the whole point! Crates that export procedural macros can't export anything else, but proc_macro can only be used in crates that export macros. This is the reason why proc_macro2 exists in the first place.

You need to use multiple crates in order to write tests for code that uses syn and proc_macro2:

• Your public crate that declares the macros with #[proc_macro] etc., and does very little except convert a proc_macro::TokenStream into a proc_macro2::TokenStream and vice versa.
• An "internal" crate, containing most of the actual code, which depends on proc_macro2 but not proc_macro. Your tests can go in here.

---

The error you are seeing is because syn::parse accepts a proc_macro::TokenStream. You can instead use syn::parse2, which is identical except that it accepts a proc_macro2::TokenStream.