Writing Tests

Most parts of the Spade compiler can be tested with one of three macros:

code_compiles!
snapshot_error!
snapshot_mir!

You can get examples of how they work in the spade-tests crate where you should write most tests. All three macros take a test name and a snippet of source code which should be tested. code_compiles simply tests to make sure that there are no compile errors when compiling the snippet.

The other two are snapshot tests, meaning that they expect some output from the compiler, but the expected output is not manually written. Instead, every time the test is run, its output is compared to the previous time the test was run. If there is a difference, you can accept or reject that change. snapshot_error is used to ensure that error messages are helpful and don’t change, and is also the preferred way to ensure that errors are emitted when they should be. While snapshot_error could be used as a code_compiles macro by simply accepting no error message as the correct output, it is better to communicate the intent of the test by using code_compiles

snapshot_mir is a snapshot error on the output of the compiler. Or rather, on the MIR output which is the final step emitted before Verilog. Use this when the goal is to test that the output of some new construct is correct. However, because the snapshot_mir tests often have variable names get changed, they should only be used when the output is likely to be relevant. For example, don’t use snapshot_mir to make sure that a new type inference feature doesn’t trigger an error, instead use code_compiles

Reviewing snapshots

To run the test suite, simply cargo test. If any snapshot tests fail, it will tell you, and you now have to review the changes. This is done with a tool called insta https://insta.rs/docs/cli/. Install it, and then run

cargo insta review

to go through the failed tests and accept or reject the changes.

If only a few tests failed and the results all look good, you can run

cargo insta accept

to accept everything, or

cargo insta reject

to reject all the failures.

Swim Tests / Testing the Generated Code

If you want to test your changes in a real project and run Verilog simulation on the generated code, you can do so by adding a test to the swim_tests project in the Spade repo. This requires adding the code to test to src/, and a python testbench to the tests/ directory. Then you can simply run swim test to run all the tests.

Testing Your Changes During Development

You can run the Spade compiler directly on a test file with

cargo run --bin spade -- -o /tmp/test.sv test.spade --omit-stdlib

However, if you have just a small single file test, it is better to use one of the test macros as that gives a test for your feature for free once you’re done!

If you want to use your new compiler in a project for testing, there are a few ways. First, you can edit swim.toml to set

compiler.git = "<your git repo URL>"
compiler.branch = "<your branch name>"

then run swim update-spade

This is useful if you have a project that needs your feature but it hasn’t been merged yet, but has the downside of requiring a push for every change.

For testing locally, you can run

swim --override-compiler <path to your compiler> <your normal swim command>

Or, you can set the path to the compiler in your project

compiler.path = "<path to your compiler>"