Simulation and Testing

At this point, we're going to move away from the playground and install Spade locally so we can run a "real" flow. If you haven't already, go back and read the installation to install Swim, Spade and a simulator. You can skip synthesis tools for now.

Because it is time-consuming and difficult to debug hardware, most hardware projects use simulation to speed up the development process and ease debugging.

Tests are not written in Spade itself, instead they are written in Python using cocotb or C++ in Verilator. Cocotb is easier to set up and nicer to use but can be quite slow.

If you haven't already, install the tools by following the installation instructions

Cocotb

Any Python files in the test directory will be run with cocotb. The first line must be a comment on the form

# top = <path to unit under test>

where the path is relative to your project. I.e. if you have a unit called top in main.spade, this will be main::top

Tests are asynchronous functions annotated with @cocotb.test(), they take a single input which is the design under test.

When working with Spade, you generally want to be able to use Spade values rather than pure bit vectors. To do so, import SpadeExt from spade and instantiate SpadeExt class, passing the dut to hits constructor.

You can then access the inputs of your unit using .i.<input name> and the output using .o. If the output of the unit is a struct, you can refer to individual fields using .o.<field name>

As an example, consider this unit which computes a+b and a*b with a latency of one cycle:

struct Output {
    sum: int<9>,
    product: int<16>
}

pipeline(1) add_mul(clk: clock, a: int<8>, b: int<8>) -> Output {
        let result = Output$(
            sum: a+b,
            product: a*b
        );
    reg;
        result
}

A test bench for this module looks like this (this assumes that the Spade code is in src/cocotb_sample.spade. If this is not the case, adjust the # top=cocotb_sample::add_mul part to reflect your module name):

#top = cocotb_sample::add_mul

import cocotb
from spade import SpadeExt
from cocotb.clock import Clock
from cocotb.triggers import FallingEdge

@cocotb.test()
async def test(dut):
    s = SpadeExt(dut) # Wrap the dut in the Spade wrapper

    # To access unmangled signals as cocotb values (without the spade wrapping) use
    # <signal_name>_i
    # For cocotb functions like the clock generator, we need a cocotb value
    clk = dut.clk_i

    await cocotb.start(Clock(
        clk,
        period=10,
        units='ns'
    ).start())

    await FallingEdge(clk)

    s.i.a = "2"
    s.i.b = "3"
    await FallingEdge(clk)
    s.o.sum.assert_eq("5")
    s.o.product.assert_eq("6")

    s.i.a = "3"
    s.i.b = "2"
    await FallingEdge(clk)
    s.o.sum.assert_eq("5")
    s.o.product.assert_eq("6")

    s.i.a = "0"
    s.i.b = "1"
    await FallingEdge(clk)
    s.o.sum.assert_eq("1")
    s.o.product.assert_eq("0")

You can then run the test using swim test or swim t. If you want to run all tests in a specific file, run swim test <pattern>. All files which contain the pattern will be run, for example, swim test abc will run tests in both abc.py and cdeabc.py.

You can also run individual tests using -t <test name> though here, the name has to be exactly the name of the test, not a pattern

For more information on the cocotb api, refer to its documentation

Viewing the waveform

One cool thing about HDL simulation is that it tracks how all signals trace over time, allowing you to see exactly how your circuit behaves. These traces are called waveforms, and stored in a file which you can view with a waveform viewer.

Once all tests are run, Swim will print the result of each test, along with a .vcd file in which the waveform is stored:

...

ok   test/cocotb_sample.py 0/1 failed
 🭼 test ok [build/cocotb_sample_test/cocotb_sample.vcd]

For waveform viewers, we recommend https://surfer-project.org/ which was developed specifically for the Spade project, but you can also use GtkWave if you are already familiar with it.

Once the waveform viewer is installed, you can simply run surfer build/cocotb_sample_test/cocotb_sample.vcd or gtkwave build/cocotb_sample_test/cocotb_sample.vcd.

If you have a relatively modern terminal, Swim also supports clickable links for opening wave files for each test, but it needs an initial automated setup. Just run swim setup-links. Now Swim will print two clickable links, one for surfer and one for gtkwave:
ok   test/cocotb_sample.py 0/1 failed
🭼 test ok [build/cocotb_sample_test/cocotb_sample.vcd ([🏄] [🌊])]

Surfer also supports translation from the raw bit patterns to Spade types which makes debugging much easier. By far the easiest way to get this is with the clickable links mentioned above, but you can also run Surfer with this manually by running

surfer <path-to-vcd> --spade-state build/state.ron --spade-top path::to::top::module

Tips and Tricks

For now, &mut wires can not be read by the Spade API in cocotb. Similarly, tests can not be run on generic units. Therefore, it is often a good idea to define a wrapper function, typically called a harness around your units which use advanced features.

Verilator

C++ source files with the .cpp extension in the test directory will be simulated using Verilator. Like cocotb, these files consist of a set of test cases, but the test cases are defined using macros rather than attributes.

The Spade path to the top module is specified using a comment, i.e.

// top = path::to::module

where the path is relative to the current project.

After that, the Verilog name of the top module must be specified using a #define. Unless you know the details of how Verilator name mangling works, you almost certainly want to specify #[no_mangle] on your unit under test.

The last thing you need to do before defining your tests is to include <verilator_util.hpp>.

After all your test have been defined, end test file with MAIN which defines a main function which is compatible with Swim.

// top=main::main

#define TOP main
#include <verilator_util.hpp>


TEST_CASE(it_works, {
    // Your test code here
    return 0;
})

MAIN

Accessing inputs and outputs

Like the cocotb API, there is a wrapper around Spade types to allow easier interactions with your design.

As an example, consider testing the following code

struct SubStruct {
    b: int<10>,
    c: int<5>,
}

struct SampleOutput {
    a: Option<int<20>>,
    sub: SubStruct
}

#[no_mangle]
fn sample(a: Option<int<20>>, b: int<10>, c: int<5>) -> SampleOutput {
    SampleOutput(a, SubStruct(b, c))
}

In the TEST_CASE macro, you have access to two variables: dut and s. dut is the raw verilator interface around your module which. s is the Spade wrapper which has a field i for its inputs and o for its output.

You can set the value of inputs to your module with s.i->input_name = "<spade expression>", for example:

    s.i->a = "Some(5)";
    s.i->b = "10";
    s.i->c = "5";

Similarly, you can compare the output to a spade expression using s.o == "<spade expression>"

If your unit under test returns a struct, you can also access its fields and sub-fields as fields on the output struct, for example s.o->field->subfield == "<spade expression>".

Finally, to assert that an output value is what you expect, you can use the ASSERT_EQ macro which takes s.o or a subfield, and compares it against a Spade expression. The advantage of using this macro over a C++ assert is that you get a diff print, both with the Spade value and the underlying bits.

For example, tests the fields in our example look like:

    ASSERT_EQ(s.o, "SampleOutput$(a: Some(5), sub: SubStruct$(b: 10, c: 5))");
    ASSERT_EQ(s.o->a, "Some(5)");
    ASSERT_EQ(s.o->sub, "SubStruct$(b: 10, c: 5)");
    ASSERT_EQ(s.o->sub->b, "10");
    ASSERT_EQ(s.o->sub->c, "5");

Clock generation

Clocks need to be ticked manually in verilator. The spade clock type does not allow direct assignment, so the clock needs to be accessed via the Verilator dut. Spade mangles inputs names as <name>_i, so if you want to set clk, you would set dut->clk_i. Or you can mark the clock input with #[no_mangle]

The following code will tick the clock once

    dut->clk_i = 1;
    ctx->timeInc(1);
    dut->eval();
    dut->clk_i = 0;
    ctx->timeInc(1);
    dut->eval();

Since this is so common, it is helpful to define a macro for it:

#define TICK \
    dut->clk_i = 1; \
    ctx->timeInc(1); \
    dut->eval(); \
    dut->clk_i = 0; \
    ctx->timeInc(1); \
    dut->eval();

which can then be used like this:

    s.i->a = "5";
    s.i->b = "10";
    TICK;
    ASSERT_EQ(s.o, "15");

Alternative test directory

If desired, you can change the name of the test directory by specifying the new name in swim.toml as follows

[simulation]
testbench_dir = "not/test"

Unless you have good reason to do this, it is better to leave the default directory.

The Spade Programming Language