Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence.
Edsger W. Dijkstra, "The Humble Programmer" (1972)
Let's talk about how to test Rust code. What we will not be talking about is the right way to test Rust code. There are many schools of thought regarding the right and wrong way to write tests. All of these approaches use the same basic tools, and so we'll show you the syntax for using them.
test
`test` attributeAt its simplest, a test in Rust is a function that's annotated with the test
`testattribute. Let's make a new project with Cargo called
`
attribute. Let's make a new project with Cargo called adder
`adder`:
$ cargo new adder
$ cd adder
Cargo will automatically generate a simple test when you make a new project.
Here's the contents of src/lib.rs
`src/lib.rs`:
#[test] fn it_works() { }
Note the #[test]
`#[test]. This attribute indicates that this is a test function. It currently has no body. That's good enough to pass! We can run the tests with
`. This attribute indicates that this is a test function. It
currently has no body. That's good enough to pass! We can run the tests with
cargo test
`cargo test`:
$ cargo test
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
Cargo compiled and ran our tests. There are two sets of output here: one for the test we wrote, and another for documentation tests. We'll talk about those later. For now, see this line:
test it_works ... ok
Note the it_works
`it_works`. This comes from the name of our function:
fn it_works() {
We also get a summary line:
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
So why does our do-nothing test pass? Any test which doesn't panic!
`panic!passes, and any test that does
` passes,
and any test that does panic!
`panic!` fails. Let's make our test fail:
#[test] fn it_works() { assert!(false); }
assert!
`assert!is a macro provided by Rust which takes one argument: if the argument is
` is a macro provided by Rust which takes one argument: if the argument
is true
`true, nothing happens. If the argument is false, it
`, nothing happens. If the argument is false, it panic!
`panic!`s. Let's run
our tests again:
$ cargo test
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test it_works ... FAILED
failures:
---- it_works stdout ----
thread 'it_works' panicked at 'assertion failed: false', /home/steve/tmp/adder/src/lib.rs:3
failures:
it_works
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
thread '<main>' panicked at 'Some tests failed', /home/steve/src/rust/src/libtest/lib.rs:247
Rust indicates that our test failed:
test it_works ... FAILED
And that's reflected in the summary line:
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
We also get a non-zero status code:
$ echo $?
101
This is useful if you want to integrate cargo test
`cargo test` into other tooling.
We can invert our test's failure with another attribute: should_panic
`should_panic`:
#[test] #[should_panic] fn it_works() { assert!(false); }
This test will now succeed if we panic!
`panic!` and fail if we complete. Let's try it:
$ cargo test
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
Rust provides another macro, assert_eq!
`assert_eq!`, that compares two arguments for
equality:
#[test] #[should_panic] fn it_works() { assert_eq!("Hello", "world"); }
Does this test pass or fail? Because of the should_panic
`should_panic` attribute, it
passes:
$ cargo test
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
should_panic
`should_panictests can be fragile, as it's hard to guarantee that the test didn't fail for an unexpected reason. To help with this, an optional
` tests can be fragile, as it's hard to guarantee that the test
didn't fail for an unexpected reason. To help with this, an optional expected
`expectedparameter can be added to the
`
parameter can be added to the should_panic
`should_panic` attribute. The test harness will
make sure that the failure message contains the provided text. A safer version
of the example above would be:
#[test] #[should_panic(expected = "assertion failed")] fn it_works() { assert_eq!("Hello", "world"); }
That's all there is to the basics! Let's write one 'real' test:
fn main() { pub fn add_two(a: i32) -> i32 { a + 2 } #[test] fn it_works() { assert_eq!(4, add_two(2)); } }pub fn add_two(a: i32) -> i32 { a + 2 } #[test] fn it_works() { assert_eq!(4, add_two(2)); }
This is a very common use of assert_eq!
`assert_eq!`: call some function with
some known arguments and compare it to the expected output.
tests
`tests` moduleThere is one way in which our existing example is not idiomatic: it's
missing the tests
`tests` module. The idiomatic way of writing our example
looks like this:
pub fn add_two(a: i32) -> i32 { a + 2 } #[cfg(test)] mod tests { use super::add_two; #[test] fn it_works() { assert_eq!(4, add_two(2)); } }
There's a few changes here. The first is the introduction of a mod tests
`mod testswith a
` with
a cfg
`cfgattribute. The module allows us to group all of our tests together, and to also define helper functions if needed, that don't become a part of the rest of our crate. The
` attribute. The module allows us to group all of our tests together, and
to also define helper functions if needed, that don't become a part of the rest
of our crate. The cfg
`cfg` attribute only compiles our test code if we're
currently trying to run the tests. This can save compile time, and also ensures
that our tests are entirely left out of a normal build.
The second change is the use
`usedeclaration. Because we're in an inner module, we need to bring our test function into scope. This can be annoying if you have a large module, and so this is a common use of the
` declaration. Because we're in an inner module,
we need to bring our test function into scope. This can be annoying if you have
a large module, and so this is a common use of the glob
`globfeature. Let's change our
` feature. Let's change
our src/lib.rs
`src/lib.rs` to make use of it:
pub fn add_two(a: i32) -> i32 { a + 2 } #[cfg(test)] mod tests { use super::*; #[test] fn it_works() { assert_eq!(4, add_two(2)); } }
Note the different use
`use` line. Now we run our tests:
$ cargo test
Updating registry `https://github.com/rust-lang/crates.io-index`
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
It works!
The current convention is to use the tests
`testsmodule to hold your "unit-style" tests. Anything that just tests one small bit of functionality makes sense to go here. But what about "integration-style" tests instead? For that, we have the
` module to hold your "unit-style"
tests. Anything that just tests one small bit of functionality makes sense to
go here. But what about "integration-style" tests instead? For that, we have
the tests
`tests` directory
tests
`tests` directoryTo write an integration test, let's make a tests
`testsdirectory, and put a
` directory, and
put a tests/lib.rs
`tests/lib.rs` file inside, with this as its contents:
extern crate adder; #[test] fn it_works() { assert_eq!(4, adder::add_two(2)); }
This looks similar to our previous tests, but slightly different. We now have
an extern crate adder
`extern crate adderat the top. This is because the tests in the
` at the top. This is because the tests in the tests
`testsdirectory are an entirely separate crate, and so we need to import our library. This is also why
`
directory are an entirely separate crate, and so we need to import our library.
This is also why tests
`tests` is a suitable place to write integration-style tests:
they use the library like any other consumer of it would.
Let's run them:
$ cargo test
Compiling adder v0.0.1 (file:///home/you/projects/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/lib-c18e7d3494509e74
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
Now we have three sections: our previous test is also run, as well as our new one.
That's all there is to the tests
`testsdirectory. The
` directory. The tests
`tests` module isn't needed
here, since the whole thing is focused on tests.
Let's finally check out that third section: documentation tests.
Nothing is better than documentation with examples. Nothing is worse than
examples that don't actually work, because the code has changed since the
documentation has been written. To this end, Rust supports automatically
running examples in your documentation. Here's a fleshed-out src/lib.rs
`src/lib.rs`
with examples:
//! The `adder` crate provides functions that add numbers to other numbers. //! //! # Examples //! //! ``` //! assert_eq!(4, adder::add_two(2)); //! ``` /// This function adds two to its argument. /// /// # Examples /// /// ``` /// use adder::add_two; /// /// assert_eq!(4, add_two(2)); /// ``` pub fn add_two(a: i32) -> i32 { a + 2 } #[cfg(test)] mod tests { use super::*; #[test] fn it_works() { assert_eq!(4, add_two(2)); } }
Note the module-level documentation with //!
`//!and the function-level documentation with
` and the function-level
documentation with ///
`///. Rust's documentation supports Markdown in comments, and so triple graves mark code blocks. It is conventional to include the
`. Rust's documentation supports Markdown in comments,
and so triple graves mark code blocks. It is conventional to include the
# Examples
`# Examples` section, exactly like that, with examples following.
Let's run the tests again:
$ cargo test
Compiling adder v0.0.1 (file:///home/steve/tmp/adder)
Running target/adder-91b3e234d4ed382a
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Running target/lib-c18e7d3494509e74
running 1 test
test it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Doc-tests adder
running 2 tests
test add_two_0 ... ok
test _0 ... ok
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured
Now we have all three kinds of tests running! Note the names of the
documentation tests: the _0
`_0is generated for the module test, and
` is generated for the module test, and add_two_0
`add_two_0for the function test. These will auto increment with names like
`
for the function test. These will auto increment with names like add_two_1
`add_two_1` as
you add more examples.