Day 29 - Tests

Spring 2023

Dr. Jared Joseph

Smith College

Overview

Timeline

What is a Test?
Automated Testing
Tools for Testing

Goal

To learn about and create automated tests for your package.

What is a Test?

A Test is …

Code to assure

the output you are getting

is the output you expect.

You Already Run Tests

We are testing constantly while writing new code.

Often we write a line, try it, get an error, adapt, and try again.

This works great while you are working on something new.

Recall this loops forever …

# make a print method for my "CoolClass" class
print.CoolClass = function(x){
  
  cat("B) You've got this!\n")
  print(x)
  
}

# test out new method
print(test_vec)

Yet this is fine…

# make a print method for my "CoolClass" class
print.CoolClass = function(x){
  
  cat("B) You've got this!\n")
  cat(x)
  
}

# test out new method
print(test_vec)

You Test While Writing Packages

You’ve (hopefully) also tested your package functions along the way.

This requires a few extra steps, but the process is the same.

Try, revise, try again …

Making Package Functions:

Write function.r
devtools::load_all()
Try it
Edit function.r
devtools::load_all()
Try again
etc.

The Problem with Manual Tests

You’re only doing them now.

What happens when you change something later? Will you come back and make sure everything still works?

Automated Testing

The Role of Automated Testing

Say you create a function for your package that you will use as an intermediate step …

test_vec = c(TRUE, FALSE, FALSE, TRUE, FALSE, FALSE, TRUE, FALSE)

Week 1

my_func_1 = function(x){
  return(letters[x])
}

out_1 = my_func_1(test_vec)
out_1

 [1] "a" "d" "g" "i" "l" "o" "q" "t" "w" "y"

my_func_2 = function(x){
  # test for vowels
  vowels = c("a", "e", "i", "o", "u") %in% x
  names(vowels) = c("a", "e", "i", "o", "u")
  
  return(vowels)
}

my_func_2(out_1)

    a     e     i     o     u 
 TRUE FALSE  TRUE  TRUE FALSE

Week 6

my_func_1 = function(x){
  return(LETTERS[x])
}

out_1 = my_func_1(test_vec)
out_1

 [1] "A" "D" "G" "I" "L" "O" "Q" "T" "W" "Y"

my_func_2 = function(x){
  # test for vowels
  vowels = c("a", "e", "i", "o", "u") %in% x
  names(vowels) = c("a", "e", "i", "o", "u")
  
  return(vowels)
}

my_func_2(out_1)

    a     e     i     o     u 
FALSE FALSE FALSE FALSE FALSE

Sanity Checks

Even simple tests can prevent heartache.

A simple test for our above example would be something like: “If all of my results are false, I should probably look into that.”

We can code these gut checks into actual tests that will be run whenever we check our package.

A simple test

# run my_func_2
out = my_func_2(out_1)
out

    a     e     i     o     u 
FALSE FALSE FALSE FALSE FALSE

if(sum(out) == 0){
  warning(
    "Something might be wrong with `my_func_2`.
     All results were false.")
}

Warning: Something might be wrong with `my_func_2`.
     All results were false.

Tools for Testing

Where Tests Live

tests is an “optional” directory which contains unit-tests for your code.

Unit tests essentially run your code and check the results against pre-determined outputs.

If you change something and one of these known outputs change, you know something broke.

R Project Name

.
├── R/
│   └── func1.R
├── man/
│   └── func1.Rd
├── inst/
│   ├── data/
│   │   └── example_data1.csv
│   └── other
├── vignettes/
│   └── v1.rmd
├── docs/
│   └── v1.html
├── tests/
│   └── testthat/
│       └── test-func1.R
├── NAMESPACE
├── DESCRIPTION
├── LICENSE
├── NEWS.md
├── README.Rmd 
├── .gitignore
└── .Rbuildignore

`testthat` Package

The testthat package integrates testing into the process of building your package.

Much like how roxygen2 gives us helpers for documentation, testthat gives us helpers for tests.

You can use usethis::use_test() to create a test file for the currently open function.

If I have the R/comma_split.R function open, I can use usethis::use_test() to create a test template.

usethis::use_test()

✔ Setting active project to 'final-project-test-team'
✔ Adding 'testthat' to Suggests field in DESCRIPTION
✔ Setting Config/testthat/edition field in DESCRIPTION to '3'
✔ Creating 'tests/testthat/'
✔ Writing 'tests/testthat.R'
✔ Writing 'tests/testthat/test-comma_split.R'
• Modify 'tests/testthat/test-comma_split.R'

This creates a matching tests/testthat/test-comma_split.R file.

Still Need to Make Tests Though

Much like help files, we have tools to scaffold for us, but we still need to build things ourselves.

Tests are just R code, but makes use of some specific functions.

Each of these functions is designed to compare the results of some code with a known answer.

Some tests you can run:

expect_equal()
expect_error()
expect_warning()
expect_message()
expect_s3_class()
expect_true()
expect_false()

Link: More here

An Example Test

Manual

# run my_func_2
out = my_func_2(out_1)
out

    a     e     i     o     u 
FALSE FALSE FALSE FALSE FALSE

if(sum(out) == 0){
  warning(
    "Something might be wrong with `my_func_2`.
     All results were false.")
}

Warning: Something might be wrong with `my_func_2`.
     All results were false.

`testthat`

# assuming my_func_2 is in package
out = my_func_2(out_1)

# test
testthat::test_that("Is my_func_2 working?", {
  
  # I expect at least 1 true
  testthat::expect_true(any(out))
})

── Failure ('<text>:8'): Is my_func_2 working? ─────────────────────────────────
any(out) is not TRUE

`actual`:   FALSE
`expected`: TRUE

Error in `reporter$stop_if_needed()`:
! Test failed

Note that when writing tests for a package, you don’t need testthat:: for everything. R will load it when it goes to test things.

Now Automated

Your tests will now run every time you use devtools::test() or devtools::check() your package.

Most of the time, things will work fine!

However, if anything ever does break, you will have an early warning, and clear hints on where and why things broke.

A working test

testthat::test_that("multiplication works", {
  testthat::expect_equal(2 * 2, 4)
})

Test passed 😀

Testing whole package

Testing Tips

Much like examples, tests need to be simple, yet realistic

They must also be completely self-contained.

They do have access to and data you store in inst/data though.

Tests should:

Be self-contained
Test only one thing at a time
Test the major functions in your package (aka have good coverage)
Be well commented (you only look at them when something goes wrong!)

Check Yourself

Your go-to tool for checking up on how your package is working is the devtools::check() function.

It will run all of your tests, and check some other basic milestones and see if your package is healthy.

You should check often to make sure you aren’t accumulating problems.

An Error I Expect

If you have been using the package::function() syntax in your functions (as you should be doing), you will get an error when you now run devtools::check()

This is because we have not added those packages to the dependancies for our package yet.

To do so, simply run usethis::use_package("PACKAGE NAME"). This will add it to your description file.

Code-Along

open function in package
`usethis::use_test()
make a simple test

testthat::test_that("multiplication works", {
  testthat::expect_equal(2 * 2, 4)
})

devtools::test()

For Next Time

Topic

Benchmarking

To-Do

Finish Worksheet
READ: 23 Measuring performance