Scala Testing with Scalatest: Reference and Examples

Last updated:
Scala Testing with Scalatest: Reference and Examples
Source

All files used can be found here

TL;DR: If you just want some working code, do this (file is available here)

  • add libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.3" % "test" to build.sbt

  • create a file under <projectRoot>/src/test/scala/SimplestPossibleSpec.scala with the following content:

    import org.scalatest._
    
    class SimplestPossibleSpec extends FlatSpec {
    
      "An empty Set" should "have size 0" in {
        assert(Set.empty.size == 0)
      }
    }
    
  • run $ sbt test

Introduction

Unless otherwise stated, we're using Scalatest 3.x and SBT 0.13.x

Scalatest is just one among many testing libraries for scala but it's quite complete and a lot of people use it so it's a good choice for starting out with.

Scalatest support many different testing styles under the same testing framework, for example:

  • XUnit - testing style made popular by tools such as JUnit, NUnit, etc.

  • Behaviour-Driven Development (BDD) - testing style made popular by (among others) RSpec and Cucumber for Ruby.

    The focus here is on high-level behaviour; this is inspired by things such as Scrum Stories and Domain-Driven Design (DDD)

  • Property-based Testing - automatically generating multiple combinations of test cases to test. Made popular by Haskell's QuickCheck library.

  • Web-based Testing - this can be done using tools such as Selenium and/or plugins for specific technologies such as akka-http.

Each of these styles is enabled by adding specific Traits to your base test class.

Adding to SBT

Add to build.sbt

libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.3" % "test"

Testing Styles: You can import statements individually even without a style:

This example does not mean that you should put test code in your main class, it just shows that it's possible to just import the assertions individually, anywhere in your code

The barely minimal way to use Scalatest is not to use any testring style at all.

I don't think this is a good idea but you can, if you want, just import individual assertions and use it the way you would use any other library code.

import org.scalatest.Assertions._

object MainTest extends App {

  // will pass
  assert("foo" == "foo")    

  // will fail  
  assertResult("foo") {
    "bar".toUpperCase
  }

}

Testing Styles: FunSuite is the simplest style for XUnit-style, low-level testing

all sorts of other assertions are also available

This is like your usual XUnit testing framework; organize test cases in test(){} blocks and use assertions to define actual test behaviour.

import org.scalatest.FunSuite

class SampleFunSuite extends FunSuite {

  val lst = List.empty[String]
  val arr = Array("foo", "bar")

  test("test list methods") {
    assert(lst.size == 0)
  }

  test("test array methods") {
    assert(arr(0) == "foo")
    assertThrows[ArrayIndexOutOfBoundsException] {
      arr(13)
    }
  }
}

You can add (use with caution) side-effects before and after each and every test case, using the BeforeAndAfter trait:

before and after are a special case of something called fixtures

import org.scalatest.{BeforeAndAfter, FunSuite}

// this is just a template .. it won't compile
class SampleFunSuiteWithBeforeAndAfter extends FunSuite with BeforeAndAfter {

  var dbConnection: DBConnection = null

  before{
    dbConnection = new DbConnection(....)
  }

  test("test insert into database"){
    dbConnection.execute("""insert into .... "foo bar" where ... """)
    assert(dbConnection.query("select ..... where ...") == "foo bar")
  }

  after{
    dbConnection.close()
  }
}

Testing Styles: FlatSpec is still low-level but more versatile

In fact, this is probably the easiest testing style to start with, if you're not sure what your want.

SampleFlatSpec.scala

import collection.mutable.Stack
import org.scalatest._

// Trait `Matchers` enables constructs like `should be()` 
class SampleFlatSpec extends FlatSpec with Matchers {

  "A Stack" should "pop values in last-in-first-out order" in {
    val stack = new Stack[Int]
    stack.push(1)
    stack.push(2)
    stack.pop() should be (2)
    stack.pop() should be (1)
  }

  it should "throw NoSuchElementException if an empty stack is popped" in {
    val emptyStack = new Stack[Int]
    a [NoSuchElementException] should be thrownBy {
      emptyStack.pop()
    } 
  }
}

Run tests via $ sbt test on the project's root directory:

$ sbt test
[info] - should throw NoSuchElementException if an empty stack is popped
[info] SampleFlatSpec:
[info] A Stack
[info] - should pop values in last-in-first-out order
[info] - should throw NoSuchElementException if an empty stack is popped
[info] Run completed in 203 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 14 s, completed 06/07/2017 03:50:18

Testing styles: Use FunSpec for BDD-like, higher-level testing

The main idea in BDD-style testing is that testing should work both as, erm, testing, and as specifications.

So specs are statements that can be used both for actually testing code and as documentation.

This is good because it reduces the work to keep documentation up to date and forces you to write tests, or things will go undocumented.

Say we have a class named Person, and a companion object with a factory method.

case class Person(firstName: String, lastName: String, age: Integer) {

  require(age >= 0, "age should be a positive integer")

  def getFullName = firstName.toLowerCase.capitalize + " " + lastName.toLowerCase.capitalize
}

here is an example of the code used to test that:

SampleFunSpec.scala

class SampleFunSpec extends FunSpec {

  describe("a person") {
    describe("when being created") {

      it("should throw an exception if it's given a zero or negative age") {
        assertThrows[IllegalArgumentException] {
          Person.mkPerson("john", "doe", -10)
        }
      }

      it("correctly formats first and last names") {
        assertResult("John Doe") {
          Person.mkPerson("john", "doe", 10).getFullName
        }
      }
    }
  }
}

Testing styles: Property-based Testing

To use property-based testing, you need to add an extra dependency to build.sbt: libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.13.4"

Property-based testing is a style of testing that forces you to think about what are the invariants in your code.

ScalaTest generates test cases automatically for some property that you define should always hold.

You get extra keywords like forAll, that validate that the enclosed block works for all possible variations of the given type (in the example below, all sorts of different and weird Strings are used to test your code)

SamplePropertyBasedTest.scala

import org.scalatest.FunSuite
import org.scalatest.prop.PropertyChecks

// this uses the Person class defined above

// see how we can use our friend FunSuite (which adds basic
// stuff like `test()` and so on) and augment the test class
// with property-based testing facilities:
class SamplePropertyBasedTest extends FunSuite with PropertyChecks {

  test("full name matches first and last name") {
    // forAll actually uses a ton of different string
    // variations to make sure all edge cases are tested
    forAll { (a: String, b: String) =>
      assert {
        Person(a, b, 10).getFullName.toLowerCase.startsWith(a.toLowerCase) &&
        Person(a, b, 10).getFullName.toLowerCase.endsWith(b.toLowerCase)
      }
    }
  }
}

Now you might think this is a waste of time (we just defined the method to merge the two string, of course it will always hold) but think about complex cases where you want to sanity test your system and you want to make sure that any input combination is handled by your function, module, or system.


Extras: Using Scalatest to Test Akka-http REST Routes

For this example, view the code (along with a full-fledged akka-http project example) here

All sorts of Scala frameworks have ScalaTest plugins. They're usually implemented as Traits, that you should add to your test class.

To test that routes (HTTP endpoints) behave as they should in akka-http based systems, add "com.typesafe.akka" %% "akka-http-testkit" % "10.0.9" to your build dependencies and have your class extend ScalatestRouteTest.

import org.scalatest.{ Matchers, WordSpec }
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.testkit.ScalatestRouteTest
import akka.http.scaladsl.server._
import Directives._

// ScalatestRouteTest is a Trait that brings the akka-http
// tesing DSL into the scope of this class
class FullTestKitExampleSpec extends WordSpec with Matchers with ScalatestRouteTest {

  // in actual apps, you would probably define your routes
  // elsewhere and import them here
  val smallRoute =
    get {
      pathSingleSlash {
        complete {
          "Captain on the bridge!"
        }
      } ~
      path("ping") {
        complete("PONG!")
      }
    }

  "The service" should {

    "return a greeting for GET requests to the root path" in {
      // tests:
      Get() ~> smallRoute ~> check {
        // parsing the reponse as a string and asserting
        responseAs[String] shouldEqual "Captain on the bridge!"
      }
    }

    "return a 'PONG!' response for GET requests to /ping" in {
      // tests:
      Get("/ping") ~> smallRoute ~> check {
        responseAs[String] shouldEqual "PONG!"
      }
    }

}

You can see that all Akka-ttp route tests follow the following template:

REQUEST ~> ROUTE ~> check {
  ASSERTIONS
}

References

Dialogue & Discussion