Scala Testing with Scalatest: Reference and Examples

Scala Testing with Scalatest: Reference and Examples

Last updated:
Scala Testing with Scalatest: Reference and Examples
Source
Table of Contents

TL;DR: If you just want some working code, just do the following:

  • add libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.3" % "test" to build.sbt

  • create a file under <projectRoot>/src/test/scala/SimplestPossibleSpec.scala with the following content:

    import org.scalatest._

class SimplestPossibleSpec extends FlatSpec {

  "An empty Set" should "have size 0" in {
    assert(Set.empty.size == 0)
  }
}

  • run $ sbt test on the project root

Introduction

Unless otherwise stated, we're using Scalatest 3.x and SBT 1.2.x (all examples can be found here)

Scalatest is just one among many testing libraries for scala but it's quite complete and a lot of people use it so it's a good choice for starting out with.

Scalatest support many different testing styles under the same testing framework, for example:

  • XUnit - testing style made popular by tools such as JUnit, NUnit, etc.

  • Behaviour-Driven Development (BDD) - testing style made popular by (among others) RSpec and Cucumber for Ruby.

    The focus here is on high-level behaviour; this is inspired by things such as Scrum Stories and Domain-Driven Design (DDD)

  • Property-based Testing - automatically generating multiple combinations of test cases to test. Made popular by Haskell's QuickCheck library.

  • Web-based Testing - this can be done using tools such as Selenium and/or plugins for specific technologies such as akka-http.

Each of these styles is enabled by adding specific Traits to your base test class.

Add scalatest to SBT

Add to build.sbt

libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.3" % "test"

No test style, just asserts

You can import statements individually even without a style

This example does not mean that you should put test code in your main class, it just shows that it's possible to just import the assertions individually, anywhere in your code

The barely minimal way to use Scalatest is not to use any testring style at all.

I don't think this is a good idea but you can, if you want, just import individual assertions and use it the way you would use any other library code.

import org.scalatest.Assertions._

object MainTest extends App {

  // will pass
  assert("foo" == "foo")    

  // will fail  
  assertResult("foo") {
    "bar".toUpperCase
  }

}

Then run it with something like $ sbt "runMain path.to.MainTest"

FunSuite

FunSuite is the simplest style for XUnit-style, low-level testing.

all sorts of other assertions are also available

This is like your usual XUnit testing framework; organize test cases in test(){} blocks and use assertions to define actual test behaviour.

import org.scalatest.FunSuite

class SampleFunSuite extends FunSuite {

  val lst = List.empty[String]
  val arr = Array("foo", "bar")

  test("test list methods") {
    assert(lst.size == 0)
  }

  test("test array methods") {
    assert(arr(0) == "foo")
    assertThrows[ArrayIndexOutOfBoundsException] {
      arr(13)
    }
  }
}

You can add (use with caution) side-effects before and after each and every test case, using the BeforeAndAfter trait:

before and after are a special case of something called fixtures

import org.scalatest.{BeforeAndAfter, FunSuite}

// this is just a template .. it won't compile
class SampleFunSuiteWithBeforeAndAfter extends FunSuite with BeforeAndAfter {

  var dbConnection: DBConnection = null

  before{
    dbConnection = new DbConnection(....)
  }

  test("test insert into database"){
    dbConnection.execute("""insert into .... "foo bar" where ... """)
    assert(dbConnection.query("select ..... where ...") == "foo bar")
  }

  after{
    dbConnection.close()
  }
}

FlatSpec

FlatSpec is still low-level but more versatile.

In fact, this is probably the easiest testing style to start with, if you're not sure what your want.

SampleFlatSpec.scala

import collection.mutable.Stack
import org.scalatest._

// Trait `Matchers` enables constructs like `should be()` 
class SampleFlatSpec extends FlatSpec with Matchers {

  "A Stack" should "pop values in last-in-first-out order" in {
    val stack = new Stack[Int]
    stack.push(1)
    stack.push(2)
    stack.pop() should be (2)
    stack.pop() should be (1)
  }

  // using `it` is an alias to the previous name (in this case, "A Stack")
  it should "throw NoSuchElementException if an empty stack is popped" in {
    val emptyStack = new Stack[Int]
    a [NoSuchElementException] should be thrownBy {
      emptyStack.pop()
    } 
  }
}

FunSpec

Use FunSpec for BDD-like, higher-level testing

The main idea in BDD-style testing is that testing should work both as, erm, testing, and as specifications.

So specs are statements that can be used both for actually testing code and as documentation.

This is good because it reduces the work to keep documentation up to date and forces you to write tests, or things will go undocumented.

Say we have a class named Person, and a companion object with a factory method.

case class Person(firstName: String, lastName: String, age: Integer) {

  require(age >= 0, "age should be a positive integer")

  def getFullName = firstName.toLowerCase.capitalize + " " + lastName.toLowerCase.capitalize
}

here is an example of the code used to test that:

SampleFunSpec.scala

class SampleFunSpec extends FunSpec {

  describe("a person") {
    describe("when being created") {

      it("should throw an exception if it's given a zero or negative age") {
        assertThrows[IllegalArgumentException] {
          Person.mkPerson("john", "doe", -10)
        }
      }

      it("correctly formats first and last names") {
        assertResult("John Doe") {
          Person.mkPerson("john", "doe", 10).getFullName
        }
      }
    }
  }
}

Property-based Testing with Scalacheck

To use property-based testing, you need to add an extra dependency to build.sbt: libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.13.4"

Property-based testing is a style of testing that forces you to think about what are the invariants in your code.

ScalaTest generates test cases automatically for some property that you define should always hold.

You get extra keywords like forAll, that validate that the enclosed block works for all possible variations of the given type (in the example below, all sorts of different and weird Strings are used to test your code)

SamplePropertyBasedTest.scala

import org.scalatest.FunSuite
import org.scalatest.prop.PropertyChecks

// this uses the Person class defined above

// see how we can use our friend FunSuite (which adds basic
// stuff like `test()` and so on) and augment the test class
// with property-based testing facilities:
class SamplePropertyBasedTest extends FunSuite with PropertyChecks {

  test("full name matches first and last name") {
    // forAll actually uses a ton of different string
    // variations to make sure all edge cases are tested
    forAll { (a: String, b: String) =>
      assert {
        Person(a, b, 10).getFullName.toLowerCase.startsWith(a.toLowerCase) &&
        Person(a, b, 10).getFullName.toLowerCase.endsWith(b.toLowerCase)
      }
    }
  }
}

Now you might think this is a waste of time (we just defined the method to merge the two string, of course it will always hold) but think about complex cases where you want to sanity test your system and you want to make sure that any input combination is handled by your function, module, or system.

Run all tests

Go to the root of the project and enter sbt then test.

This will run all tests under src/test/scala/.

$ sbt
> test
[info] SampleFunSuiteWithBeforeAndAfter:
[info] SampleFunSuite:
[info] - test list methods
[info] - test array methods
[info] SampleFunSpec:
[info] a person
[info]   when being created
[info]   - should throw an exception if it's given a zero or negative age
[info]   - correctly formats first and last names
[info] SimplestPossibleSpec:
[info] An empty Set
[info] - should have size 0
[info] SampleFlatSpec:
[info] A Stack
[info] - should pop values in last-in-first-out order
[info] - should throw NoSuchElementException if an empty stack is popped
[info] SamplePropertyBasedTest:
[info] - first and last names
[info] ScalaTest
[info] Run completed in 321 milliseconds.
[info] Total number of tests run: 8
[info] Suites: completed 6, aborted 0
[info] Tests: succeeded 8, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 8, Failed 0, Errors 0, Passed 8

Run single test file

Go to the root of the project and enter sbt then testOnly path.to.TestClass.

$ sbt
> testOnly SampleFlatSpec
[info] SampleFlatSpec:
[info] A Stack
[info] - should pop values in last-in-first-out order
[info] - should throw NoSuchElementException if an empty stack is popped
[info] ScalaTest
[info] Run completed in 191 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 2, Failed 0, Errors 0, Passed 2
[success] Total time: 0 s, completed 02/09/2019 23:50:02

Run single test case

Use testOnly path.to.TestClass -- -z "text to match".

Only test cases matching "text to match" will be run.

In the following example, only the test case whose text matches "should pop values" in class SampleFlatSpec will be run:

$ sbt
> testOnly SampleFlatSpec -- -z "should pop values"
[info] SampleFlatSpec:
[info] A Stack
[info] - should pop values in last-in-first-out order
[info] ScalaTest
[info] Run completed in 151 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 0 s, completed 03/09/2019 00:00:56

scala-test-sbt-single-test-case Only the highlighted text will run, because it
contains the string "should pop values"

Extras

Extra: Use Scalatest to Test Akka-http REST Routes

For this example, view the code (along with a full-fledged akka-http project example) here

All sorts of Scala frameworks have ScalaTest plugins. They're usually implemented as Traits, that you should add to your test class.

To test that routes (HTTP endpoints) behave as they should in akka-http based systems, add "com.typesafe.akka" %% "akka-http-testkit" % "10.0.9" to your build dependencies and have your class extend ScalatestRouteTest.

import org.scalatest.{ Matchers, WordSpec }
import akka.http.scaladsl.model.StatusCodes
import akka.http.scaladsl.testkit.ScalatestRouteTest
import akka.http.scaladsl.server._
import Directives._

// ScalatestRouteTest is a Trait that brings the akka-http
// tesing DSL into the scope of this class
class FullTestKitExampleSpec extends WordSpec with Matchers with ScalatestRouteTest {

  // in actual apps, you would probably define your routes
  // elsewhere and import them here
  val smallRoute =
    get {
      pathSingleSlash {
        complete {
          "Captain on the bridge!"
        }
      } ~
      path("ping") {
        complete("PONG!")
      }
    }

  "The service" should {

    "return a greeting for GET requests to the root path" in {
      // tests:
      Get() ~> smallRoute ~> check {
        // parsing the reponse as a string and asserting
        responseAs[String] shouldEqual "Captain on the bridge!"
      }
    }

    "return a 'PONG!' response for GET requests to /ping" in {
      // tests:
      Get("/ping") ~> smallRoute ~> check {
        responseAs[String] shouldEqual "PONG!"
      }
    }

}

You can see that all Akka-http route tests follow the following template:

REQUEST ~> ROUTE ~> check {
  ASSERTIONS
}

References

testing scala

Archive

Dialogue & Discussion