prove

by fukamachi

fukamachi / prove

Yet another unit testing framework for Common Lisp

210 Stars 21 Forks Last release: Not found 167 Commits 1 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

prove

This project was originally called 'CL-TEST-MORE'.

'prove' is yet another unit testing framework for Common Lisp.

The advantages of 'prove' are:

Quickstart

1. Writing a test file

(in-package :cl-user)
(defpackage my-test
  (:use :cl
        :prove))
(in-package :my-test)

(plan 3)

(ok (not (find 4 '(1 2 3)))) (is 4 4) (isnt 1 #\1)

(finalize)

2. Run a test file

(prove:run #P"myapp/tests/my-test.lisp")
(prove:run #P"myapp/tests/my-test.lisp" :reporter :list)

See also: ASDF integration, Reporters

3. Get a report

Installation

You can install 'prove' via Quicklisp.

(ql:quickload :prove)

Testing functions

(ok test &optional desc)

Checks if

test
is true (non-NIL).
(ok 1)
;->  ✓ 1 is expected to be T

(is got expected &rest test-args)

Checks if

got
is equivalent to
expected
.
(is 1 1)
;->  ✓ 1 is expected to be 1

(is #(1 2 3) #(1 2 3)) ;-> × #(1 2 3) is expected to be #(1 2 3)

(is #(1 2 3) #(1 2 3) :test #'equalp) ;-> ✓ #(1 2 3) is expected to be #(1 2 3)

;; with description (is 1 #\1 "Integer = Character ?") ;-> × Integer = Character ?

(isnt got expected &rest test-args)

Checks if

got
is not equivalent to
expected
.
(isnt 1 1)
;->  × 1 is not expected to be 1

(isnt #(1 2 3) #(1 2 3)) ;-> ✓ #(1 2 3) is not expected to be #(1 2 3)

(is-values got expected &rest test-args)

Checks if the multiple values of

got
is equivalent to
expected
. This is same to
(is (multiple-value-list got) expected)
.
(defvar *person* (make-hash-table))

(is-values (gethash :name person) '("Eitaro" T)) ;-> × (NIL NIL) is expected to be ("Eitaro" T)

(setf (gethash :name person) "Eitaro")

(is-values (gethash :name person) '("Eitaro" T)) ;-> ✓ ("Eitaro" T) is expected to be ("Eitaro" T)

(is-type got expected-type &optional desc)

Checks if

got
is a type of
expected-type
.
(is-type #(1 2 3) 'simple-vector)
;->  ✓ #(1 2 3) is expected to be a type of SIMPLE-VECTOR (got (SIMPLE-VECTOR 3))

(is-type (make-array 0 :adjustable t) 'simple-vector) ;-> × #() is expected to be a type of SIMPLE-VECTOR (got (VECTOR T 0))

(like got regex &optional desc)

Checks if

got
matches a regular expression
regex
.
(like "Hatsune 39" "\\d")
;->  ✓ "Hatsune 39" is expected to be like "\\d"

(like "初音ミク" "\d") ;-> × "初音ミク" is expected to be like "\d"

(is-print got expected &optional desc)

Checks if

got
outputs
expected
to
*standard-output*
(is-print (princ "Hi, there") "Hi, there")
;->  ✓ (PRINC "Hi, there") is expected to output "Hi, there" (got "Hi, there")

(is-error form condition &optional desc)

Checks if

form
raises a condition and that is a subtype of
condition
.
(is-error (error "Something wrong") 'simple-error)
;->  ✓ (ERROR "Something wrong") is expected to raise a condition SIMPLE-ERROR (got #)

(define-condition my-error (simple-error) ())

(is-error (error "Something wrong") 'my-error) ;-> × (ERROR "Something wrong") is expected to raise a condition MY-ERROR (got #)

(is-expand got expected &optional desc)

Checks if

got
will be
macroexpand
ed to
expected
.
(is-expand (when T (princ "Hi")) (if T (progn (princ "Hi"))))
;->  ✓ (WHEN T (PRINC "Hi")) is expected to be expanded to (IF T
;                                                          (PROGN (PRINC "Hi"))) (got (IF T
;                                                                                         (PROGN
;                                                                                          (PRINC
;                                                                                           "Hi"))
;                                                                                         NIL))

If a symbol that starts with "$" is contained, it will be treated as a gensym.

(pass desc)

This will always be passed. This is convenient if the test case is complicated and hard to test with

ok
.
(pass "Looks good")
;->  ✓ Looks good

(fail desc)

This will always be failed. This is convenient if the test case is complicated and hard to test with

ok
.
(fail "Hopeless")
;->  × Hopeless

(skip how-many why)

Skip a number of

how-many
tests and mark them passed.
(skip 3 "No need to test these on Mac OS X")
;->  ✓ No need to test these on Mac OS X (Skipped)
;    ✓ No need to test these on Mac OS X (Skipped)
;    ✓ No need to test these on Mac OS X (Skipped)

(subtest desc &body body)

Run tests of

body
in a new sub test suite.
(subtest "Testing integers"
  (is 1 1)
  (is-type 1 'bit)
  (is-type 10 'fixnum))
;->      ✓ 1 is expected to be 1
;        ✓ 1 is expected to be a type of BIT (got BIT)
;        ✓ 10 is expected to be a type of FIXNUM (got (INTEGER 0 4611686018427387903))
;->  ✓ Testing integers

Other functions

(diag desc)

Outputs

desc
to a
*test-result-output*
.
(diag "Gonna run tests")
;-> # Gonna run tests

(plan num)

Declares a number of

num
tests are going to run. If
finalize
is called with no
plan
, a warning message will be output.
num
is allows to be
NIL
if you have no plan yet.

(finalize)

Finalizes the current test suite and outputs the test reports.

(slow-threshold milliseconds)

Set the threshold of slow test durations for the current test suite. The default threshold value is

prove:*default-slow-threshold*
.
(slow-threshold 150)

Reporters

You can change the test report formats by setting

prove:*default-reporter*
to
:list
,
:dot
,
:tap
or
:fiveam
. The default value is
:list
.

prove:run
also takes a keyword argument
:reporter
.

List (Default)

The

:list
repoter outputs test results list as test cases pass or fail.

Dot

The

:dot
reporter outputs a series of dots that represent test cases, failures highlight in red, skipping in cyan.

FiveAM

The

:fiveam
reporter outputs test results like FiveAM does.

TAP

The

:tap
reporter outputs in Test Anything Protocol format.

Tips

Debugging with CL debugger

Set

prove:*debug-on-error*
T for invoking CL debugger whenever getting an error during running tests.

Colorize test reports on SLIME

SLIME doesn't support to color with ANSI colors in the REPL buffer officially.

You can add the feature by using slime-repl-ansi-color.el.

After installing it, set

prove:*enable-colors*
to
T
before running tests.
;; A part of my ~/.sbclrc
(ql:quickload :prove)
(setf prove:*enable-colors* t)

The following snippet is a little bit complicated, however it would be better if you don't like to load

prove
in all sessions.
(defmethod asdf:perform :after ((op asdf:load-op) (c (eql (asdf:find-system :prove))))
  (setf (symbol-value (intern (string :*enable-colors*) :prove)) t))

ASDF integration

Add

:defsystem-depends-on (:prove-asdf)
to your testing ASDF system to enable
:test-file
in the
:components
.

:test-file
is same as
:file
except it will be loaded only when
asdf:test-system
.
;; Main ASDF system
(defsystem my-app

;; ...

:in-order-to ((test-op (test-op my-app-test))))

;; Testing ASDF system (defsystem my-app-test :depends-on (:my-app :prove) :defsystem-depends-on (:prove-asdf) :components ((:test-file "my-app")) :perform (test-op :after (op c) (funcall (intern #.(string :run) :prove) c)))

To run tests, execute

asdf:test-system
or
prove:run
in your REPL.
(asdf:test-system :my-app)
(asdf:test-system :my-app-test)

;; Same as 'asdf:test-system' except it returns T or NIL as the result of tests. (prove:run :my-app-test)

Changing default test function

Test functions like

is
uses
prove:*default-test-function*
for testing if no
:test
argument is specified. The default value is
#'equal
.

Changing output stream

Test reports will be output to

prove:*test-result-output*
. The default value is
T
, which means
*standard-output*
.

Running tests on Travis CI

Although Common Lisp isn't supported by Travis CI officially, you can run tests by using cl-travis.

Here's a list of

.travis.yml
from projects using
prove
on Travis CI:

Bugs

Please report any bugs to [email protected], or post an issue to GitHub.

License

Copyright (c) 2010-2014 Eitaro Fukamachi <[email protected]>
'prove' and CL-TEST-MORE is freely distributable under the MIT License (http://www.opensource.org/licenses/mit-license).

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.