Integration testing framework using a state monad in the backend for building and composing flows
StateFlow is a testing framework designed to support the composition and reuse of individual test steps.
A flow is a sequence of steps or bindings to be executed with some state as a reference. Use the
flowmacro to define a flow:
(flow *)
Once defined, you can run it with
(state-flow.api/run* (flow ...)).
You can think flows and the steps within them as functions of the state, e.g.
(fn [] [, ])
Each step is executed in sequence, passing the state to the next step. The return value from running the flow is the return value of the last step that was run.
Primitive steps are the fundamental building blocks of flows.
(state-flow.api/get-state f)
(state-flow.api/swap-state f)
(state-flow.api/fmap xform )
(state-flow.api/return v)
(state-flow.api/invoke no-arg-fn)
Bindings bind return values of steps to symbols you can use in other steps.
[( )+]
They are like
letbindings but the symbol on the left binds to the return value of the step on the right.
[ ]
You can also bind directly to values using the
:letkeyword:
[:let [ ]]
You can bind any number of symbols in a single binding vector, e.g.
[a step-1 b step-2 :let [c expression-1] d step-3]
If you are using StateFlow for integration testing, the initial state is usually a representation of your service components, a system using Stuart Sierra's Component library or other similar facility. You can also run the same flow with different initial states, e.g.
(def a-flow (flow ...))(defn build-initial-state [] { ... }) (state-flow.api/run* {:init build-initial-state} flow)
(state-flow.api/run* {:init (constantly {:service-system (atom nil))} flow)
Flows follow the Composite Pattern: a single flow has the same interface as a collection of flows.
You can compose flows by nesting them in other flows:
(flow "do many things" (flow "do one thing" ,,,) (flow "do another thing" ,,,))
Use
state-flow.api/forwhen you have a flow that you'd like to apply to different inputs with the same outcome, e.g.
(flow "even? returns true for even numbers" (flow/for [x (filter even? (range 10))] (match? even? x)))
By default, a flow continues to be evaluated even if an assertion fails. The
:fail-fast?option to
state-flow.api/run*can be used if you would like to stop evaluation after the first assertion failure.
(state-flow.api/run* {:fail-fast? true} (flow "evaluation stops after `failing-flow-b`" flow-a failing-flow-b flow-c))
Suppose our system state is made out of a map with
{:value }. We can make a flow that just fetches the value bound to
:value.
(require '[state-flow.api :as flow :refer [flow]]) (def get-value (flow "get-value" (flow/get-state :value))) (flow/run* {:init (constantly {:value 4})} get-value) ; => [4 {:value 4}]
Primitive steps have the same underlying structure as flows and can be passed directly to
run*:
(def get-value (flow/get-state :value)) (flow/run* {:init (constantly {:value 4})} get-value) ; => [4 {:value 4}]
We can use
state-flow.api/swap-stateto modify the state. Here's a primitive that increments the value:
(def inc-value (flow/swap-state update :value inc)) (flow/run* {:init (constantly {:value 4})} inc-value) ; => [{:value 4} {:value 5}]
Bindings enable us to compose simple flows into more complex flows. If, instead of returning the value, we wanted to return the value multiplied by two, we could do it like this:
(def double-value (flow "get double value" [value get-value] (flow/return (* value 2)))) (flow/run* {:init (constantly {:value 4})} double-value) ; => [8 {:value 4}]
Or we could increment the value first and then return it doubled:
(def inc-and-double-value (flow "increment and double value" inc-value [value get-value] (flow/return (* value 2)))) (flow/run* {:init (constantly {:value 4})} inc-and-double-value) ; => [10 {:value 5}]
We use the
defflowand
match?macros to build
clojure.testtests out of flows.
state-flow.cljtest.defflowdefines a test (using
deftest) that will execute the flow with the parameters that we set.
state-flow.assertions.matcher-combinators/match?produces a flow that will make an assertion, which will be reported via clojure.test when used within a
defflow. It uses the
nubank/matcher-combinatorslibrary for the actual check and failure messages.
match?asks for:
match?will wrap it in
(state-flow.api/return )
:times-to-try(default 1)
:sleep-time(default 200)
Here are some very simple examples of tests defined using
defflow:
(defflow my-flow (match? 1 1) (match? {:a 1} {:a 1 :b 2}))
Wrap them in
flows to get descriptions when the expected and actual values need some explanation:
(deftest fruits-and-veggies (flow "surprise! Tomatoes are fruits!" (match? #{:tomato} (fruits #{:tomato :potato}))))
Or with custom parameters:
(defflow my-flow {:init aux.init! :runner (comp run* s/with-fn-validation)} (match? 1 1))
(defflow my-flow {:init (constantly {:value 1 :map {:a 1 :b 2}})} [value (flow/get-state :value)] (match? 1 value) (flow "uses matcher-combinator embeds" (match? {:b 2} (flow/get-state :map)))
:times-to-tryand
:sleep-time
By default,
match?will evaluate
actualonly once. For tests with asynchrony/concurrency concerns, you can direct
match?to try up to
:times-to-trytimes, waiting
:sleep-timebetween each try. It will keep trying until it produces a value that matches the
expectedexpression, up to
:times-to-try.
(defflow add-data (flow "try up to 5 times with 250 ms between each try (total 1000ms)" (produce-message-that-causes-database-update) (match? expected-data-in-database (fetch-data) {:times-to-try 5 :sleep-time 250})))
We introduced
state-flow.assertions.matcher-combinators/match?in state-flow-2.2.4, and deprecated
state-flow.cljtest.match?in that release. The signature for the old version was
(match? ). We removed the description because it was quite common for the description to add no context that wasn't already made clear by the expected and actual values.
We also reversed the order of expected and actual in order to align with the
match?function in the matcher-combinators library and with clojure.test's
(is (= expected actual)).
We also added a script to help refactor this for you. Here's how you use it:
# if you don't already have the state-flow repo cloned git clone https://github.com/nubank/state-flow.git ;; or git clone [email protected]:nubank/state-flow.git ;; then cd state-flowif you already have the state-flow repo cloned
cd state-flow git co master git pull
the rest is the same either way
lein pom # needed for tools.deps to recognize this repo as a
:local/root
dependency ./bin/refactor-match.sh --help ;; now follow the instructions
Note that if you have a
defflowdefined in a different namespace, and it depends on
state-flow.cljtest, you may need to require it in that namespace.
We use
verifyto write midje tests with StateFlow.
verifyis a function that of three arguments: a description, a value or step, and another value or midje checker. It produces a step that, when executed, verifies that the second argument matches the third argument. It replicates the functionality of a
factfrom midje. In fact, if a simple value is passed as second argument, what it does is simply call
factinternally when the flow is executed.
verifyreturns a step that will make the check and return something. If the second argument is a value, it will return this argument. If the second argument is itself a step, it will return the last return value of the step that was passed. This makes it possible to use the result of verify on a later part of the flow execution if that is desired.
Say we have a step for making a POST request that stores data in datomic (
store-data-request), and we also have a step that fetches this data from db (
fetch-data). We want to check that after we make the POST, the data is persisted:
(:require [state-flow.api :refer [flow]] [state-flow.midje :refer [verify]])(defn stores-data-in-db [data] (flow "save data" (store-data-request data) [saved-data (fetch-data)] (verify "data is stored in db" saved-data expected-data)))
Test helpers specific to your domain can make state-flow tests more readable and intention-revealing. When writing them, we recommend that you start with state-flow functions in the
state-flow.apinamespace. If, for example, you're testing a webapp, you might want a
requesthelper like this:
(defflow users (flow "fetch registered users" (http-helpers/request {:method :post :uri "/users" :body {:user/first-name "David"}}) [users (http-helpers/request {:method :get :uri "/users"})] (match? ["David"] (map :user/first-name users)))
Presuming that you have an
:http-componentkey in the initial state, the
http-helpers/requesthelper could be implemented something like this:
(ns http-helpers (:require [my-app.http :as http] [state-flow.api :as flow :refer [flow]]))(defn request [req] (flow "make request" [http (flow/get-state :http-component)] (flow/return (http/request http req)))
This produces a step that can be used in a flow, as above.
state-flowis built on the
funcool.catslibrary, which supports monads in Clojure.
state-flowexposes some, but not all,
catsfunctions as its own API. As mentioned above, we recommend that you stick with
state-flowfunctions as much as possible, however, if the available functions do not suit your need for a helper, you can always drop down to functions directly in the
catslibrary.
Add
"defflow"to the list defined by
cider-test-defining-formsto enable commands like
cider-test-run-testfor flows defined with
defflow.
See https://docs.cider.mx/cider/testing/runningtests.html#configuration