hato

An HTTP client for Clojure, wrapping JDK 11's HttpClient.

It supports both HTTP/1.1 and HTTP/2, with synchronous and asynchronous execution modes as well as websockets.

In general, it will feel familiar to users of http clients like clj-http. The API is designed to be idiomatic and to make common tasks convenient, whilst still allowing the underlying HttpClient to be configured via native Java objects.

Status

hato has a stable API and is used in production for both synchronous and asynchronous use cases. Please try it out and raise any issues you may find.

Installation

hato requires JDK 11 and above. If you are running an older version of Java, please look at clj-http.

For Leinengen, add this to your project.clj

[hato "0.8.0"]

Quickstart

The main client is available in

hato.client

Require it to get started and make a request:

(ns my.app
  (:require [hato.client :as hc]))

  (hc/get "https://httpbin.org/get")
  ; =>
  ; {:request-time 112
  ;  :status 200
  ;  :body "{"url" ...}"
  ;  ...}

Usage

Building a client

Generally, you want to make a reusable client first. This will give you nice things like persistent connections and connection pooling.

This can be done with

build-http-client

; Build the client
(def c (hato/build-http-client {:connect-timeout 10000
                                :redirect-policy :always}))

; Use it for multiple requests
(hc/get "https://httpbin.org/get" {:http-client c})
(hc/head "https://httpbin.org/head" {:http-client c})

build-http-client options

authenticator

basic-auth

• A map of

  {:user "username" :pass "password"}

• a

  java.net.Authenticator

cookie-handler

java.net.CookieHandler

cookie-policy

cookie-policy

cookie-handler

• :none

  Accepts no cookies

• :all

  Accepts all cookies

• :original-server

  (default) Accepts cookies from original server
• An implementation of

  java.net.CookiePolicy

  .

connect-timeout

redirect-policy

• :never

  (default) Never follow redirects.

• :normal

  Always redirect, except from HTTPS URLs to HTTP URLs.

• :always

  Always redirect

priority

proxy

-Dhttp.proxyHost=somehost

-Dhttp.proxyPort=80

• :no-proxy

  to explicitly disable the default behavior, implying a direct connection; or
• a

  java.net.ProxySelector

ssl-context

(SSLContext/getDefault)

• a map of

  :keystore

  :keystore-pass

  :trust-store

  :trust-store-pass

  . See client authentication examples for more details.
• an

  javax.net.ssl.SSLContext

ssl-parameters

javax.net.ssl.SSLParameters

version

:http-1.1

:http-2

Making requests

The core function for making requests is

hato.client/request

get

post

put

; The main request function
(hc/request {:method :get, :url "https://httpbin.org/get"})

; Convenience wrappers
(hc/get "https://httpbin.org/get")
(hc/get "https://httpbin.org/get" {:as :json})
(hc/post "https://httpbin.org/post" {:body "{"a": 1}" :content-type :json})

request options

method

url

"http://moo.com/api/1"

accept

accept

:json

"text/html"

accept-encoding

[:gzip]

decompress-body?

content-type

:json

body

java.net.http.HttpRequest$BodyPublisher

form-params

content-type

as

• Return an object type:

  :string

  (default),

  :byte-array

  ,

  :stream

  ,

• :auto

  , to automatically coerce response body based on the response (e.g.

  content-type

  ). This is an alpha feature and the implementation may change.
• Coerce response body with certain format:

  :json

  ,

  :json-string-keys

  ,

  :clojure

  ,

  :transit+json

  ,

  :transit+msgpack

  . JSON and transit coercion require optional dependencies cheshire (5.9.0 or later) and com.cognitect/transit-clj to be installed, respectively.

coerce

:unexceptional

:always

:exceptional

query-params

form-params

content-type

:content-type :json

multi-param-style

query-params

• When unset (default), a repeating parameter

  a=1&a=2&a=3

• :array

  , a repeating param with array suffix:

  a[]=1&a[]=2&a[]=3

• :index

  , a repeating param with array suffix and index:

  a[0]=1&a[1]=2&a[2]=3

multipart

• :name

  The name of the param

• :part-name

  To preserve the order of entities,

  :name

  will be used as the part name unless

  :part-name

  is specified

• :content

  The part's data. May be a

  String

  ,

  InputStream

  ,

  Reader

  ,

  File

  ,

  char-array

  , or a

  byte-array

• :file-name

  The part's file name. If the

  :content

  is a

  File

  , it will use

  .getName

  by default but may be overridden.

• :content-type

  The part's content type. By default, if

  :content

  is a

  String

  it will be

  text/plain; charset=UTF-8

  and if

  :content

  is a

  File

  it will attempt to guess the best content type or fallback to

  application/octet-stream

  .

headers

basic-auth

Basic

{:user "user" :pass "pass"}

url

http://user:[email protected]

oauth-token

Bearer

decompress-body?

false

throw-exceptions?

false

async?

http-client

HttpClient

build-http-client

build-http-client

expect-continue

timeout

version

• :http-1.1

  prefer HTTP/1.1

• :http-2

  (default) tries to upgrade to HTTP/2, falling back to HTTP/1.1

Usage examples

Async requests

By default, hato performs synchronous requests and directly returns a response map.

By providing

async?

Alternatively, callbacks can be used by passing in

respond

raise

; A standard synchronous request
(hc/get "https://httpbin.org/get")

; An async request
(hc/get "https://httpbin.org/get" {:async? true})
; =>
; #object[jdk.internal.net.http.common.MinimalFuture...
; Deref it to get the value
(-> @(hc/get "https://httpbin.org/get" {:async? true})
    :body)
; =>
; { ...some json body }
; Pass in a callback
(hc/get "https://httpbin.org/get"
       { :async? true }
       (fn [resp] (println "Got status" (:status resp)))
       identity)
; =>
; #object[jdk.internal.net.http.common.MinimalFuture...
; Got status 200
(future-done? *1)
; =>
; true
; Exceptional status codes by default will call raise with an ex-info containing the response map.
; This means we can use ex-data to get the data back out.
@(hc/get "https://httpbin.org/status/400" {:async? true} identity #(-> % ex-data :status))
; =>
; 400

Making queries

hato can generate url encoded query strings in multiple ways

; Via un url
(hc/get "http://moo.com?hello=world&a=1&a=2" {})

; Via query-params
(hc/get "http://moo.com" {:query-params {:hello "world" :a [1 2]}})
; Values are urlencoded
(hc/get "http://moo.com" {:query-params {:q "a-space and-some-chars$&!"}})
; Generates query: "q=a-space+and-some-chars%24%26%21"
; Nested params are flattened by default
(hc/get "http://moo.com" {:query-params {:a {:b {:c 5} :e {:f 6}}}})
; => "a[b][c]=5&a[e][f]=6", url encoded
; Flattening can be disabled
(hc/get "http://moo.com" {:query-params {:a {:b {:c 5} :e {:f 6}}} :ignore-nested-query-string true})
; => "a={:b {:c 5}, :e {:f 6}}", url encoded

Form parameters can also be passed as a map:

(hc/post "http://moo.com" {:form-params {:hello "world"}})

; Send a json body "{"a": {"b": 5}}"
(hc/post "http://moo.com" {:form-params {:a {:b 5}} :content-type :json})
; Nested params are not flattened by default
; Sends a body of "a={:b {:c 5}, :e {:f 6}}", x-www-form-urlencoded
(hc/post "http://moo.com" {:form-params {:a {:b {:c 5} :e {:f 6}}}})
; Flattening can be enabled
; Sends a body of "a[b][c]=5&a[e][f]=6", url encoded
(hc/post "http://moo.com" {:form-params {:a {:b {:c 5} :e {:f 6}}}
                           :flatten-nested-form-params true})

As a convenience, nesting can also be controlled by

:flatten-nested-keys

; Flattens both query and form params
(hc/post "http://moo.com" {... :flatten-nested-keys [:query-params :form-params]})

; Flattens only query params
(hc/post "http://moo.com" {... :flatten-nested-keys [:query-params]})

Output coercion

You can control whether you like hato to return an

InputStream

:as :stream

byte-array

:as :byte-array

String

:as :string

; Returns a string response
(hc/get "http://moo.com" {})

; Returns a byte array
(hc/get "http://moo.com" {:as :byte-array})
; Returns an InputStream
(hc/get "http://moo.com" {:as :stream})
; Coerces clojure strings
(hc/get "http://moo.com" {:as :clojure})
; Coerces transit. Requires optional dependency com.cognitect/transit-clj.
(hc/get "http://moo.com" {:as :transit+json})
(hc/get "http://moo.com" {:as :transit+msgpack})
; Coerces JSON strings into clojure data structure
; Requires optional dependency cheshire
(hc/get "http://moo.com" {:as :json})
(hc/get "http://moo.com" {:as :json-string-keys})
; Coerce responses with exceptional status codes
(hc/get "http://moo.com" {:as :json :coerce :always})

By default, hato only coerces JSON responses for unexceptional statuses. Control this with the

:coerce

:unexceptional ; default - only coerce response bodies for unexceptional status codes
:exceptional ; only coerce for exceptional status codes
:always ; coerce for any status code

Certificate authentication

Client authentication can be done by passing in an SSLContext:

; Directly pass in an SSLContext that you made yourself
(hc/get "https://secure-url.com" {:http-client {:ssl-context SomeSSLContext}})

; Pass in your credentials
(hc/get "https://secure-url.com" {:http-client {:ssl-context {:keystore (io/resource "somepath.p12")
                                                              :keystore-pass "password"
                                                              :trust-store (io/resource "cacerts.p12"
                                                              :trust-store-pass "another-password")}}})

If either

:keystore

:trust-store

The defaults can be overridden with java options, so the below is equivalent to the above (with the caveat that the path should be on the filesystem rather than in the jar resources):

-Djavax.net.ssl.keyStore=somepath.12
-Djavax.net.ssl.keyStoreType=pkcs12
-Djavax.net.ssl.keyStorePassword=password
-Djavax.net.ssl.trustStore=cacerts.p12
-Djavax.net.ssl.trustStoreType=pkcs12
-Djavax.net.ssl.trustStorePassword=another-password

Redirects

By default, hato does not follow redirects. To change this behaviour, use the

redirect-policy

Implementation notes from the docs:

When automatic redirection occurs, the request method of the redirected request may be modified depending on the specific 30X status code, as specified in RFC 7231. In addition, the 301 and 302 status codes cause a POST request to be converted to a GET in the redirected request.

; Always redirect, except from HTTPS URLs to HTTP URLs
(hc/get "http://moo.com" {:http-client {:redirect-policy :normal}})

; Always redirect
(hc/get "http://moo.com" {:http-client {:redirect-policy :always}})

The Java HttpClient does not provide a direct option for max redirects. By default, it is 5. To change this, set the java option to e.g.

-Djdk.httpclient.redirects.retrylimit=10

The client does not throw an exception if the retry limit has been breached. Instead, it will return a response with the redirect status code (30x) and empty body.

Multipart Requests

To send a multipart request,

:multipart

:body

InputStream

(hc/post "http://moo.com"
        {:multipart [{:name "title" :content "My Awesome Picture"}
                     {:name "Content/type" :content "image/jpeg"}
                     {:name "foo.txt" :part-name "eggplant" :content "Eggplants"}
                     {:name "file" :content (io/file ".test-data")}
                     {:name "data" :content (.getBytes "hi" "UTF-8") :content-type "text/plain" :file-name "data.txt"}
                     {:name "jsonParam" :content (io/file ".test-data") :content-type "application/json" :file-name "data.json"}]})

Custom middleware

hato has a stack of middleware that it applies by default if you use the built in request function. You can supply different middleware by using

wrap-request

; Using the default middleware
(hc/request {:url "https://httpbin.org/get" :method :get})

; With convenience method
(hc/get "https://httpbin.org/get")
; Let's write an access log middleware
; Define a new middleware
(defn log-and-return
    [resp]
    (println :access-log (:uri resp) (:status resp) (:request-time resp))
    resp)
(defn wrap-log
  [client]
  (fn
    ([req]
     (let [resp (client req)]
       (log-and-return resp)))
    ([req respond raise]
     (client req
             #(respond (log-and-return %))
             raise))))
; Create your own middleware stack.
; Note that ordering is important here:
; - After wrap-request-timing so :request-time is available on the response
; - Before wrap-exceptions so that exceptional responses have not yet caused an exception to be thrown
(def my-middleware (concat [(first hm/default-middleware) wrap-log] (drop 1 hm/default-middleware)))
; Now it logs
(request {:url "https://httpbin.org/get" :method :get :middleware my-middleware})
; :access-log https://httpbin.org/get 200 1069
; => Returns response map
(request {:url "https://httpbin.org/status/404" :method :get :middleware my-middleware})
; :access-log https://httpbin.org/status/404 404 1924
; ...Throws some ExceptionInfo
(get "https://httpbin.org/get" {:middleware my-middleware})
; :access-log https://httpbin.org/get 200 1069
; => Returns string body

WebSockets

The simplest way to get started is with the

websocket

(require '[hato.websocket :as ws])

(let [ws @(ws/websocket "ws://echo.websocket.org"
                        {:on-message (fn [ws msg last?]
                                       (println "Received message:" msg))
                         :on-close   (fn [ws status reason]
                                       (println "WebSocket closed!"))})]
  (ws/send! ws "Hello World!")
  (Thread/sleep 1000)
  (ws/close! ws))

By default, hato WebSocket functions are asynchronous and most return a CompletableFuture. This can be wrapped in e.g. manifold, to give you promise chains etc.

(require '[hato.websocket :as ws])
(require '[manifold.deferred :as d])

(-> (ws/websocket "ws://echo.websocket.org"
                  {:on-message (fn [ws msg last?]
                                 (println "Received message:" msg))
                   :on-close   (fn [ws status reason]
                                 (println "WebSocket closed!"))})
    (d/chain #(ws/send! % "Hello")
             #(ws/send! % "World!")
             #(ws/close! %))
    (d/catch Exception #(println "Something went wrong!" %)))

WebSocket options

uri

"ws://echo.websocket.org"

opts

• :http-client

  An

  HttpClient

  (e.g. created by

  hato.client/build-http-client

  ). If not provided, a default client will be used.

• :headers

  Adds the given name-value pair to the list of additional HTTP headers sent during the opening handshake.

• :connect-timeout

  Sets a timeout for establishing a WebSocket connection, in milliseconds.

• :subprotocols

  Sets a request for the given subprotocols.

• :listener

  A WebSocket listener. If a

  WebSocket$Listener

  is provided, it will be used directly. Otherwise one will be created from any handlers (

  on-

  ) passed into the options map.

• :on-open

  Called when a

  WebSocket

  has been connected. Called with the WebSocket instance.

• :on-message

  A textual/binary data has been received. Called with the WebSocket instance, the data, and whether this invocation completes the message.

• :on-ping

  A Ping message has been received. Called with the WebSocket instance and the ping message.

• :on-pong

  A Pong message has been received. Called with the WebSocket instance and the pong message.

• :on-close

  Receives a Close message indicating the WebSocket's input has been closed. Called with the WebSocket instance, the status code, and the reason.

• :on-error

  An error has occurred. Called with the WebSocket instance and the error.

Debugging

To view the logs of the Java client, add the java option

-Djdk.httpclient.HttpClient.log=all

:jvm-opts

project.clj

Other advanced options

# Default keep alive for connection pool is 1200 seconds
-Djdk.httpclient.keepalivetimeout=1200

Default connection pool size is 0 (unbounded)
-Djdk.httpclient.connectionPoolSize=0

License

Released under the MIT License: http://www.opensource.org/licenses/mit-license.php