An HTTP client for Clojure, wrapping JDK 11's HttpClient
An HTTP client for Clojure, wrapping JDK 11's HttpClient.
It supports both HTTP/1.1 and HTTP/2, with synchronous and asynchronous execution modes as well as websockets.
In general, it will feel familiar to users of http clients like clj-http. The API is designed to be idiomatic and to make common tasks convenient, whilst still allowing the underlying HttpClient to be configured via native Java objects.
hato has a stable API and is used in production for both synchronous and asynchronous use cases. Please try it out and raise any issues you may find.
hato requires JDK 11 and above. If you are running an older version of Java, please look at clj-http.
For Leinengen, add this to your project.clj
[hato "0.8.0"]
The main client is available in
hato.client.
Require it to get started and make a request:
(ns my.app (:require [hato.client :as hc]))(hc/get "https://httpbin.org/get") ; => ; {:request-time 112 ; :status 200 ; :body "{"url" ...}" ; ...}
Generally, you want to make a reusable client first. This will give you nice things like persistent connections and connection pooling.
This can be done with
build-http-client:
; Build the client (def c (hato/build-http-client {:connect-timeout 10000 :redirect-policy :always})); Use it for multiple requests (hc/get "https://httpbin.org/get" {:http-client c}) (hc/head "https://httpbin.org/head" {:http-client c})
authenticatorUsed for non-preemptive basic authentication. See the
basic-authrequest option for pre-emptive authentication. Accepts:
{:user "username" :pass "password"}
java.net.Authenticator
cookie-handlera
java.net.CookieHandlerif you need full control of your cookies. See
cookie-policyfor a more convenient option.
cookie-policyDetermines whether to accept cookies. The
cookie-handleroption will take precedence if it is set. If an invalid option is provided, a CookieManager with the default policy (original-server) will be created. Valid options:
:noneAccepts no cookies
:allAccepts all cookies
:original-server(default) Accepts cookies from original server
java.net.CookiePolicy.
connect-timeoutTimeout to making a connection, in milliseconds (default: unlimited).
redirect-policySets the redirect policy.
:never(default) Never follow redirects.
:normalAlways redirect, except from HTTPS URLs to HTTP URLs.
:alwaysAlways redirect
priorityan integer between 1 and 256 (both inclusive) for HTTP/2 requests
proxySets a proxy selector. If not set, uses the default system-wide ProxySelector, which can be configured by Java opts such as
-Dhttp.proxyHost=somehostand
-Dhttp.proxyPort=80(see all options). Also accepts:
:no-proxyto explicitly disable the default behavior, implying a direct connection; or
java.net.ProxySelector
ssl-contextSets the SSLContext. If not specified, uses the default
(SSLContext/getDefault). Accepts:
:keystore
:keystore-pass
:trust-store
:trust-store-pass. See client authentication examples for more details.
javax.net.ssl.SSLContext
ssl-parametersa
javax.net.ssl.SSLParameters
versionSets preferred HTTP protocol version. -
:http-1.1prefer HTTP/1.1 -
:http-2(default) tries to upgrade to HTTP/2, falling back to HTTP/1.1
The core function for making requests is
hato.client/request, which takes a ring request and returns a response. Convenience wrappers are provided for the http verbs (
get,
post,
putetc.).
; The main request function (hc/request {:method :get, :url "https://httpbin.org/get"}); Convenience wrappers (hc/get "https://httpbin.org/get") (hc/get "https://httpbin.org/get" {:as :json}) (hc/post "https://httpbin.org/post" {:body "{"a": 1}" :content-type :json})
methodLowercase keyword corresponding to a HTTP request method, such as :get or :post.
urlAn absolute url to the requested resource (e.g.
"http://moo.com/api/1").
acceptSets the
acceptheader. a keyword (e.g.
:json, for any application/* type) or string (e.g.
"text/html") for anything else.
accept-encodingList of string/keywords (e.g.
[:gzip]). By default, "gzip, deflate" will be concatenated unless
decompress-body?is false.
content-typea keyword (e.g.
:json, for any application/* type) or string (e.g. "text/html") for anything else. Sets the appropriate header.
bodythe body of the request. This should be a string, byte array, input stream, or a
java.net.http.HttpRequest$BodyPublisher. To send a clojure map as json (or some other format), use the
form-paramsoption with the appropriate
content-type.
asReturn response body in a certain format. Valid options:
:string(default),
:byte-array,
:stream,
:auto, to automatically coerce response body based on the response (e.g.
content-type). This is an alpha feature and the implementation may change.
:json,
:json-string-keys,
:clojure,
:transit+json,
:transit+msgpack. JSON and transit coercion require optional dependencies cheshire (5.9.0 or later) and com.cognitect/transit-clj to be installed, respectively.
coerceDetermine which status codes to coerce response bodies.
:unexceptional(default),
:always,
:exceptional. This presently only has an effect for json coercions.
query-paramsA map of options to turn into a query string. See usage examples for details.
form-paramsA map of options that will be sent as the body, depending on the
content-typeoption. For example, set
:content-type :jsonto coerce the form-params to a json string (requires cheshire). See usage examples for details.
multi-param-styleDecides how to represent array values when converting
query-paramsinto a query string. Accepts:
a=1&a=2&a=3
:array, a repeating param with array suffix:
a[]=1&a[]=2&a[]=3
:index, a repeating param with array suffix and index:
a[0]=1&a[1]=2&a[2]=3
multipartA sequence of maps with the following keys:
:nameThe name of the param
:part-nameTo preserve the order of entities,
:namewill be used as the part name unless
:part-nameis specified
:contentThe part's data. May be a
String,
InputStream,
Reader,
File,
char-array, or a
byte-array
:file-nameThe part's file name. If the
:contentis a
File, it will use
.getNameby default but may be overridden.
:content-typeThe part's content type. By default, if
:contentis a
Stringit will be
text/plain; charset=UTF-8and if
:contentis a
Fileit will attempt to guess the best content type or fallback to
application/octet-stream.
headersMap of lower case strings to header values, concatenated with ',' when multiple values for a key. This is presently a slight incompatibility with clj-http, which accepts keyword keys and list values.
basic-authPerforms basic authentication (sending
Basicauthorization header). Accepts
{:user "user" :pass "pass"}Note that basic auth can also be passed via the
url(e.g.
http://user:[email protected])
oauth-tokenString, will set
Bearerauthorization header
decompress-body?By default, sets request header to accept "gzip, deflate" encoding, and decompresses the response. Set to
falseto turn off this behaviour.
throw-exceptions?By default, the client will throw exceptions for exceptional response statuses. Set this to
falseto return the response without throwing.
async?Boolean, defaults to false. See below section on async requests.
http-clientAn
HttpClientcreated by
build-http-clientor other means. For single-use clients, it also accepts a map of the options accepted by
build-http-client.
expect-continueRequests the server to acknowledge the request before sending the body. This is disabled by default.
timeoutTimeout to receiving a response, in milliseconds (default: unlimited).
versionSets preferred HTTP protocol version per request.
:http-1.1prefer HTTP/1.1
:http-2(default) tries to upgrade to HTTP/2, falling back to HTTP/1.1
By default, hato performs synchronous requests and directly returns a response map.
By providing
async?option to the request, the request will be performed asynchronously, returning a CompletableFuture of the response map. This can be wrapped in e.g. manifold, to give you promise chains etc.
Alternatively, callbacks can be used by passing in
respondand
raisefunctions, in which case the CompletableFuture returned can be used to indicate when processing has completed.
; A standard synchronous request (hc/get "https://httpbin.org/get"); An async request (hc/get "https://httpbin.org/get" {:async? true}) ; => ; #object[jdk.internal.net.http.common.MinimalFuture...
; Deref it to get the value (-> @(hc/get "https://httpbin.org/get" {:async? true}) :body) ; => ; { ...some json body }
; Pass in a callback (hc/get "https://httpbin.org/get" { :async? true } (fn [resp] (println "Got status" (:status resp))) identity) ; => ; #object[jdk.internal.net.http.common.MinimalFuture... ; Got status 200
(future-done? *1) ; => ; true
; Exceptional status codes by default will call raise with an ex-info containing the response map. ; This means we can use ex-data to get the data back out. @(hc/get "https://httpbin.org/status/400" {:async? true} identity #(-> % ex-data :status)) ; => ; 400
hato can generate url encoded query strings in multiple ways
; Via un url (hc/get "http://moo.com?hello=world&a=1&a=2" {}); Via query-params (hc/get "http://moo.com" {:query-params {:hello "world" :a [1 2]}})
; Values are urlencoded (hc/get "http://moo.com" {:query-params {:q "a-space and-some-chars$&!"}}) ; Generates query: "q=a-space+and-some-chars%24%26%21"
; Nested params are flattened by default (hc/get "http://moo.com" {:query-params {:a {:b {:c 5} :e {:f 6}}}}) ; => "a[b][c]=5&a[e][f]=6", url encoded
; Flattening can be disabled (hc/get "http://moo.com" {:query-params {:a {:b {:c 5} :e {:f 6}}} :ignore-nested-query-string true}) ; => "a={:b {:c 5}, :e {:f 6}}", url encoded
Form parameters can also be passed as a map:
(hc/post "http://moo.com" {:form-params {:hello "world"}}); Send a json body "{"a": {"b": 5}}" (hc/post "http://moo.com" {:form-params {:a {:b 5}} :content-type :json})
; Nested params are not flattened by default ; Sends a body of "a={:b {:c 5}, :e {:f 6}}", x-www-form-urlencoded (hc/post "http://moo.com" {:form-params {:a {:b {:c 5} :e {:f 6}}}})
; Flattening can be enabled ; Sends a body of "a[b][c]=5&a[e][f]=6", url encoded (hc/post "http://moo.com" {:form-params {:a {:b {:c 5} :e {:f 6}}} :flatten-nested-form-params true})
As a convenience, nesting can also be controlled by
:flatten-nested-keys:
; Flattens both query and form params (hc/post "http://moo.com" {... :flatten-nested-keys [:query-params :form-params]}); Flattens only query params (hc/post "http://moo.com" {... :flatten-nested-keys [:query-params]})
You can control whether you like hato to return an
InputStream(using
:as :stream),
byte-array(using
:as :byte-array) or
String(
:as :string) with no further coercion.
; Returns a string response (hc/get "http://moo.com" {}); Returns a byte array (hc/get "http://moo.com" {:as :byte-array})
; Returns an InputStream (hc/get "http://moo.com" {:as :stream})
; Coerces clojure strings (hc/get "http://moo.com" {:as :clojure})
; Coerces transit. Requires optional dependency com.cognitect/transit-clj. (hc/get "http://moo.com" {:as :transit+json}) (hc/get "http://moo.com" {:as :transit+msgpack})
; Coerces JSON strings into clojure data structure ; Requires optional dependency cheshire (hc/get "http://moo.com" {:as :json}) (hc/get "http://moo.com" {:as :json-string-keys})
; Coerce responses with exceptional status codes (hc/get "http://moo.com" {:as :json :coerce :always})
By default, hato only coerces JSON responses for unexceptional statuses. Control this with the
:coerceoption:
:unexceptional ; default - only coerce response bodies for unexceptional status codes :exceptional ; only coerce for exceptional status codes :always ; coerce for any status code
Client authentication can be done by passing in an SSLContext:
; Directly pass in an SSLContext that you made yourself (hc/get "https://secure-url.com" {:http-client {:ssl-context SomeSSLContext}}); Pass in your credentials (hc/get "https://secure-url.com" {:http-client {:ssl-context {:keystore (io/resource "somepath.p12") :keystore-pass "password" :trust-store (io/resource "cacerts.p12" :trust-store-pass "another-password")}}})
If either
:keystoreor
:trust-storeare not provided, the respective system default will be used.
The defaults can be overridden with java options, so the below is equivalent to the above (with the caveat that the path should be on the filesystem rather than in the jar resources):
-Djavax.net.ssl.keyStore=somepath.12 -Djavax.net.ssl.keyStoreType=pkcs12 -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStore=cacerts.p12 -Djavax.net.ssl.trustStoreType=pkcs12 -Djavax.net.ssl.trustStorePassword=another-password
By default, hato does not follow redirects. To change this behaviour, use the
redirect-policyoption.
Implementation notes from the docs:
When automatic redirection occurs, the request method of the redirected request may be modified depending on the specific 30X status code, as specified in RFC 7231. In addition, the 301 and 302 status codes cause a POST request to be converted to a GET in the redirected request.
; Always redirect, except from HTTPS URLs to HTTP URLs (hc/get "http://moo.com" {:http-client {:redirect-policy :normal}}); Always redirect (hc/get "http://moo.com" {:http-client {:redirect-policy :always}})
The Java HttpClient does not provide a direct option for max redirects. By default, it is 5. To change this, set the java option to e.g.
-Djdk.httpclient.redirects.retrylimit=10.
The client does not throw an exception if the retry limit has been breached. Instead, it will return a response with the redirect status code (30x) and empty body.
To send a multipart request,
:multipartmay be supplied as a sequence of maps as described in request options. This will add the appropriate Content-Type header as well as replace the
:bodyof the request with an
InputStreamof the supplied parts.
(hc/post "http://moo.com" {:multipart [{:name "title" :content "My Awesome Picture"} {:name "Content/type" :content "image/jpeg"} {:name "foo.txt" :part-name "eggplant" :content "Eggplants"} {:name "file" :content (io/file ".test-data")} {:name "data" :content (.getBytes "hi" "UTF-8") :content-type "text/plain" :file-name "data.txt"} {:name "jsonParam" :content (io/file ".test-data") :content-type "application/json" :file-name "data.json"}]})
hato has a stack of middleware that it applies by default if you use the built in request function. You can supply different middleware by using
wrap-requestyourself:
; Using the default middleware (hc/request {:url "https://httpbin.org/get" :method :get}); With convenience method (hc/get "https://httpbin.org/get")
; Let's write an access log middleware
; Define a new middleware (defn log-and-return [resp] (println :access-log (:uri resp) (:status resp) (:request-time resp)) resp)
(defn wrap-log [client] (fn ([req] (let [resp (client req)] (log-and-return resp))) ([req respond raise] (client req #(respond (log-and-return %)) raise))))
; Create your own middleware stack. ; Note that ordering is important here: ; - After wrap-request-timing so :request-time is available on the response ; - Before wrap-exceptions so that exceptional responses have not yet caused an exception to be thrown (def my-middleware (concat [(first hm/default-middleware) wrap-log] (drop 1 hm/default-middleware)))
; Now it logs (request {:url "https://httpbin.org/get" :method :get :middleware my-middleware}) ; :access-log https://httpbin.org/get 200 1069 ; => Returns response map
(request {:url "https://httpbin.org/status/404" :method :get :middleware my-middleware}) ; :access-log https://httpbin.org/status/404 404 1924 ; ...Throws some ExceptionInfo
(get "https://httpbin.org/get" {:middleware my-middleware}) ; :access-log https://httpbin.org/get 200 1069 ; => Returns string body
The simplest way to get started is with the
websocketfunction:
(require '[hato.websocket :as ws])(let [ws @(ws/websocket "ws://echo.websocket.org" {:on-message (fn [ws msg last?] (println "Received message:" msg)) :on-close (fn [ws status reason] (println "WebSocket closed!"))})] (ws/send! ws "Hello World!") (Thread/sleep 1000) (ws/close! ws))
By default, hato WebSocket functions are asynchronous and most return a CompletableFuture. This can be wrapped in e.g. manifold, to give you promise chains etc.
(require '[hato.websocket :as ws]) (require '[manifold.deferred :as d])(-> (ws/websocket "ws://echo.websocket.org" {:on-message (fn [ws msg last?] (println "Received message:" msg)) :on-close (fn [ws status reason] (println "WebSocket closed!"))}) (d/chain #(ws/send! % "Hello") #(ws/send! % "World!") #(ws/close! %)) (d/catch Exception #(println "Something went wrong!" %)))
uriA WebSocket uri (e.g.
"ws://echo.websocket.org").
optsAdditional options may be a map of any of the following keys:
:http-clientAn
HttpClient(e.g. created by
hato.client/build-http-client). If not provided, a default client will be used.
:headersAdds the given name-value pair to the list of additional HTTP headers sent during the opening handshake.
:connect-timeoutSets a timeout for establishing a WebSocket connection, in milliseconds.
:subprotocolsSets a request for the given subprotocols.
:listenerA WebSocket listener. If a
WebSocket$Listeneris provided, it will be used directly. Otherwise one will be created from any handlers (
on-) passed into the options map.
:on-openCalled when a
WebSockethas been connected. Called with the WebSocket instance.
:on-messageA textual/binary data has been received. Called with the WebSocket instance, the data, and whether this invocation completes the message.
:on-pingA Ping message has been received. Called with the WebSocket instance and the ping message.
:on-pongA Pong message has been received. Called with the WebSocket instance and the pong message.
:on-closeReceives a Close message indicating the WebSocket's input has been closed. Called with the WebSocket instance, the status code, and the reason.
:on-errorAn error has occurred. Called with the WebSocket instance and the error.
To view the logs of the Java client, add the java option
-Djdk.httpclient.HttpClient.log=all. In Leinengen, this can be done using
:jvm-optsin
project.clj.
# Default keep alive for connection pool is 1200 seconds -Djdk.httpclient.keepalivetimeout=1200Default connection pool size is 0 (unbounded)
-Djdk.httpclient.connectionPoolSize=0
Released under the MIT License: http://www.opensource.org/licenses/mit-license.php