andrew Flower

Custom Auth Header for WebClient

Adding an HMAC Signature to Spring WebClient requests

The aim of this article is to demonstrate how to add a Custom Authorization Header to requests made by a Spring WebClient, which has replaced RestTemplate as the recommended Spring HTTP client.

It is very common these days to use HMAC-based Authorization schemes, whereby the parts of the request are signed using a secret key and the signature is sent with the request in the HTTP Authorization header.

AWS' Signature Version 4 is an example of this. I'll demonstrate two ways to do this with WebClient. The first is in the case that you don't need to sign the body of the request, such as read-only requests. The second will show how the body can be intercepted after serialization to solve the general case that includes mutating requests like POST, PUT or PATCH. This article does not go into showing how to sign streamed requests

The accompanying source code is available here:

Skip to the next section if you just want to see how it's done.

What is WebClient?

In short, WebClient is a reactive, non-blocking HTTP client introduced in Spring Framework 5.0 as part of Spring WebFlux. The API for WebClient is functional and fluent, allowing for progamming that fits the recently popular reactive paradigm. More can be read about WebClient in the Spring documentation. The client can be included in Spring Boot projects by adding spring-boot-starter-webflux as a Gradle or Maven dependency.

RestTemplate was the recommended HTTP client used in Spring Boot projects up until recently, but the JavaDocs have been updated to indicate its deprecation:

As of 5.0, the non-blocking, reactive org.springframework.web.reactive.client.WebClient offers a modern alternative to the RestTemplate with efficient support for both sync and async, as well as streaming scenarios. The RestTemplate will be deprecated in a future version and will not have major new features added going forward. See the WebClient section of the Spring Framework reference documentation for more details and example code.

Simple GET Request Signing

A Request-Processing Filter

When building a WebClient, we use the builder object and can provide a significant amount of customization. Ideally we don't want to have to perform authorization logic on every request but rather configure the client once to do it automatically for every request.

Looking through what's available we realise that we need a filter that is applied to a request before sending. ExchangeFilterFunction.ofRequestProcessor allows us to pass a lambda function as a request processor.

final Signer signer = new Signer(environment, clientId, secretKey);

final WebClient client = WebClient
        .builder()
        .baseUrl(environment.getBaseUrl())
        .filter(ExchangeFilterFunction.ofRequestProcessor(signer::injectHeader))
        .build();

Above is an example of delegating request processing to another method, as you might do when augmenting a request using common logic, such as adding an Authorization header containing a custom auth signature.

Above we specified a method injectHeader, on another class that is responsible for signature building, to process the request and add an Authorization header. That request processor method might look like this:

public Mono<ClientRequest> injectHeader(final ClientRequest clientRequest) {
    final String dateString = ZonedDateTime.now().format(DateTimeFormatter.RFC_1123_DATE_TIME);
    final String authHeader = buildAuthHeaderForRequest(
            clientRequest.method(),
            clientRequest.url().getPath(),
            clientRequest.url().getQuery(),
            dateString);

    return Mono.just(ClientRequest.from(clientRequest)
                                  .header(HttpHeaders.DATE, dateString)
                                  .header(HttpHeaders.AUTHORIZATION, authHeader)
                                  .build());
}

This Request Processor takes some data from the request and uses it to build an Authorization header using these attributes, according to type of auth being used. We use ClientRequest.from to generate a new ClientRequest based on the incoming one and wrap it using Mono.just in order to return it for further processing and, finally, sending.

Using Per-Request Data

In some cases the data needed for building the signature will not be retrievable from the given ClientRequest instance. In this case it's possible to attach extra data to the request object as an attribute when making the request. In the following example we actually pass in the DateTime of the request, which is especially useful if you are (as you should be) unit-testing the code.

final Mono<String> pair = webClient.get()
        .attribute("date", ZonedDateTime.now())
        .retrieve()
        .bodyToMono(String.class);

Then in the request processor, you can access this data by retrieving the attribute:

    final String dateString = ZonedDateTime.now().format(DateTimeFormatter.RFC_1123_DATE_TIME);
    final String dateString = clientRequest.attribute("date").get().toString();
    final String authHeader = buildAuthHeaderForRequest(
            clientRequest.method(),
            clientRequest.url().getPath(),
            clientRequest.url().getQuery(),
            dateString);

POST Data Signing - The General Approach

The previous approach only works for GET requests where there isn't a body payload to sign.

With HMAC-based Auth schemes, signatures for POST requests usually include the hash of the entire data payloads. In this case we will have to work a little harder, dig a little deeper and customize a little further.

First let's imagine a typical POST request:

webClient.post()
         .uri("/users")
         .contentType(MediaType.APPLICATION_JSON)
         .body(BodyInserters.fromValue(new User("Someone Nobody", "someone@example.com")))
         .exchange();

The server will receive that data in bytes and perform the signing using the serialized data, therefore we need to calculate our client-side signature on the data post-serialization too.

Because WebClient supports streaming of data with a reactive API, the serialization of the data is done just in time for sending over the network. In fact, in the streaming case, we would have to encode the payload in chunks as sending is done in parallel with serialization by the client.

We will not cover the case of streaming protocols/encoding, but you can adapt the same principle based on your use-case.

One option would be to manually serialize the data first into a String, and then pass that String to both our filter from the first section and as the body for the request. But this is not ideal. We will follow a different approach shown in the following sections

Capturing the Serialized Data

For the purpose of this article I am assuming that we are trying to serialize the data into JSON. By default WebClient uses Jackson to encode the data into JSON.

The API for creating a WebClient actually allows us to specify "codecs" to help with deserializing and serializing. What we can do is provide our own serializer which wraps the default Jackson serializer and intercepts the serialized body before it is sent.

Below we create our own wrapper class around Jackson2JsonEncoder that allows us to intercept the encoded body. Specifically we are wrapping the encodeValue method implementation from AbstractJackson2Encoder.

public class BodyProvidingJsonEncoder extends Jackson2JsonEncoder {
    private final Consumer<byte[]> bodyConsumer;

    public BodyProvidingJsonEncoder(final Consumer<byte[]> bodyConsumer) {
        this.bodyConsumer = bodyConsumer;
    }

    @Override
    public DataBuffer encodeValue(final Object value, final DataBufferFactory bufferFactory,
                                  final ResolvableType valueType, @Nullable final MimeType mimeType,
                                  @Nullable final Map<String, Object> hints) {
        final DataBuffer data = super.encodeValue(value, bufferFactory, valueType, mimeType, hints);
        bodyConsumer.accept(extractBytes(data));
        return data;
    }

    private byte[] extractBytes(final DataBuffer data) {
        final byte[] bytes = new byte[data.readableByteCount()];
        data.read(bytes);
        data.readPosition(0);
        return bytes;
    }
}

In the above class we have intercepted encodeValue() (by calling the superclass method) in order to pass the serialized JSON byte data to be used in signing and Authorization header injection. The bytes are extracted in extractBytes() where the read-position must also be reset to 0 for when it is used later by the message writer.

Note that in the constructor we receive a Consumer to which we later (in line 13) pass the serialized bytes. Instantiation of this object is shown in the final section/

Capturing the Message

Unfortunately, as far as I've seen, there is no other way to hook into the request process between data encoding and actually sending the data. This is why we have to capture the message early on and wait for the encoding step to provide the serialized body so that the signature can be formed and injected into the message's Authorization header.

The WebClient builder allows us to define an ExchangeFunction using the utility ExchangeFunctions.create() which accepts a custom HttpConnector. This connector has access to the function that makes the request. It is at this point that we can get a handle on the ClientHttpRequest and wait for the body to be serialized so that the header can be added.

Below you can see the custom HTTP connector which inherits from ReactorClientHttpConnector which is the default connector created in DefaultWebClientBuilder.

public class MessageSigningHttpConnector extends ReactorClientHttpConnector {
    private final Set<HttpMethod> BODYLESS_METHODS = Set.of(GET, DELETE, TRACE, HEAD, OPTIONS);
    private final ThreadLocal<ClientHttpRequest> request = new ThreadLocal<>();
    private final Signer signer;

    public MessageSigningHttpConnector(final Signer signer) {
        this.signer = signer;
    }

    @Override
    public Mono<ClientHttpResponse> connect(final HttpMethod method, final URI uri,
                                            final Function<? super ClientHttpRequest, Mono<Void>> requestCallback) {
        // execute the super-class method as usual, but insert an interception into the requestCallback that can
        // capture the request to be saved for this thread.
        return super.connect(method, uri, incomingRequest -> {
            sign(incomingRequest);
            return requestCallback.apply(incomingRequest);
        });
    }

    private void sign(final ClientHttpRequest request) {
        if (BODYLESS_METHODS.contains(request.getMethod())) {
            signer.injectHeader(request.get(), null);
        } else {
            this.request.set(request);
        }
    }

    public void signWithBody(byte[] bodyData) {
        signer.injectHeader(request.get(), bodyData);

        // release the request from the thread-local
        request.remove();
    }
}

Important points:

  • We need to handle all HTTP methods (those that have bodies and those that don't).
  • We override the default connect method, only so that we can wrap the `requestCallback` and grab the `ClientHttpRequest` for signing.
  • On line 16 the request is intercepted for signing. In the case of Http methods that don't send a body, the signing can be done immediately, but otherwise the request is stored (line 25) in a ThreadLocal ready to be supplied to our encoder for signature injection.
  • The signWithBody method is what we expose to be called with the serialized data when ready.
warning I have not yet tested this in the wild. This code assumes that the encoding will be done by the same thread that captures the messages, but I haven't verified that yet and even if so, it might change in the future.

Building the Customized Client

Now that we have the two necessary parts we can put them together and build our WebClient:

Signer signer
    = new Signer(clientId, secret);
MessageSigningHttpConnector httpConnector
    = new MessageSigningHttpConnector(signer);
BodyProvidingJsonEncoder bodyProvidingJsonEncoder
    = new BodyProvidingJsonEncoder(httpConnector::signWithBody);

WebClient client
    = WebClient.builder()
               .exchangeFunction(ExchangeFunctions.create(
                       httpConnector,
                       ExchangeStrategies
                              .builder()
                               .codecs(clientDefaultCodecsConfigurer -> {
                                   clientDefaultCodecsConfigurer.defaultCodecs().jackson2JsonEncoder(bodyProvidingJsonEncoder);
                                   clientDefaultCodecsConfigurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(new ObjectMapper(), MediaType.APPLICATION_JSON));
                               })
                               .build()
               ))
               .baseUrl(String.format("%s://%s/%s", environment.getProtocol(), environment.getHost(), environment.getPath()))
               .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
               .build();

In the example above, we create a custom ExchangeFunction providing our HTTP Connector (line 11), and ExchangeStrategies that use our JSON Encoder (line 15). Note that we pass a Consumer, signWithBody from the connector to allow the encoder to provide the data payload, and trigger the signing.

Summary

In summary these are the points:

  • RestTemplate will be receiving no new features and will be deprecated in the near future
  • WebClient is the new favourite HTTP Client in Spring
  • ExchangeFilterFunction.ofRequestProcessor can be used to pass a request processor method to the .filter(..) method on the WebClient.builder()
    • The request processor can use data from the ClientRequest and return a new augmented request wrapped using Mono.just
  • For POST requests, where body-signing is required, we can create customized ExchangeFunctions with an Encoder-wrapper that captures the body just after serialization and before transfer.
If you use this approach, be sure to write tests and verify it works for your use-case.

The accompanying source code is available here:

Read More