Java 11 HttpClient and cookies

The HTTP Client was introduced in Java 11. It can be used to request HTTP resources over the network. It supports HTTP/1.1 and HTTP/2, both synchronous and asynchronous programming models, handles request and response bodies as reactive-streams, and follows the familiar builder pattern.

https://openjdk.java.net/groups/net/httpclient/intro.html

Recently we had a requirement of setting an authentication session cookie and we had to configure HttpClient to handle that.

Below is how to configure HttpClient to use a CookieManager. The HttpCookie will be translated into a “Cookie:” header in the final Request.

        

        CookieHandler.setDefault(new CookieManager());

        HttpCookie sessionCookie = new HttpCookie("session", "53616c7465645f5f9d46701e259f8099dd2224786e1ed13");
        

        sessionCookie.setPath("/");
        sessionCookie.setVersion(0);

        ((CookieManager) CookieHandler.getDefault()).getCookieStore().add(new URI("https://adventofcode.com"),
                sessionCookie);

        HttpClient client = HttpClient.newBuilder()
                .cookieHandler(CookieHandler.getDefault())
                .connectTimeout(Duration.ofSeconds(10))
                .build();

        HttpRequest req = HttpRequest.newBuilder()
                .uri(URI.create("https://adventofcode.com/2020/day/1/input"))
                .GET().build();


        client.send(req, HttpResponse.BodyHandlers.ofString()).body();

HttpClient has a few filters that it runs through before the final Request is produced. CookieFilter is the filter responsible for settings the cookies, depending if you have added them to your CookieStore and if they match in path as well.

The HttpCookie has to match on Path in order to be added into your CookieStore, otherwise it will be ignored and your “Cookie: ” header will be empty.

package jdk.internal.net.http;

class CookieFilter implements HeaderFilter {

    public CookieFilter() {
    }

    @Override
    public void request(HttpRequestImpl r, MultiExchange<?> e) throws IOException {
        HttpClientImpl client = e.client();
        Optional<CookieHandler> cookieHandlerOpt = client.cookieHandler();
        if (cookieHandlerOpt.isPresent()) {
            CookieHandler cookieHandler = cookieHandlerOpt.get();
            Map<String,List<String>> userheaders = r.getUserHeaders().map();
            Map<String,List<String>> cookies = cookieHandler.get(r.uri(), userheaders);

            // add the returned cookies
            HttpHeadersBuilder systemHeadersBuilder = r.getSystemHeadersBuilder();
            if (cookies.isEmpty()) {
                Log.logTrace("Request: no cookie to add for {0}", r.uri());
            } else {
                Log.logTrace("Request: adding cookies for {0}", r.uri());
            }
            for (Map.Entry<String,List<String>> entry : cookies.entrySet()) {
                final String hdrname = entry.getKey();
                if (!hdrname.equalsIgnoreCase("Cookie")
                        && !hdrname.equalsIgnoreCase("Cookie2"))
                    continue;
                List<String> values = entry.getValue();
                if (values == null || values.isEmpty()) continue;
                for (String val : values) {
                    if (Utils.isValidValue(val)) {
                        systemHeadersBuilder.addHeader(hdrname, val);
                    }
                }
            }
        } else {
            Log.logTrace("Request: No cookie manager found for {0}", r.uri());
        }
    }

In HttpCookie, the default version is 1.

private int version = 1;    // Version=1 ... RFC 2965 style

The difference in versions affects how the “Cookie: ” header will look like on the Request. Most websites support version 0 (original Netscape format) by default.

Version 0:

INFO: HEADERS: HEADERS FRAME (stream=1)
    :authority: adventofcode.com
    :method: GET
    :path: /2020/day/1/input
    :scheme: https
    Cookie: session=53616c7465645f5f9d467....
    User-Agent: Java-http-client/11.0.7

Version 1:

INFO: HEADERS: HEADERS FRAME (stream=1)
    :authority: adventofcode.com
    :method: GET
    :path: /2020/day/1/input
    :scheme: https
    Cookie: $Version="1"
    Cookie: session="53616c7465645f5f9d467.....13";$Path="/"
    User-Agent: Java-http-client/11.0.7

Therefore, the CookieFilter will not add your HttpCookie, unless it matches path; and your service will most likely yield 500 Internal Server Error or 400 Bad Request, if HttpCookie is using version 1.

Logging HttpClient:

As we were investigating this, we had to enable logging, which allows us to see the final Request and how the “Cookie: ” header being passed looks like.

To enable HttpClient debug logging, add the below to your VM arguments:

-Djdk.httpclient.HttpClient.log=errors,requests,headers,frames[:control:data:window:all],content,ssl,trace,channel,all

The log:

nov. 05, 2021 12:52:00 EM jdk.internal.net.http.HttpClientImpl$SelectorManager run
INFO: CHANNEL: HttpClient-1-SelectorManager: starting
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.MultiExchange requestFilters
INFO: MISC: Applying request filters
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.MultiExchange requestFilters
INFO: MISC: Applying jdk.internal.net.http.AuthenticationFilter@74fe5c40
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.MultiExchange requestFilters
INFO: MISC: Applying jdk.internal.net.http.RedirectFilter@158d2680
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.MultiExchange requestFilters
INFO: MISC: Applying jdk.internal.net.http.CookieFilter@77847718
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.CookieFilter request
INFO: MISC: Request: adding cookies for https://adventofcode.com/2020/day/1/input
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.MultiExchange requestFilters
INFO: MISC: All filters applied
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.AbstractAsyncSSLConnection createSSLParameters
INFO: SSL: AbstractAsyncSSLConnection: Setting application protocols: [h2, http/1.1]
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.AbstractAsyncSSLConnection <init>
INFO: SSL: SSLParameters:
    cipher: TLS_AES_128_GCM_SHA256
    cipher: TLS_AES_256_GCM_SHA384
    cipher: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_RSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
    cipher: TLS_DHE_DSS_WITH_AES_256_GCM_SHA384
    cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_RSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
    cipher: TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
    cipher: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
    cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
    cipher: TLS_RSA_WITH_AES_256_CBC_SHA256
    cipher: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
    cipher: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
    cipher: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
    cipher: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
    cipher: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
    cipher: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
    cipher: TLS_RSA_WITH_AES_256_CBC_SHA
    cipher: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
    cipher: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
    cipher: TLS_DHE_RSA_WITH_AES_256_CBC_SHA
    cipher: TLS_DHE_DSS_WITH_AES_256_CBC_SHA
    cipher: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_RSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
    cipher: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
    cipher: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
    cipher: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
    cipher: TLS_RSA_WITH_AES_128_CBC_SHA
    cipher: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
    cipher: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
    cipher: TLS_DHE_RSA_WITH_AES_128_CBC_SHA
    cipher: TLS_DHE_DSS_WITH_AES_128_CBC_SHA
    cipher: TLS_EMPTY_RENEGOTIATION_INFO_SCSV
    application protocol: h2
    application protocol: http/1.1
    protocol: TLSv1.3
    protocol: TLSv1.2
    endpointIdAlg: HTTPS
    server name: type=host_name (0), value=adventofcode.com

nov. 05, 2021 12:52:00 EM jdk.internal.net.http.HttpClientImpl registerTimer
INFO: MISC: Registering timer ConnectTimerEvent, TimeoutEvent[id=1, duration=PT10S, deadline=2021-11-05T11:52:10.668403400Z]
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.SocketTube$InternalReadPublisher$InternalReadSubscription handleSubscribeEvent
INFO: CHANNEL: Start reading from java.nio.channels.SocketChannel[connected local=/100.120.31.175:50177 remote=adventofcode.com/50.16.234.137:443]
nov. 05, 2021 12:52:00 EM jdk.internal.net.http.SocketTube$InternalWriteSubscriber startSubscription
INFO: CHANNEL: Start requesting bytes for writing to channel: java.nio.channels.SocketChannel[connected local=/100.120.31.175:50177 remote=adventofcode.com/50.16.234.137:443]
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.HttpClientImpl cancelTimer
INFO: MISC: Canceling timer ConnectTimerEvent, TimeoutEvent[id=1, duration=PT10S, deadline=2021-11-05T11:52:10.668403400Z]
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection <init>
INFO: MISC: Connection send window size 65 535 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: MISC: /100.120.31.175:50177: start sending connection preface to adventofcode.com/50.16.234.137:443
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: FRAME: OUT: SETTINGS: length=30, streamid=0, flags=0  Settings: HEADER_TABLE_SIZE=16384 ENABLE_PUSH=1 MAX_CONCURRENT_STREAMS=100 INITIAL_WINDOW_SIZE=16777216 MAX_FRAME_SIZE=16384 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: MISC: PREFACE_BYTES sent
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: MISC: Settings Frame sent
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: CHANNEL: Sending initial connection window update frame: 33 488 897 (33 554 432 - 65 535)
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection encodeFrame
INFO: FRAME: OUT: WINDOW_UPDATE: length=4, streamid=0, flags=0  WindowUpdate: 33488897
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection sendConnectionPreface
INFO: MISC: finished sending connection preface
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream sendHeadersAsync
INFO: REQUEST: https://adventofcode.com/2020/day/1/input GET
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection encodeHeaders
INFO: HEADERS: HEADERS FRAME (stream=1)
    :authority: adventofcode.com
    :method: GET
    :path: /2020/day/1/input
    :scheme: https
    Cookie: session=53616c7465645f5f9d46701e259f8099dd2224786e1ed13
    User-Agent: Java-http-client/11.0.7


nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection lambda$encodeFrames$9
INFO: FRAME: OUT: HEADERS: length=126, streamid=1, flags=END_STREAM END_HEADERS 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream getResponseAsync
INFO: MISC: Response future (stream=1) is: jdk.internal.net.http.common.MinimalFuture@747ca99c[Not completed] (id=43)
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: SETTINGS: length=18, streamid=0, flags=0  Settings: MAX_CONCURRENT_STREAMS=128 INITIAL_WINDOW_SIZE=65536 MAX_FRAME_SIZE=16777215 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection encodeFrame
INFO: FRAME: OUT: SETTINGS: length=0, streamid=0, flags=ACK  Settings: 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: WINDOW_UPDATE: length=4, streamid=0, flags=0  WindowUpdate: 2147418112
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: SETTINGS: length=0, streamid=0, flags=ACK  Settings: 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: HEADERS: length=118, streamid=1, flags=END_HEADERS 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): :status: 200
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): date: Fri, 05 Nov 2021 11:52:01 GMT
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): content-type: text/plain
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): content-length: 990
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): server: Apache
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): server-ip: 172.31.16.87
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): vary: Accept-Encoding
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream$HeadersConsumer onDecoded
INFO: MISC: RECEIVED HEADER (streamid=1): strict-transport-security: max-age=300
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream incoming
INFO: MISC: handling response (streamid=1)
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream handleResponse
INFO: HEADERS: RESPONSE HEADERS:
    :status: 200
    content-length: 990
    content-type: text/plain
    date: Fri, 05 Nov 2021 11:52:01 GMT
    server: Apache
    server-ip: 172.31.16.87
    strict-transport-security: max-age=300
    vary: Accept-Encoding

nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream completeResponse
INFO: MISC: Completing response (streamid=1): jdk.internal.net.http.common.MinimalFuture@23761c8[Not completed, 1 dependents] (id=42)
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Exchange lambda$wrapForLog$11
INFO: RESPONSE: (GET https://adventofcode.com/2020/day/1/input) 200 HTTP_2 Local port:  50177
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.MultiExchange responseFilters
INFO: MISC: Applying response filters
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.MultiExchange responseFilters
INFO: MISC: Applying jdk.internal.net.http.CookieFilter@77847718
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.CookieFilter response
INFO: MISC: Response: processing cookies for https://adventofcode.com/2020/day/1/input
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.CookieFilter response
INFO: MISC: Response: parsing cookies from {:status=[200], content-length=[990], content-type=[text/plain], date=[Fri, 05 Nov 2021 11:52:01 GMT], server=[Apache], server-ip=[172.31.16.87], strict-transport-security=[max-age=300], vary=[Accept-Encoding]}
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.MultiExchange responseFilters
INFO: MISC: Applying jdk.internal.net.http.RedirectFilter@158d2680
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.MultiExchange responseFilters
INFO: MISC: Applying jdk.internal.net.http.AuthenticationFilter@74fe5c40
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.MultiExchange responseFilters
INFO: MISC: All filters applied
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream readBodyAsync
INFO: MISC: Reading body on stream 1
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: DATA: length=990, streamid=1, flags=0 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream schedule
INFO: MISC: responseSubscriber.onNext 990
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Http2Connection processFrame
INFO: FRAME: IN: DATA: length=0, streamid=1, flags=END_STREAM 
nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream schedule
INFO: MISC: responseSubscriber.onComplete
1337
1906
2007

nov. 05, 2021 12:52:01 EM jdk.internal.net.http.Stream close
INFO: MISC: Stream 1 closed

Process finished with exit code 0

Java 11 HttpClient and cookies

Related

Leave a comment Cancel reply