Class HttpClient

java.lang.Object
org.eclipse.jetty.util.component.AbstractLifeCycle
org.eclipse.jetty.util.component.ContainerLifeCycle
org.eclipse.jetty.client.HttpClient
All Implemented Interfaces:
Container, Destroyable, Dumpable, Dumpable.DumpableContainer, LifeCycle
@ManagedObject("The HTTP client") public class HttpClient extends ContainerLifeCycle

HttpClient provides an efficient, asynchronous, non-blocking implementation to perform HTTP requests to a server through a simple API that offers also blocking semantic.

HttpClient provides easy-to-use methods such as GET(String) that allow to perform HTTP requests in a one-liner, but also gives the ability to fine tune the configuration of requests via newRequest(URI).

HttpClient acts as a central configuration point for network parameters (such as idle timeouts) and HTTP parameters (such as whether to follow redirects).

HttpClient transparently pools connections to servers, but allows direct control of connections for cases where this is needed.

HttpClient also acts as a central configuration point for cookies, via getCookieStore().

Typical usage:

HttpClient httpClient = new HttpClient();
httpClient.start();

// One liner:
httpClient.GET("http://localhost:8080/").getStatus();

// Building a request with a timeout
ContentResponse response = httpClient.newRequest("http://localhost:8080")
        .timeout(5, TimeUnit.SECONDS)
        .send();
int status = response.status();

// Asynchronously
httpClient.newRequest("http://localhost:8080").send(new Response.CompleteListener()
{
    @Override
    public void onComplete(Result result)
    {
        ...
    }
});

  • Field Details

    • LOG

      private static final Logger LOG

    • destinations

      private final ConcurrentMap<Origin, HttpDestination> destinations

    • handlers

      private final ProtocolHandlers handlers

    • requestListeners

      private final List<Request.Listener> requestListeners

    • decoderFactories

      private final Set<ContentDecoder.Factory> decoderFactories

    • proxyConfig

      private final ProxyConfiguration proxyConfig

    • transport

      private final HttpClientTransport transport

    • sslContextFactory

      private final SslContextFactory sslContextFactory

    • authenticationStore

      private AuthenticationStore authenticationStore

    • cookieManager

      private CookieManager cookieManager

    • cookieStore

      private CookieStore cookieStore

    • executor

      private Executor executor

    • byteBufferPool

      private ByteBufferPool byteBufferPool

    • scheduler

      private Scheduler scheduler

    • resolver

      private SocketAddressResolver resolver

    • agentField

      private HttpField agentField

    • followRedirects

      private boolean followRedirects

    • maxConnectionsPerDestination

      private int maxConnectionsPerDestination

    • maxRequestsQueuedPerDestination

      private int maxRequestsQueuedPerDestination

    • requestBufferSize

      private int requestBufferSize

    • responseBufferSize

      private int responseBufferSize

    • maxRedirects

      private int maxRedirects

    • bindAddress

      private SocketAddress bindAddress

    • connectTimeout

      private long connectTimeout

    • addressResolutionTimeout

      private long addressResolutionTimeout

    • idleTimeout

      private long idleTimeout

    • tcpNoDelay

      private boolean tcpNoDelay

    • strictEventOrdering

      private boolean strictEventOrdering

    • encodingField

      private HttpField encodingField

    • removeIdleDestinations

      private boolean removeIdleDestinations

    • connectBlocking

      private boolean connectBlocking

    • name

      private String name

    • httpCompliance

      private HttpCompliance httpCompliance

    • defaultRequestContentType

      private String defaultRequestContentType

  • Constructor Details

  • Method Details

    • dump

      public void dump(Appendable out, String indent) throws IOException
      Description copied from interface: Dumpable
      Dump this object (and children) into an Appendable using the provided indent after any new lines. The indent should not be applied to the first object dumped.
      Specified by:
      dump in interface Dumpable
      Overrides:
      dump in class ContainerLifeCycle
      Parameters:
      out - The appendable to dump to
      indent - The indent to apply after any new lines.
      Throws:
      IOException - if unable to write to Appendable

    • getTransport

      public HttpClientTransport getTransport()

    • getSslContextFactory

      public SslContextFactory getSslContextFactory()
      Returns:
      the SslContextFactory that manages TLS encryption
      See Also:

    • doStart

      protected void doStart() throws Exception
      Description copied from class: ContainerLifeCycle
      Starts the managed lifecycle beans in the order they were added.
      Overrides:
      doStart in class ContainerLifeCycle
      Throws:
      Exception

    • newCookieManager

      private CookieManager newCookieManager()

    • doStop

      protected void doStop() throws Exception
      Description copied from class: ContainerLifeCycle
      Stops the managed lifecycle beans in the reverse order they were added.
      Overrides:
      doStop in class ContainerLifeCycle
      Throws:
      Exception

    • getRequestListeners

      public List<Request.Listener> getRequestListeners()
      Returns a non thread-safe list of Request.Listeners that can be modified before performing requests.
      Returns:
      a list of Request.Listener that can be used to add and remove listeners

    • getCookieStore

      public CookieStore getCookieStore()
      Returns:
      the cookie store associated with this instance

    • setCookieStore

      public void setCookieStore(CookieStore cookieStore)
      Parameters:
      cookieStore - the cookie store associated with this instance

    • getCookieManager

      CookieManager getCookieManager()
      Keep this method package-private because its interface is so ugly that we really don't want to expose it more than strictly needed.
      Returns:
      the cookie manager

    • getAuthenticationStore

      public AuthenticationStore getAuthenticationStore()
      Returns:
      the authentication store associated with this instance

    • setAuthenticationStore

      public void setAuthenticationStore(AuthenticationStore authenticationStore)
      Parameters:
      authenticationStore - the authentication store associated with this instance

    • getContentDecoderFactories

      public Set<ContentDecoder.Factory> getContentDecoderFactories()
      Returns a non thread-safe set of ContentDecoder.Factorys that can be modified before performing requests.
      Returns:
      a set of ContentDecoder.Factory that can be used to add and remove content decoder factories

    • GET

      public ContentResponse GET(String uri) throws InterruptedException, ExecutionException, TimeoutException
      Performs a GET request to the specified URI.
      Parameters:
      uri - the URI to GET
      Returns:
      the ContentResponse for the request
      Throws:
      InterruptedException - if send threading has been interrupted
      ExecutionException - the execution failed
      TimeoutException - the send timed out
      See Also:

    • GET

      public ContentResponse GET(URI uri) throws InterruptedException, ExecutionException, TimeoutException
      Performs a GET request to the specified URI.
      Parameters:
      uri - the URI to GET
      Returns:
      the ContentResponse for the request
      Throws:
      InterruptedException - if send threading has been interrupted
      ExecutionException - the execution failed
      TimeoutException - the send timed out
      See Also:

    • FORM

      public ContentResponse FORM(String uri, Fields fields) throws InterruptedException, ExecutionException, TimeoutException
      Performs a POST request to the specified URI with the given form parameters.
      Parameters:
      uri - the URI to POST
      fields - the fields composing the form name/value pairs
      Returns:
      the ContentResponse for the request
      Throws:
      InterruptedException - if send threading has been interrupted
      ExecutionException - the execution failed
      TimeoutException - the send timed out

    • FORM

      public ContentResponse FORM(URI uri, Fields fields) throws InterruptedException, ExecutionException, TimeoutException
      Performs a POST request to the specified URI with the given form parameters.
      Parameters:
      uri - the URI to POST
      fields - the fields composing the form name/value pairs
      Returns:
      the ContentResponse for the request
      Throws:
      InterruptedException - if send threading has been interrupted
      ExecutionException - the execution failed
      TimeoutException - the send timed out

    • POST

      public Request POST(String uri)
      Creates a POST request to the specified URI.
      Parameters:
      uri - the URI to POST to
      Returns:
      the POST request
      See Also:

    • POST

      public Request POST(URI uri)
      Creates a POST request to the specified URI.
      Parameters:
      uri - the URI to POST to
      Returns:
      the POST request

    • newRequest

      public Request newRequest(String host, int port)
      Creates a new request with the "http" scheme and the specified host and port
      Parameters:
      host - the request host
      port - the request port
      Returns:
      the request just created

    • newRequest

      public Request newRequest(String uri)
      Creates a new request with the specified absolute URI in string format.
      Parameters:
      uri - the request absolute URI
      Returns:
      the request just created

    • newRequest

      public Request newRequest(URI uri)
      Creates a new request with the specified absolute URI.
      Parameters:
      uri - the request absolute URI
      Returns:
      the request just created

    • copyRequest

      protected Request copyRequest(HttpRequest oldRequest, URI newURI)

    • newHttpRequest

      protected HttpRequest newHttpRequest(HttpConversation conversation, URI uri)

    • checkHost

      private URI checkHost(URI uri)

      Checks uri for the host to be non-null host.

      URIs built from strings that have an internationalized domain name (IDN) are parsed without errors, but uri.getHost() returns null.

      Parameters:
      uri - the URI to check for non-null host
      Returns:
      the same uri if the host is non-null
      Throws:
      IllegalArgumentException - if the host is null

    • getDestination

      public Destination getDestination(String scheme, String host, int port)
      Returns a Destination for the given scheme, host and port. Applications may use Destinations to create Connections that will be outside HttpClient's pooling mechanism, to explicitly control the connection lifecycle (in particular their termination with Connection.close()).
      Parameters:
      scheme - the destination scheme
      host - the destination host
      port - the destination port
      Returns:
      the destination
      See Also:

    • destinationFor

      protected HttpDestination destinationFor(String scheme, String host, int port)

    • resolveDestination

      protected HttpDestination resolveDestination(String scheme, String host, int port, Object tag)

    • createOrigin

      protected Origin createOrigin(String scheme, String host, int port, Object tag)

    • resolveDestination

      public HttpDestination resolveDestination(Origin origin)

      Returns, creating it if absent, the destination with the given origin.

      Parameters:
      origin - the origin that identifies the destination
      Returns:
      the destination for the given origin

    • removeDestination

      protected boolean removeDestination(HttpDestination destination)

    • getDestinations

      public List<Destination> getDestinations()
      Returns:
      the list of destinations known to this HttpClient.

    • send

      protected void send(HttpRequest request, List<Response.ResponseListener> listeners)

    • newConnection

      protected void newConnection(HttpDestination destination, Promise<Connection> promise)

    • newConversation

      private HttpConversation newConversation()

    • getProtocolHandlers

      public ProtocolHandlers getProtocolHandlers()

    • findProtocolHandler

      protected ProtocolHandler findProtocolHandler(Request request, Response response)

    • getByteBufferPool

      public ByteBufferPool getByteBufferPool()
      Returns:
      the ByteBufferPool of this HttpClient

    • setByteBufferPool

      public void setByteBufferPool(ByteBufferPool byteBufferPool)
      Parameters:
      byteBufferPool - the ByteBufferPool of this HttpClient

    • getName

      @ManagedAttribute("The name of this HttpClient") public String getName()
      Returns:
      the name of this HttpClient

    • setName

      public void setName(String name)

      Sets the name of this HttpClient.

      The name is also used to generate the JMX ObjectName of this HttpClient and must be set before the registration of the HttpClient MBean in the MBeanServer.

      Parameters:
      name - the name of this HttpClient

    • getConnectTimeout

      @ManagedAttribute("The timeout, in milliseconds, for connect() operations") public long getConnectTimeout()
      Returns:
      the max time, in milliseconds, a connection can take to connect to destinations. Zero value means infinite timeout.

    • setConnectTimeout

      public void setConnectTimeout(long connectTimeout)
      Parameters:
      connectTimeout - the max time, in milliseconds, a connection can take to connect to destinations. Zero value means infinite timeout.
      See Also:

    • getAddressResolutionTimeout

      public long getAddressResolutionTimeout()
      Returns:
      the timeout, in milliseconds, for the default SocketAddressResolver created at startup
      See Also:

    • setAddressResolutionTimeout

      public void setAddressResolutionTimeout(long addressResolutionTimeout)

      Sets the socket address resolution timeout used by the default SocketAddressResolver created by this HttpClient at startup.

      For more fine tuned configuration of socket address resolution, see setSocketAddressResolver(SocketAddressResolver).

      Parameters:
      addressResolutionTimeout - the timeout, in milliseconds, for the default SocketAddressResolver created at startup
      See Also:

    • getIdleTimeout

      @ManagedAttribute("The timeout, in milliseconds, to close idle connections") public long getIdleTimeout()
      Returns:
      the max time, in milliseconds, a connection can be idle (that is, without traffic of bytes in either direction)

    • setIdleTimeout

      public void setIdleTimeout(long idleTimeout)
      Parameters:
      idleTimeout - the max time, in milliseconds, a connection can be idle (that is, without traffic of bytes in either direction)

    • getBindAddress

      public SocketAddress getBindAddress()
      Returns:
      the address to bind socket channels to
      See Also:

    • setBindAddress

      public void setBindAddress(SocketAddress bindAddress)
      Parameters:
      bindAddress - the address to bind socket channels to
      See Also:

    • getUserAgentField

      public HttpField getUserAgentField()
      Returns:
      the "User-Agent" HTTP field of this HttpClient

    • setUserAgentField

      public void setUserAgentField(HttpField agent)
      Parameters:
      agent - the "User-Agent" HTTP header string of this HttpClient

    • isFollowRedirects

      @ManagedAttribute("Whether HTTP redirects are followed") public boolean isFollowRedirects()
      Returns:
      whether this HttpClient follows HTTP redirects
      See Also:

    • setFollowRedirects

      public void setFollowRedirects(boolean follow)
      Parameters:
      follow - whether this HttpClient follows HTTP redirects
      See Also:

    • getExecutor

      public Executor getExecutor()
      Returns:
      the Executor of this HttpClient

    • setExecutor

      public void setExecutor(Executor executor)
      Parameters:
      executor - the Executor of this HttpClient

    • getScheduler

      public Scheduler getScheduler()
      Returns:
      the Scheduler of this HttpClient

    • setScheduler

      public void setScheduler(Scheduler scheduler)
      Parameters:
      scheduler - the Scheduler of this HttpClient

    • getSocketAddressResolver

      public SocketAddressResolver getSocketAddressResolver()
      Returns:
      the SocketAddressResolver of this HttpClient

    • setSocketAddressResolver

      public void setSocketAddressResolver(SocketAddressResolver resolver)
      Parameters:
      resolver - the SocketAddressResolver of this HttpClient

    • getMaxConnectionsPerDestination

      @ManagedAttribute("The max number of connections per each destination") public int getMaxConnectionsPerDestination()
      Returns:
      the max number of connections that this HttpClient opens to Destinations

    • setMaxConnectionsPerDestination

      public void setMaxConnectionsPerDestination(int maxConnectionsPerDestination)
      Sets the max number of connections to open to each destinations.

      RFC 2616 suggests that 2 connections should be opened per each destination, but browsers commonly open 6. If this HttpClient is used for load testing, it is common to have only one destination (the server to load test), and it is recommended to set this value to a high value (at least as much as the threads present in the executor).

      Parameters:
      maxConnectionsPerDestination - the max number of connections that this HttpClient opens to Destinations

    • getMaxRequestsQueuedPerDestination

      @ManagedAttribute("The max number of requests queued per each destination") public int getMaxRequestsQueuedPerDestination()
      Returns:
      the max number of requests that may be queued to a Destination.

    • setMaxRequestsQueuedPerDestination

      public void setMaxRequestsQueuedPerDestination(int maxRequestsQueuedPerDestination)
      Sets the max number of requests that may be queued to a destination.

      If this HttpClient performs a high rate of requests to a destination, and all the connections managed by that destination are busy with other requests, then new requests will be queued up in the destination. This parameter controls how many requests can be queued before starting to reject them. If this HttpClient is used for load testing, it is common to have this parameter set to a high value, although this may impact latency (requests sit in the queue for a long time before being sent).

      Parameters:
      maxRequestsQueuedPerDestination - the max number of requests that may be queued to a Destination.

    • getRequestBufferSize

      @ManagedAttribute("The request buffer size") public int getRequestBufferSize()
      Returns:
      the size of the buffer used to write requests

    • setRequestBufferSize

      public void setRequestBufferSize(int requestBufferSize)
      Parameters:
      requestBufferSize - the size of the buffer used to write requests

    • getResponseBufferSize

      @ManagedAttribute("The response buffer size") public int getResponseBufferSize()
      Returns:
      the size of the buffer used to read responses

    • setResponseBufferSize

      public void setResponseBufferSize(int responseBufferSize)
      Parameters:
      responseBufferSize - the size of the buffer used to read responses

    • getMaxRedirects

      public int getMaxRedirects()
      Returns:
      the max number of HTTP redirects that are followed in a conversation
      See Also:

    • setMaxRedirects

      public void setMaxRedirects(int maxRedirects)
      Parameters:
      maxRedirects - the max number of HTTP redirects that are followed in a conversation, or -1 for unlimited redirects
      See Also:

    • isTCPNoDelay

      @ManagedAttribute(value="Whether the TCP_NODELAY option is enabled", name="tcpNoDelay") public boolean isTCPNoDelay()
      Returns:
      whether TCP_NODELAY is enabled

    • setTCPNoDelay

      public void setTCPNoDelay(boolean tcpNoDelay)
      Parameters:
      tcpNoDelay - whether TCP_NODELAY is enabled
      See Also:

    • isDispatchIO

      @Deprecated public boolean isDispatchIO()
      Deprecated.
      Returns:
      true to dispatch I/O operations in a different thread, false to execute them in the selector thread
      See Also:

    • setDispatchIO

      @Deprecated public void setDispatchIO(boolean dispatchIO)
      Deprecated.
      Whether to dispatch I/O operations from the selector thread to a different thread.

      This implementation never blocks on I/O operation, but invokes application callbacks that may take time to execute or block on other I/O. If application callbacks are known to take time or block on I/O, then parameter dispatchIO should be set to true. If application callbacks are known to be quick and never block on I/O, then parameter dispatchIO may be set to false.

      Parameters:
      dispatchIO - true to dispatch I/O operations in a different thread, false to execute them in the selector thread

    • getHttpCompliance

      public HttpCompliance getHttpCompliance()
      Gets the http compliance mode for parsing http responses. The default http compliance level is HttpCompliance.RFC7230 which is the latest HTTP/1.1 specification
      Returns:
      the HttpCompliance instance

    • setHttpCompliance

      public void setHttpCompliance(HttpCompliance httpCompliance)
      Sets the http compliance mode for parsing http responses. This affect how weak the HttpParser parses http responses and which http protocol level is supported
      Parameters:
      httpCompliance - The compliance level which is used to actually parse http responses

    • isStrictEventOrdering

      @ManagedAttribute("Whether request/response events must be strictly ordered") public boolean isStrictEventOrdering()
      Returns:
      whether request events must be strictly ordered
      See Also:

    • setStrictEventOrdering

      public void setStrictEventOrdering(boolean strictEventOrdering)
      Whether request/response events must be strictly ordered with respect to connection usage.

      From the point of view of connection usage, the connection can be reused just before the "complete" event notified to Response.CompleteListeners (but after the "success" event).

      When a request/response exchange is completing, the destination may have another request queued to be sent to the server. If the connection for that destination is reused for the second request before the "complete" event of the first exchange, it may happen that the "begin" event of the second request happens before the "complete" event of the first exchange.

      Enforcing strict ordering of events so that a "begin" event of a request can never happen before the "complete" event of the previous exchange comes with the cost of increased connection usage. In case of HTTP redirects and strict event ordering, for example, the redirect request will be forced to open a new connection because it is typically sent from the complete listener when the connection cannot yet be reused. When strict event ordering is not enforced, the redirect request will reuse the already open connection making the system more efficient.

      The default value for this property is false.

      Parameters:
      strictEventOrdering - whether request/response events must be strictly ordered

    • isRemoveIdleDestinations

      @ManagedAttribute("Whether idle destinations are removed") public boolean isRemoveIdleDestinations()
      Returns:
      whether destinations that have no connections should be removed
      See Also:

    • setRemoveIdleDestinations

      public void setRemoveIdleDestinations(boolean removeIdleDestinations)
      Whether destinations that have no connections (nor active nor idle) should be removed.

      Applications typically make request to a limited number of destinations so keeping destinations around is not a problem for the memory or the GC. However, for applications that hit millions of different destinations (e.g. a spider bot) it would be useful to be able to remove the old destinations that won't be visited anymore and leave space for new destinations.

      Parameters:
      removeIdleDestinations - whether destinations that have no connections should be removed
      See Also:

    • isConnectBlocking

      @ManagedAttribute("Whether the connect() operation is blocking") public boolean isConnectBlocking()
      Returns:
      whether connect() operations are performed in blocking mode

    • setConnectBlocking

      public void setConnectBlocking(boolean connectBlocking)

      Whether connect() operations are performed in blocking mode.

      If connect() are performed in blocking mode, then Socket.connect(SocketAddress, int) will be used to connect to servers.

      Otherwise, SocketChannel.connect(SocketAddress) will be used in non-blocking mode, therefore registering for SelectionKey.OP_CONNECT and finishing the connect operation when the NIO system emits that event.

      Parameters:
      connectBlocking - whether connect() operations are performed in blocking mode

    • getDefaultRequestContentType

      @ManagedAttribute("The default content type for request content") public String getDefaultRequestContentType()
      Returns:
      the default content type for request content

    • setDefaultRequestContentType

      public void setDefaultRequestContentType(String contentType)
      Parameters:
      contentType - the default content type for request content

    • getProxyConfiguration

      public ProxyConfiguration getProxyConfiguration()
      Returns:
      the forward proxy configuration

    • getAcceptEncodingField

      protected HttpField getAcceptEncodingField()

    • normalizeHost

      @Deprecated protected String normalizeHost(String host)
      Deprecated.
      no replacement, do not use it
      Parameters:
      host - the host to normalize
      Returns:
      the host itself

    • normalizePort

      public static int normalizePort(String scheme, int port)

    • isDefaultPort

      public boolean isDefaultPort(String scheme, int port)

    • isSchemeSecure

      static boolean isSchemeSecure(String scheme)

    • newSslClientConnectionFactory

      @Deprecated protected ClientConnectionFactory newSslClientConnectionFactory(ClientConnectionFactory connectionFactory)
      Deprecated.
      use newSslClientConnectionFactory(SslContextFactory, ClientConnectionFactory) instead
      Creates a new SslClientConnectionFactory wrapping the given connection factory.
      Parameters:
      connectionFactory - the connection factory to wrap
      Returns:
      a new SslClientConnectionFactory

    • newSslClientConnectionFactory

      protected ClientConnectionFactory newSslClientConnectionFactory(SslContextFactory sslContextFactory, ClientConnectionFactory connectionFactory)