Because HTTP is layered directly on TCP, the performance of HTTPtransactions depends critically on the performance of the underlyingTCP plumbing. This section highlights some significant performanceconsiderations of these TCP connections. By understanding some of thebasic performance characteristics of TCP, you’llbetter appreciate HTTP’s connection optimizationfeatures, and you’ll be able to design and implementhigher-performance HTTP applications.
This section requires some understanding of the internal details ofthe TCP protocol. If you are not interested in (or are comfortablewith) the details of TCP performance considerations, feel free toskip ahead to Section 4.3. BecauseTCP is a complex topic, we can provide only a brief overview of TCPperformance here. Refer to Section 4.8 at the end of this chapterfor a list of excellent TCP references.
Let’s start our TCPperformance tour by reviewing what networking delays occur in thecourse of an HTTP request. Figure 4-7 depicts themajor connect, transfer, and processing delays for an HTTPtransaction.
Notice that the transaction processing time can be quite smallcompared to the time required to set up TCP connections and transferthe request and response messages. Unless the client or server isoverloaded or executing complex dynamic resources, most HTTP delaysare caused by TCP network delays.
There are several possible causes of delay in an HTTP transaction:
A client first needs to determine the IP address and port number ofthe web server from the URI. If the hostname in the URI was notrecently visited, it may take tens of seconds to convert the hostnamefrom a URI into an IP address using the DNS resolutioninfrastructure.[3]
Next, the client sends a TCP connection request to the server andwaits for the server to send back a connection acceptance reply.Connection setup delay occurs for every new TCP connection. Thisusually takes at most a second or two, but it can add up quickly whenhundreds of HTTP transactions are made.
Once the connection is established, the client sends the HTTP requestover the newly established TCP pipe. The web server reads the requestmessage from the TCP connection as the data arrives and processes therequest. It takes time for the request message to travel over theInternet and get processed by the server.
The web server then writes back the HTTP response, which also takestime.
The magnitude of these TCP network delays depends on hardware speed,the load of the network and server, the size of the request andresponse messages, and the distance between client and server. Thedelays also are significantly affected by technical intricacies ofthe TCP protocol.
The remainder of this section outlines some of the most commonTCP-related delays affecting HTTPprogrammers, including the causes and performance impacts of:
The TCP connection setup handshake
TCP slow-start congestion control
Nagle’s algorithm for data aggregation
TCP’s delayed acknowledgment algorithm forpiggybacked acknowledgments
TIME_WAIT delays and port exhaustion
If you are writing high-performance HTTP software, you shouldunderstand each of these factors. If you don’t needthis level of performance optimization, feel free to skip ahead.
When you set up a new TCP connection,even before you send any data, the TCP software exchanges a series ofIP packets to negotiate the terms of the connection (see Figure 4-8). These exchanges can significantly degradeHTTP performance if the connections are used for small datatransfers.
Here are the steps in the TCP connection handshake:
To request a new TCP connection, the client sends a small TCP packet(usually 40-60 bytes) to the server. The packet has a special“SYN” flag set, which meansit’s a connection request. This is shown in Figure 4-8a.
If the server accepts the connection, it computes some connectionparameters and sends a TCP packet back to the client, with both the“SYN” and“ACK” flags set, indicating thatthe connection request is accepted (see Figure 4-8b).
Finally, the client sends an acknowledgment back to the server,letting it know that the connection was established successfully (seeFigure 4-8c). Modern TCP stacks let the client senddata in this acknowledgment packet.
The HTTP programmer never sees these packets—they are managedinvisibly by the TCP/IP software. All the HTTP programmer sees is adelay when creating a new TCP connection.
The SYN/SYN+ACK handshake (Figure 4-8a and b)creates a measurable delay when HTTP transactions do not exchangemuch data, as is commonly the case. The TCP connect ACK packet (Figure 4-8c) often is large enough to carry the entireHTTP request message,[4] and many HTTP server response messagesfit into a single IP packet (e.g., when the response is a small HTMLfile of a decorative graphic, or a 304 Not Modified response to abrowser cache request).
The end result is that small HTTP transactions may spend 50% or moreof their time doing TCP setup. Later sections will discuss how HTTPallows reuse of existing connections to eliminate the impact of thisTCP setup delay.
Because the Internet itselfdoes not guarantee reliable packet delivery (Internet routers arefree to destroy packets at will if they are overloaded), TCPimplements its own acknowledgment scheme to guarantee successful datadelivery.
Each TCP segment gets a sequence number and a data-integritychecksum. The receiver of each segment returns small acknowledgmentpackets back to the sender when segments have been received intact.If a sender does not receive an acknowledgment within a specifiedwindow of time, the sender concludes the packet was destroyed orcorrupted and resends the data.
Because acknowledgments are small, TCP allows them to“piggyback” on outgoing datapackets heading in the same direction. By combining returningacknowledgments with outgoing data packets, TCP can make moreefficient use of the network. To increase the chances that anacknowledgment will find a data packet headed in the same direction,many TCP stacks implement a “delayedacknowledgment” algorithm. Delayed acknowledgmentshold outgoing acknowledgments in a buffer for a certain window oftime (usually 100-200 milliseconds), looking for an outgoing datapacket on which to piggyback. If no outgoing data packet arrives inthat time, the acknowledgment is sent in its own packet.
Unfortunately, the bimodal request-reply behavior of HTTP reduces thechances that piggybacking can occur. There justaren’t many packets heading in the reverse directionwhen you want them. Frequently, the delayed acknowledgmentalgorithms introduce significant delays. Depending on your operatingsystem, you may be able to adjust or disable the delayedacknowledgment algorithm.
Before you modify any parameters of your TCP stack, be sure you knowwhat you are doing. Algorithms inside TCP were introduced to protectthe Internet from poorly designed applications. If you modify any TCPconfigurations, be absolutely sure your application will not createthe problems the algorithms were designed to avoid.
The performance of TCP data transferalso depends on the age of the TCP connection.TCP connections “tune” themselvesover time, initially limiting the maximum speed of the connection andincreasing the speed over time as data is transmitted successfully.This tuning is called TCP slow start, and it isused to prevent sudden overloading and congestion of the Internet.
TCP slow start throttles the number of packets a TCP endpoint canhave in flight at any one time. Put simply, each time a packet isreceived successfully, the sender gets permission to send two morepackets. If an HTTP transaction has a large amount of data to send,it cannot send all the packets at once. It must send one packet andwait for an acknowledgment; then it can send two packets, each ofwhich must be acknowledged, which allows four packets, etc. This iscalled “opening the congestionwindow.”
Because of this congestion-control feature, new connections areslower than “tuned” connectionsthat already have exchanged a modest amount of data. Because tunedconnections are faster, HTTP includes facilities that let you reuseexisting connections. We’ll talk about these HTTP“persistent connections” later inthis chapter.
TCP has a data stream interface thatpermits applications to stream data of any size to the TCPstack—even a single byte at a time! But because each TCPsegment carries at least 40 bytes of flags and headers, networkperformance can be degraded severely if TCP sends large numbers ofpackets containing small amounts of data.[5]
Nagle’s algorithm (named for its creator, JohnNagle) attempts to bundle up a large amount of TCP data beforesending a packet, aiding network efficiency. The algorithm isdescribed in RFC 896, “Congestion Control in IP/TCPInternetworks.”
Nagle’s algorithm discourages the sending ofsegments that are not full-size (a maximum-size packet is around1,500 bytes on a LAN, or a few hundred bytes across the Internet).Nagle’s algorithm lets you send a non-full-sizepacket only if all other packets have been acknowledged. If otherpackets are still in flight, the partial data is buffered. Thisbuffered data is sent only when pending packets are acknowledged orwhen the buffer has accumulated enough data to send a fullpacket.[6]
Nagle’s algorithm causes several HTTP performanceproblems. First, small HTTP messages may not fill a packet, so theymay be delayed waiting for additional data that will never arrive.Second, Nagle’s algorithm interacts poorly withdelayed acknowledgments—Nagle’s algorithmwill hold up the sending of data until an acknowledgment arrives, butthe acknowledgment itself will be delayed 100-200 milliseconds by thedelayed acknowledgment algorithm.[7]
HTTP applications often disable Nagle’s algorithm toimprove performance, by setting the TCP_NODELAY parameter on theirstacks. If you do this, you must ensure that you write large chunksof data to TCP so you don’t create a flurry of smallpackets.
TIME_WAIT port exhaustion is aserious performance problem that affects performance benchmarking butis relatively uncommon in real deployments. It warrants specialattention because most people involved in performance benchmarkingeventually run into this problem and get unexpectedly poorperformance.
When a TCP endpoint closes a TCP connection, it maintains in memory asmall control block recording the IP addresses and port numbers ofthe recently closed connection. This information is maintained for ashort time, typically around twice the estimated maximum segmentlifetime (called "2MSL”; often twominutes[8]), tomake sure a new TCP connection with the same addresses and portnumbers is not created during this time. This prevents any strayduplicate packets from the previous connection from accidentallybeing injected into a new connection that has the same addresses andport numbers. In practice, this algorithm prevents two connectionswith the exact same IP addresses and port numbers from being created,closed, and recreated within two minutes.
Today’s higher-speed routers make it extremelyunlikely that a duplicate packet will show up on aserver’s doorstep minutes after a connection closes.Some operating systems set 2MSL to a smaller value, but be carefulabout overriding this value. Packets do get duplicated, and TCP datawill be corrupted if a duplicate packet from a past connection getsinserted into a new stream with the same connection values.
The 2MSL connection close delay normally is not a problem, but inbenchmarking situations, it can be. It’s common thatonly one or a few test load-generation computers are connecting to asystem under benchmark test, which limits the number of client IPaddresses that connect to the server. Furthermore, the servertypically is listening on HTTP’s default TCP port,80. These circ*mstances limit the available combinations ofconnection values, at a time when port numbers are blocked from reuseby TIME_WAIT.
In a pathological situation with one client and one web server, ofthe four values that make up a TCP connection:
<source-IP-address, source-port, destination-IP-address, destination-port>
three of them are fixed—only the source port is free to change:
<client-IP, source-port, server-IP, 80>
Each time the client connects to the server, it gets a new sourceport in order to have a unique connection. But because a limitednumber of source ports are available (say, 60,000) and no connectioncan be reused for 2MSL seconds (say, 120 seconds), this limits theconnect rate to 60,000 / 120 = 500 transactions/sec. If you keepmaking optimizations, and your server doesn’t getfaster than about 500 transactions/sec, make sure you are notexperiencing TIME_WAIT port exhaustion. You can fix this problem byusing more client load-generator machines or making sure the clientand server rotate through several virtual IP addresses to add moreconnection combinations.
Even if you do not suffer port exhaustion problems, be careful abouthaving large numbers of open connections or large numbers of controlblocks allocated for connection in wait states. Some operatingsystems slow down dramatically when there are numerous openconnections or control blocks.
[3] Luckily, most HTTP clients keep asmall DNS cache of IP addresses for recently accessed sites. When theIP address is already “cached”(recorded) locally, the lookup is instantaneous. Because most webbrowsing is to a small number of popular sites, hostnames usually areresolved very quickly.
[4] IP packets are usually a fewhundred bytes for Internet traffic and around 1,500 bytes for localtraffic.
[5] Sending astorm of single-byte packets is called “sender sillywindow syndrome.” This is inefficient, anti-social,and can be disruptive to other Internet traffic.
[6] Several variations of this algorithm exist,including timeouts and acknowledgment logic changes, but the basicalgorithm causes buffering of data smaller than a TCP segment.
[7] These problems canbecome worse when using pipelined connections (described later inthis chapter), because clients may have several messages to send tothe same server and do not want delays.
[8] The 2MSL value of two minutes is historical.Long ago, when routers were much slower, it was estimated that aduplicate copy of a packet might be able to remain queued in theInternet for up to a minute before being destroyed. Today, themaximum segment lifetime is much smaller.
