(Hypertext Transfer Protocols Headers)
Request
GET /announce.html HTTP/1.0
Host: www.joes-hardware.com
Client
HTTP/1.0 200 OK
Content-Type: text/plain
Content-Length:19
Hi! I’m a message
Response
Server





Provide information about request or response, or about
object sent in the message body.
Headers names are followed by a colon (:), followed by
optional whitespace, followed by a value.
Syntax: - “Header-Name: value”
Header name is not case-sensitive.
Headers are terminated by a blank line (CRLF), marking
the end of the list of headers & beginning of the entity
body.






Accept: image/gif, image/jpg, */*
Accept-Encoding: gzip
Connection: Keep-Alive
Host: www.somebookstore.com
Referer: http://www.somebookstore.com/findbooks.html
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.0)





General Headers: Used by both clients & servers e.g.,
Connection, Date etc.
Request Headers: Provides extra information to servers of
request e.g., Accept, Host etc.
Response Headers: Provides information to the client e.g. ,
Server, Age etc.
Entity Headers: Describes size & contents of body, or
resources itself e.g., Content Length, Expires etc.
Extension Headers: Are nonstandard headers that have
been created by application developers that are not
defined in the specification e.g., Client-ip, Cookie etc.
Cache’s Headers









Age
Cache-Control
Expires
If-Modified-Since
If-Match
If-None-Match
If-Range
If-Unmodified
Pragma etc.
Cookies’s Headers
Cookie
Cookie2
Set-Cookie etc.
REQUEST HEADERS





Tells servers what media types are acceptable.
Its value is a list of media types that client can use.
May include a list of quality values(q values) that tell the
server which media is preferred, in case of multiple media
type. Q values range from 0.0 to 1.0
“*/*” represents all types, & “image/*” represents all image
types.
EXAMPLES:
Accept: text/*, image/jpeg;q=1

Allowing clients to inform the server about what languages
are acceptable.

EXAMPLES:
Accept-Language: en
Accept-Language: en;q=0.5, fr;q=0.8

Indicate what character sets are acceptable by client for
response.
Each charset may be given an associated quality value
which represents user's preference. The default value is
q=1.

EXAMPLES:

Accept-Charset: iso-latin-1;q=0.8

Used by clients to tell servers what encoding are
acceptable.

EXAMPLES:
Accept-Encoding: compress,gzip
Accept-Encoding: compress;q=0.5, gzip;q=1.0
Accept-Encoding:

Empty Accept-Encoding means only the uuencoded
content is acceptable.




Sent by a client to authenticate itself with a server.
A client will include this header in its request after
receiving a 401 Authentication Required response from a
server.
BASIC SYNTAX: Authorization: authentication-scheme #
authentication-param.
EXAMPLE:
Authorization: Basic kerjtojrk56kjf





From where the request is coming from.
Gives email address of the user making the request.
Used for logging purposes and as a means for identifying
the source of invalid or unwanted requests.
Client should not send this header without user's approval,
as it might conflict with user's privacy.
EXAMPLE:
From:[email protected]






Used only with the TRACE method(allows clients to see how its request
looks when it finally makes it to the server), to limit no. of proxies or
gateways, or other application intermediaries that a request goes
through.
Its value is an integer.
Each application that receives a Trace request with this header should
decrement the value before it forwards the request.
If it is missing from Trace request, then there is no maximum no. of
forwards.
If value is zero when application receives request, it should send back a
200 OK response, with an entity body containing original request.
EXAMPLE:
Max-Forwards :5






Used by clients to provide the server with the Internet hostname &
port no. of the server from which the client wants the response.
HTTP/1.1 clients must include a Host header in all request.
All HTTP/1.1 servers must respond with the 400 Bad Request status
code to HTTP/1.1 clients that do not provide a Host header.
If the URL contains an IP address, the Host header should contain the
same address.
Web proxies must add Host headers to request messages before
forwarding them.
EXAMPLE: Host: www.hotbot.com:80
Host: www.google.co.in
Any origin server that does differentiate resources based
on the host, must use the following rules for determining
the resource to send on an HTTP/1.1 request:
1)
2)
3)
If the URL is absolute (http://www.abc.com/index.html),
Host header ignored, Host is taken from URL.
If the URL doesn’t have a host, the value is obtained from
Host header.
If no valid host determined through step 1 or 2, 400(Bad
Request) response is send.





Included to let the server know from where the client got
the URL .
E.g., If you get home page A & click on a link to go to page
B, your browser will insert a Referer header in the request
with value A.
Inserted only when you click on links.
Requests for URLs you type yourself will not contain it.
EXAMPLE:
Referer:http://www.in.com/index.html

Used by client applications to identify themselves.
Its value is the product name & possibly describing the
client application.
For example, browser sends the user-agent string which
indicates which browser you are using, its version number,
and details about your system, such as operating system
and its version.

EXAMPLE:


User-Agent: Mozilla/5.0

The Compatibility flag-indicates that Internet Explorer is
compatible with a common set of features.

The Version token-identifies the browser and its version
number.

The Platform token-identifies your operating system and
its version number
Version token









MSIE 8.0
MSIE 7.0
MSIE 6.0
MSIE 6.0b
MSIE 5.5
MSIE 5.01
MSIE 5.0
MSIE 5.0b1
MSIE 4.01
Description
Internet Explorer 8 (pre-release)
Internet Explorer 7
Microsoft Internet Explorer 6
Internet Explorer 6 (pre-release)
Internet Explorer 5.5
Internet Explorer 5.01
Internet Explorer 5
Internet Explorer 5 (pre-release)
Internet Explorer 4.01
Platform token






Windows NT 6.0
Windows NT 5.1
Windows NT 5.0
Windows NT 4.0
Windows 98
Windows 95
Description
Windows Vista
Windows XP
Windows 2000
Microsoft
Windows 98
Windows 95

When AJAX request is made, X-Requested-With header
with
the
value
"XMLHttpRequest"
is
used.

EXAMPLE:
X-Requested-With: XMLHttpRequest
RESPONSE HEADERS





Identifies the server product.
Can contain additional comments about product.
Product tokens are listed in order of their significance for
identifying the application.
If response is being forwarded through a proxy, the proxy
application must not modify it.
EXAMPLE:
Server: Microsoft-Internet-Information-Server/1.0


Used to redirect the client to a location other than the
Request-URI for completion of the request or identification
of a new resource.
For 201 (Created) responses, the Location is that of the
new resource which was created by the request.

BASIC SYNTAX:
Location: absoluteURL

EXAMPLE:
Location: http://www.hotbot.com

Used by servers to tell clients whether they accept
requests for ranges of a resource.

EXAMPLES:

Servers that do not accept any kind of range request for a
resource may send Accept-Ranges: none
to advise the client not to attempt a range request.
Accept-Ranges: bytes





Allows a server to tell a client what methods it supports.
These methods can be used in future requests by the client.
Proxies need to be careful when they receive a response
from a server with the public header.
The header indicates the capabilities of the server, not the
proxy, so the proxy needs to edit or remove the list of
methods.
EXAMPLE:
Public: GET,HEAD,POST,TRACE
GENERAL HEADERS

Tells media type of the object in the message.

EXAMPLE:
Content-Type: text/html
Content-Type: application/x-www-form-urlencoded



Content-Type that GET requests must use, and POST
requests use by default.
Involves percent encoding format that looks something
like: %C3%86.
No official way of determining the character encoding of
such a request, since the percent encoding operates on a
byte level.





In HTTP 1.0, keep-alive means browser can handle
persistent connection.
In HTTP 1.1, persistent connection is by default.
Persistent connections mean that the connection will stay
open across transaction, until either the client or the
server decides to close them.
EXAMPLE:
Connection: close
This token means that the connection is going to be closed
after the response is completed.




To transfer the HTTP message body safely.
Its value is a list of encodings that were performed on the
message body.
If multiple encodings were performed, they are listed in
order.
EXAMPLE:
Transfer-Encoding: chunked



Transfer encoding is performed by a server or other
intermediary application .
Transfer encoding is applied for architectural reasons &
are independent of format of content.
Content-Encoded message just encodes entity section of
message while Transfer-Encoded encodes the entire
message, changing the structure of the message itself.
 If total size of response is not known, then chunked
transfer-encoding is used (only for HTTP 1.1).
 Server breaks the complete response into smaller chunks
and sends them in series.
 A chunked message body contains a series of chunks,
followed by a line with "0" (zero), followed by optional
footers (just like headers), and a blank line. Each chunk
consists of two parts:
◦
size of the chunk data, in hexadecimal followed by CRLF.
◦
the data, followed by CRLF
Response
stream
HTTP Response
HTTP/1.1 200 OK <CR><LF>
Content-Type: text/plain <CR><LF>
Transfer-Encoding: chunked<CR><LF>
Trailer:Content-MD5 <CR><LF>
<CR><LF>
Chunk#1
14<CR><LF>
We hold these truths<CR><LF>
Chunk#2
Last Chunk
12<CR><LF>
To be self-evident<CR><LF>
0<CR><LF>
Trailer*
optional
ContentMD5:njfhrjehn34jbnj5n<CR><LF>
HTTP/1.1 200 OK
Date: Fri, 31 Dec 1999 23:59:59 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
<blank line>
1a; ignore-stuff-here
abcdefghijklmnopqrstuvwxyz
10
1234567890abcdef
0
some-footer: some-value
<blank line>



Gives date & time at which the message was created.
This header is required in servers response because the
time & date at which the server believes the message was
created can be used by caches in evaluating the freshness
of a response.
EXAMPLE:
Date: Tue, 15 Nov 2010 08:12:31 GMT
ENTITY HEADERS



Tells the client the natural language that should be
understood in order to understand the object.
If this header is not present in the response, the object is
intended for all audiences.
EXAMPLE:
Content-Language: en,fr



Included in an HTTP message to give the URL
corresponding to the entity in the message.
For objects that may have multiple URLS, a response
message can include a Content-Location header indicating
the URL of the object used to generate the response.
BASIC SYNTAX:
Content-Location: ( absoluteURL| relativeURL)



Used to specify whether any encodings have been
performed on the object.
Primarily used to allow a document to be compressed
without losing the identity of its underlying media type.
EXAMPLE:
Content-Encoding: compress,gzip



This header gives the length or size of the entity body.
If the header is in a response message to a HEAD HTTP
request, the value of the header indicates the size that the
entity body would have been had it been sent.
EXAMPLE:
Content-Length: 2417

Requests for ranges of a document allow for more efficient
requests of large objects or for recovery from failed
transfers.
Can also be used to request multiple ranges.

EXAMPLE:

Range:bytes=500-1500
Range:bytes=1500(requesting the remainder of the document after first 1500
bytes)
EXTENSION HEADER

Extension header used by some older clients & some
proxies to transmit the IP address of the machine on which
the client is running.

TYPE: Extension request header

EXAMPLE:
Client-ip: 201.1.101.67
HTTP HEADERS SEND BY NETSTORM
NetStorm by default add following headers:
 Accept
 Host
 User-Agent
 Connection
 Accept-Encoding
 Keep-Alive
 Content-Length
And along with the headers of the recorded script.
Following headers do not capture during Script Recording:







User-Agent
Accept-Encoding
Keep-Alive
Referer
Accept
Connection
Host header is added only in script.detail file but not in the
script.capture file.

case X153424_jpg:
think_time = pre_page_X153424_jpg();
web_url (X153424_jpg,
METHOD=GET,
URL=http://cache1.artprintimages.com/images/framing/hires/
1153424/thumb/1153424.jpg,
HEADER=Accept-Language: en-US,
HEADER=Cache-Control: no-transform,
HEADER=UA-CPU: x86,
NUM_EMBED=42);
next_page = check_page_X153424_jpg();
break;
---- HTTP TX_RAT:0 RX_RAT:0
POST /asp/include/TeaLeafTarget.asp HTTP/1.1
Host: www.art.com
Cookie: ENDECA=BCOrder=ArtStyle&bc0=c%7cphotography%7c23946;art=presc=1&
addtocart=N&asPurchased=false&sc=1;CustSessionID=E670CFE9345;sessionid=E670CFE9,
Accept-Language: en-us
Content-Type: text/xml
UA-CPU: x86
Content-Length: 1779
Cache-Control: no-cache
<ClientEvent
count="1“
Type="PERFORMANCE"
SubType="INIT"
PageId="ID18H26M37S591"TimeDuration="3753" DateSince1970="1279198601344" >
</ClientEvent>

----
For performance purposes we can also disable headers by
adding following keywords in scenario file.







G_DISABLE_ACCEPT_HEADER
G_DISABLE_HOST_HEADER
G_DISABLE_UA_HEADER
G_DISABLE_CONNECTION_HEADER
G_DISABLE_ACCEPT_ENC_HEADER
G_DISABLE_KA_HEADER
G_DISABLE_ALL_HEADER


By default, NetStorm do not send Referer header.
You can add this header in scenario file by keyword
G_ENABLE_REFERER

Case 1: Recorded Host, no Server mapping.

For example: URL is www.yahoo.com then the Host header
is www.yahoo.com only.

Case 2: Recorded Host, simple server mapping.

For example: if recorded host is www.yahoo.com &
following simple mapping is used in scenario file:
SERVER_HOST www.yahoo.com 192.168.1.36 –

then the Host header will be
Host: 192.168.1.36

Case 3: Recorded Host, complex mapping.

For example: if recorded host is www.yahoo.com & following
complex mapping is used in scenario file:
SERVER_HOST www.yahoo.com 192.168.1.36 – 192.168.1.40 –

then the NetStorm will randomly select one IP from these two
IP’s & that IP will be the Host header value.




Sometimes you may need Recorded Host in Host header.
For example, HPD supports virtual host.
To enable Recorded Host in Host header, add this keyword
in scenario file
G_USE_RECORDED_HOST_IN_HOST_HDR
This will give the recorded host in host header irrespective
of server mapping.
HTTP HEADERS SEND BY NETOCEAN
NetOcean by default sends following headers:


Content-Length
Content-Type (If file extension is not in extension.type of
conf directory, then Content-Type header is not send in the
URL response.)
You can also add headers for static/CGI URL which are as
follow:
1) GLOBAL HEADER: To send additional headers for all URL
accessed, HPD supports additional global headers. These
headers can be added in the global_headers file in
following directory
cd /var/www/hpd/conf
2) URL SPECIFIC HEADERS FOR STATIC URLS:





You can add additional headers based on the URL in URL
control file.
Go to this directoy
/var/www/hpd/control/default/
Make directory as per the path of the URL. E.g.
/custom/samples/url_specific_hdrs/url_specific_hdr.html
Add desired header in URL control file.
For example:
Content-Encoding: gzip
Global header for all correlation Services
Add in correlaton.conf
Service based additional headers
Add in the service.conf file of the service.
For example:
SET_CR_URL_HEADER MyHdr: value








What are Quality values?
What is the default value of quality values?
What is Trace method?
What is the reason phrase for 201 status code?
What is the name of the header whose value is
application/x-www-form-urlencoded?
If the value of Accept-Encoding header is empty then what
does it mean?
If no valid host is determined through URL & Host header
then what status code will be send by the server?
What are tokens of the user-agent string in User-Agent
header?




How the Transfer-Encoding header differs from the
Content-Encoding?
What does each chunk contains in the Chunked TranferEncoding?
What is the form of length value in a chunk of Chunked
Tranfer-Encoding?
If the value of Max-Forwards header is zero when
application receives the request, then what response
should it send back ?
THANK YOU