schlitt.info - php, photography and private stuff
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:Author: Tobias Schlitt
:Date: Wed, 19 Nov 2008 23:29:46 +0100
:Revision: 1
:Copyright: CC by-nc-sa
===================================
Sending HEAD requests with ext/curl
===================================
:Description:
ext/curl is the common tool of choice, if one needs to perform more
advanced HTTP requests from a PHP script (for simple ones, use a stream!).
I recently wanted to perform a HEAD request to a file, after which I
wanted to perform some more advanced HTTP interaction, so CURL was also
the tool of choice here.
`ext/curl`__ is the common tool of choice, if one needs to perform more
advanced HTTP requests from a PHP script (for simple ones, use a `stream`__!).
I recently wanted to perform a HEAD request to a file, after which I wanted to
perform some more advanced HTTP interaction, so `CURL`__ was also the tool of
choice here.
.. __: http://php.net/curl
.. __: http://php.net/stream
.. __: http://curl.haxx.se/
Trying it out on the shell with a local web server, CURL was operating quite
slow, in contrast to a GET request. The *-i* command line switch makes curl
include the headers in the printed output, *-X* lets you define a custom HTTP
request.
::
dotxp@tango ~ $ time curl -i -X HEAD http://localhost/admin/
HTTP/1.1 200 OK
X-Powered-By: PHP/5.2.7-dev
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:10:59 GMT
Server: lighttpd/1.4.19
real 0m6.079s
user 0m0.004s
sys 0m0.000s
::
dotxp@tango ~ $ time curl -i -X GET http://localhost/admin/
HTTP/1.1 200 OK
Transfer-Encoding: chunked
X-Powered-By: PHP/5.2.7-dev
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:12:27 GMT
Server: lighttpd/1.4.19
real 0m0.180s
user 0m0.004s
sys 0m0.000s
A difference of 6 seconds runtime of a HEAD in contrast to 0.2 seconds for a
GET is quite contrary to the original idea of a HEAD request. HEAD is used to
just receive the headers of an URI instead of receiving the whole contents, to
save bandwidth, memory and execution time.
ext/curl showed the exact same problem. Fiddling a bit with the command line
switches, I found to replace *-i* with *-I* which makes curl print only the
headers, but not the body of the response.
::
dotxp@tango ~ $ time curl -I -X HEAD http://localhost/admin/
HTTP/1.1 200 OK
X-Powered-By: PHP/5.2.7-dev
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:19:05 GMT
Server: lighttpd/1.4.19
real 0m0.044s
user 0m0.004s
sys 0m0.000s
0.04 seconds is now even faster than the corresponding GET request, with the
-I switch, which took me 0.09 seconds. Now I just needed to transfer the
command line options to the corresponding ext/curl ones:
::
$c = curl_init();
curl_setopt( $c, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $c, CURLOPT_CUSTOMREQUEST, 'HEAD' );
curl_setopt( $c, CURLOPT_HEADER, 1 );
curl_setopt( $c, CURLOPT_NOBODY, true );
curl_setopt( $c, CURLOPT_URL, 'http://localhost/admin/' );
$res = curl_exec( $c );
The *RETURNTRANSFER* makes ext/curl return the HTTP response instead of
printing it. Using the *CUSTOMREQUEST* option you define to send a HEAD
request instead of a standard GET or POST request. The *HEADER* option makes
ext/curl include the response headers in the return value of *curl_exec()*
call and NOBODY avoids the inclusion of the body content here. The *URL*
option as usually sets the URL to request and *curl_exec()* makes ext/curl
execute the request.
The runtime was even a fraction of a second faster here, compared to the
command line version, but that can be subjectively. However, the HEAD request
works as expected now. Maybe it's useful for someone to know this.
..
Local Variables:
mode: rst
fill-column: 79
End:
vim: et syn=rst tw=79
Trackbacks
==========
Comments
========
- adnan. at Mon, 23 Jun 2008 12:43:13 +0200
if the curl statements were run in succession, is it not possible that
server caching caused the remaining calls to be faster?
maybe queries are getting cached at the mysql level? or there's some
application caching going on?
maybe on a clean restart (of database and lighttpd), try GET first and then
HEAD?
- at Mon, 23 Jun 2008 12:47:02 +0200
Hi!
I restarted the server and also tried multiple GET and HEAD requests in a
sequence on their own. Since I'm requesting a PHP generated page that does
not send cache headers on its own, caching is also very unlikely.
I think the problem without the NOBODY option is, that CURL waits for a
repsonse body, but the server does not send any. Therefore CURL timesout
somewhen.
However, the solution above works well.
Regards, Toby
- Alexandr Egorov at Mon, 23 Jun 2008 17:54:04 +0200
I had the same problem some time ago. cURL does not "understand" HEAD
request and thats why he waits for the real body (he receives Content-length
header and waits for the data). Actually I don't know what is NOBODY =) But
I solved the problem sending "Connection: close" header in my HEAD requests.
Seems to work fine.
- Shahar Evron at Mon, 23 Jun 2008 20:31:09 +0200
With Zend_Http_Client you can do it so:
request('HEAD'); ?>
It does understand the HEAD request so it will not try to read the body.
Probably won't time as fast as cURL though, as it's implemented in PHP...
- Toby at Mon, 23 Jun 2008 21:03:54 +0200
Hi!
Sorry, your code seems to have been stripped your code example. If possible,
please post it again without any markup (including the PHP tags).
Thanks! Toby
- Shahar Evron at Wed, 25 Jun 2008 13:37:15 +0200
Hi,
Yep - I enclosed my code in php tags.. here it is without it:
require_once 'Zend/Http/Client.php'; $client = new
Zend_Http_Client('http://localhost/admin/'); $response =
$client->request('HEAD');
- Toby at Wed, 25 Jun 2008 14:02:01 +0200
Hi Shahar!
Thanks for posting the code again. :)
Looks easy. Does Zend_HTTP_Client abstract different methods of sending the
request or does is just use a manual socket?
Regards, Toby
- Shahar Evron at Wed, 25 Jun 2008 14:28:26 +0200
Zend_Http_Client has adapters architecture - so theoretically, one could
write different adapters that use different methods (cURL, wrappers,
pecl_http etc.) to do the actual connection.
In practice, right now we only have a socket based adapter and a proxy
adapter (which extends the socket one). a cURL based wrapper was written
but was never finalized.
- M. van der Klip at Fri, 27 Jun 2008 16:52:48 +0200
Did you have a look at the manpage for curl? The definition of the '-I'
option is not to display headers only, but in fact to perform a HEAD
request. Exactly what you want and no need at all to specify a custom
request method.
The reason the custom request method took so long is probably curl waiting
for the body to arrive. Eventually the server closes the (keepalive)
connection and curl displays the result.
- Raj at Mon, 03 Aug 2009 07:25:13 +0200
hey where can i find php curl tutorials
- Tom at Mon, 14 Dec 2009 23:58:30 +0100
Thank you for the hints using curl. But this hint in the comments make my
script work:
<