schlitt.info - php, photography and private stuff
Trace this
- Revision 1
- by Tobias Schlitt
- Wed, 19 Nov 2008
- 23:29:46 +0100
Hire me
Surprise me
Follow this
Read this
Get more
Praise these
Respect this
Conforms to
F**k spammers
Sending HEAD requests with ext/curl
ext/curl is the common tool of choice, if one needs to perform more advanced HTTP requests from a PHP script (for simple ones, use a stream!). I recently wanted to perform a HEAD request to a file, after which I wanted to perform some more advanced HTTP interaction, so CURL was also the tool of choice here.
Trying it out on the shell with a local web server, CURL was operating quite slow, in contrast to a GET request. The -i command line switch makes curl include the headers in the printed output, -X lets you define a custom HTTP request.
dotxp@tango ~ $ time curl -i -X HEAD http://localhost/admin/
HTTP/1.1 200 OK
X-Powered-By: PHP/5.2.7-dev
<snip type="more http headers" />
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:10:59 GMT
Server: lighttpd/1.4.19
real 0m6.079s
user 0m0.004s
sys 0m0.000sdotxp@tango ~ $ time curl -i -X GET http://localhost/admin/
HTTP/1.1 200 OK
Transfer-Encoding: chunked
X-Powered-By: PHP/5.2.7-dev
<snip type="more http headers" />
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:12:27 GMT
Server: lighttpd/1.4.19
<snip content="html source" />
real 0m0.180s
user 0m0.004s
sys 0m0.000sA difference of 6 seconds runtime of a HEAD in contrast to 0.2 seconds for a GET is quite contrary to the original idea of a HEAD request. HEAD is used to just receive the headers of an URI instead of receiving the whole contents, to save bandwidth, memory and execution time.
ext/curl showed the exact same problem. Fiddling a bit with the command line switches, I found to replace -i with -I which makes curl print only the headers, but not the body of the response.
dotxp@tango ~ $ time curl -I -X HEAD http://localhost/admin/
HTTP/1.1 200 OK
X-Powered-By: PHP/5.2.7-dev
<snip type="more http headers" />
Content-Type: text/html; charset=utf-8
Date: Mon, 23 Jun 2008 09:19:05 GMT
Server: lighttpd/1.4.19
real 0m0.044s
user 0m0.004s
sys 0m0.000s0.04 seconds is now even faster than the corresponding GET request, with the -I switch, which took me 0.09 seconds. Now I just needed to transfer the command line options to the corresponding ext/curl ones:
$c = curl_init();
curl_setopt( $c, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $c, CURLOPT_CUSTOMREQUEST, 'HEAD' );
curl_setopt( $c, CURLOPT_HEADER, 1 );
curl_setopt( $c, CURLOPT_NOBODY, true );
curl_setopt( $c, CURLOPT_URL, 'http://localhost/admin/' );
$res = curl_exec( $c );The RETURNTRANSFER makes ext/curl return the HTTP response instead of printing it. Using the CUSTOMREQUEST option you define to send a HEAD request instead of a standard GET or POST request. The HEADER option makes ext/curl include the response headers in the return value of curl_exec() call and NOBODY avoids the inclusion of the body content here. The URL option as usually sets the URL to request and curl_exec() makes ext/curl execute the request.
The runtime was even a fraction of a second faster here, compared to the command line version, but that can be subjectively. However, the HEAD request works as expected now. Maybe it's useful for someone to know this.
If you liked this blog post or learned something, please consider using flattr to contribute back: .
Trackbacks
Comments
Fields with bold names are mandatory.
adnan.
if the curl statements were run in succession, is it not possible that server caching caused the remaining calls to be faster?
Link to commentmaybe queries are getting cached at the mysql level? or there's some application caching going on?
maybe on a clean restart (of database and lighttpd), try GET first and then HEAD?
Anonymous
Hi!
Link to commentI restarted the server and also tried multiple GET and HEAD requests in a sequence on their own. Since I'm requesting a PHP generated page that does not send cache headers on its own, caching is also very unlikely.
I think the problem without the NOBODY option is, that CURL waits for a repsonse body, but the server does not send any. Therefore CURL timesout somewhen.
However, the solution above works well.
Regards,
Toby
Alexandr Egorov
I had the same problem some time ago. cURL does not "understand" HEAD request and thats why he waits for the real body (he receives Content-length header and waits for the data).
Link to commentActually I don't know what is NOBODY =) But I solved the problem sending "Connection: close" header in my HEAD requests. Seems to work fine.
Shahar Evron
With Zend_Http_Client you can do it so:
Link to comment<?php
require_once 'Zend/Http/Client.php';
$client = new Zend_Http_Client('http://localhost/admin');
$response = $client->request('HEAD');
?>
It does understand the HEAD request so it will not try to read the body.
Probably won't time as fast as cURL though, as it's implemented in PHP...
Toby
Hi!
Link to commentSorry, your code seems to have been stripped your code example. If possible, please post it again without any markup (including the PHP tags).
Thanks!
Toby
Shahar Evron
Hi,
Link to commentYep - I enclosed my code in php tags.. here it is without it:
require_once 'Zend/Http/Client.php';
$client = new Zend_Http_Client('http://localhost/admin/');
$response = $client->request('HEAD');
Toby
Hi Shahar!
Link to commentThanks for posting the code again. :)
Looks easy. Does Zend_HTTP_Client abstract different methods of sending the request or does is just use a manual socket?
Regards,
Toby
Shahar Evron
Zend_Http_Client has adapters architecture - so theoretically, one could write different adapters that use different methods (cURL, wrappers, pecl_http etc.) to do the actual connection.
Link to commentIn practice, right now we only have a socket based adapter and a proxy adapter (which extends the socket one). a cURL based wrapper was written but was never finalized.
M. van der Klip
Did you have a look at the manpage for curl? The definition of the '-I' option is not to display headers only, but in fact to perform a HEAD request. Exactly what you want and no need at all to specify a custom request method.
Link to commentThe reason the custom request method took so long is probably curl waiting for the body to arrive. Eventually the server closes the (keepalive) connection and curl displays the result.
Raj
hey where can i find php curl tutorials
Link to commentTom
Thank you for the hints using curl. But this hint in the comments make my script work:
Link to comment<<But I solved the problem sending "Connection: close" <<header in my HEAD requests. Seems to work fine.
BeccasGraceBlog.com
All in all I really enjoyed digging into the parts I already read and I expect the rest to be as well as those. While some books only touch the theory of design patterns and avoid giving practical examples to the user, Stephan manages very good to combine both
Link to commentComposedBlog.com
XML and its related technologies are ubiquitous in todays web development. PHP offers many ways to create and process XML content. This workshop will give you an overview on the most important XML extensions for PHP, focusing on the use of XPath in cooperation with them. Do you still scrape web content using regular expressions? Ever wondered you people do all those nifty operations in their XSLTs? Don't know, what axis are in terms of XPath? If you can answer any of the questions ab
Link to commentGreen Earth Green Home Blog
This post is helpful with an analysis I am doing for a specific group of people. Do you have any other articles to suggest on this topic? Thanks
Link to commentgolden gate
Amazing cheers for the info. A fabolous piece of an article.
Link to comment