Cover photo for post Sending HEAD requests with ext/curl

Sending HEAD requests with ext/curl

ext/curl is the common tool of choice, if one needs to perform more advanced HTTP requests from a PHP script (for simple ones, use a stream!). I recently wanted to perform a HEAD request to a file, after which I wanted to perform some more advanced HTTP interaction, so CURL was also the tool of choice here.

Trying it out on the shell with a local web server, CURL was operating quite slow, in contrast to a GET request. The -i command line switch makes curl include the headers in the printed output, -X lets you define a custom HTTP request.

dotxp@tango ~ $ time curl -i -X HEAD http://localhost/admin/ HTTP/1.1 200 OK X-Powered-By: PHP/5.2.7-dev <snip type="more http headers" /> Content-Type: text/html; charset=utf-8 Date: Mon, 23 Jun 2008 09:10:59 GMT Server: lighttpd/1.4.19 real 0m6.079s user 0m0.004s sys 0m0.000sdotxp@tango ~ $ time curl -i -X GET http://localhost/admin/ HTTP/1.1 200 OK Transfer-Encoding: chunked X-Powered-By: PHP/5.2.7-dev <snip type="more http headers" /> Content-Type: text/html; charset=utf-8 Date: Mon, 23 Jun 2008 09:12:27 GMT Server: lighttpd/1.4.19 <snip content="html source" /> real 0m0.180s user 0m0.004s sys 0m0.000s

A difference of 6 seconds runtime of a HEAD in contrast to 0.2 seconds for a GET is quite contrary to the original idea of a HEAD request. HEAD is used to just receive the headers of an URI instead of receiving the whole contents, to save bandwidth, memory and execution time.

ext/curl showed the exact same problem. Fiddling a bit with the command line switches, I found to replace -i with -I which makes curl print only the headers, but not the body of the response.

dotxp@tango ~ $ time curl -I -X HEAD http://localhost/admin/ HTTP/1.1 200 OK X-Powered-By: PHP/5.2.7-dev <snip type="more http headers" /> Content-Type: text/html; charset=utf-8 Date: Mon, 23 Jun 2008 09:19:05 GMT Server: lighttpd/1.4.19 real 0m0.044s user 0m0.004s sys 0m0.000s

0.04 seconds is now even faster than the corresponding GET request, with the -I switch, which took me 0.09 seconds. Now I just needed to transfer the command line options to the corresponding ext/curl ones:

$c = curl_init(); curl_setopt( $c, CURLOPT_RETURNTRANSFER, true ); curl_setopt( $c, CURLOPT_CUSTOMREQUEST, 'HEAD' ); curl_setopt( $c, CURLOPT_HEADER, 1 ); curl_setopt( $c, CURLOPT_NOBODY, true ); curl_setopt( $c, CURLOPT_URL, 'http://localhost/admin/' ); $res = curl_exec( $c );

The RETURNTRANSFER makes ext/curl return the HTTP response instead of printing it. Using the CUSTOMREQUEST option you define to send a HEAD request instead of a standard GET or POST request. The HEADER option makes ext/curl include the response headers in the return value of curl_exec() call and NOBODY avoids the inclusion of the body content here. The URL option as usually sets the URL to request and curl_exec() makes ext/curl execute the request.

The runtime was even a fraction of a second faster here, compared to the command line version, but that can be subjectively. However, the HEAD request works as expected now. Maybe it's useful for someone to know this.

Comments

if the curl statements were run in succession, is it not possible that server caching caused the remaining calls to be faster?

maybe queries are getting cached at the mysql level? or there's some application caching going on?

maybe on a clean restart (of database and lighttpd), try GET first and then HEAD?

adnan. at 2008-06-23

Hi!

I restarted the server and also tried multiple GET and HEAD requests in a sequence on their own. Since I'm requesting a PHP generated page that does not send cache headers on its own, caching is also very unlikely.

I think the problem without the NOBODY option is, that CURL waits for a repsonse body, but the server does not send any. Therefore CURL timesout somewhen.

However, the solution above works well.

Regards, Toby

at 2008-06-23

I had the same problem some time ago. cURL does not "understand" HEAD request and thats why he waits for the real body (he receives Content-length header and waits for the data). Actually I don't know what is NOBODY =) But I solved the problem sending "Connection: close" header in my HEAD requests. Seems to work fine.

Alexandr Egorov at 2008-06-23

With Zend_Http_Client you can do it so:

<?php require_once 'Zend/Http/Client.php'; $client = new Zend_Http_Client('http://localhost/admin'); $response = $client->request('HEAD'); ?>

It does understand the HEAD request so it will not try to read the body.

Probably won't time as fast as cURL though, as it's implemented in PHP...

Shahar Evron at 2008-06-23

Hi!

Sorry, your code seems to have been stripped your code example. If possible, please post it again without any markup (including the PHP tags).

Thanks! Toby

Toby at 2008-06-23

Hi,

Yep - I enclosed my code in php tags.. here it is without it:

require_once 'Zend/Http/Client.php'; $client = new Zend_Http_Client('http://localhost/admin/'); $response = $client->request('HEAD');

Shahar Evron at 2008-06-25

Hi Shahar!

Thanks for posting the code again. :)

Looks easy. Does Zend_HTTP_Client abstract different methods of sending the request or does is just use a manual socket?

Regards, Toby

Toby at 2008-06-25

Zend_Http_Client has adapters architecture - so theoretically, one could write different adapters that use different methods (cURL, wrappers, pecl_http etc.) to do the actual connection.

In practice, right now we only have a socket based adapter and a proxy adapter (which extends the socket one). a cURL based wrapper was written but was never finalized.

Shahar Evron at 2008-06-25

Did you have a look at the manpage for curl? The definition of the '-I' option is not to display headers only, but in fact to perform a HEAD request. Exactly what you want and no need at all to specify a custom request method.

The reason the custom request method took so long is probably curl waiting for the body to arrive. Eventually the server closes the (keepalive) connection and curl displays the result.

M. van der Klip at 2008-06-27

hey where can i find php curl tutorials

Raj at 2009-08-03