schlitt.info - php, photography and private stuff
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:Author: 		Tobias Schlitt
:Date:			Wed, 19 Nov 2008 23:29:46 +0100
:Revision: 		1
:Copyright:		CC by-nc-sa


===================================
Sending HEAD requests with ext/curl
===================================

:Description:
    ext/curl is the common tool of choice, if one needs to perform more
    advanced HTTP requests from a PHP script (for simple ones, use a stream!).
    I recently wanted to perform a HEAD request to a file, after which I
    wanted to perform some more advanced HTTP interaction, so CURL was also
    the tool of choice here.

`ext/curl`__ is the common tool of choice, if one needs to perform more
advanced HTTP requests from a PHP script (for simple ones, use a `stream`__!).
I recently wanted to perform a HEAD request to a file, after which I wanted to
perform some more advanced HTTP interaction, so `CURL`__ was also the tool of
choice here.

.. __: http://php.net/curl
.. __: http://php.net/stream
.. __: http://curl.haxx.se/

Trying it out on the shell with a local web server, CURL was operating quite
slow, in contrast to a GET request. The *-i* command line switch makes curl
include the headers in the printed output, *-X* lets you define a custom HTTP
request.

::

    
    dotxp@tango ~ $ time curl -i -X HEAD http://localhost/admin/
    HTTP/1.1 200 OK
    X-Powered-By: PHP/5.2.7-dev
    
    <snip type="more http headers" />
    
    Content-Type: text/html; charset=utf-8
    Date: Mon, 23 Jun 2008 09:10:59 GMT
    Server: lighttpd/1.4.19
    
    real    0m6.079s
    user    0m0.004s
    sys    0m0.000s

::

    
    dotxp@tango ~ $ time curl -i -X GET http://localhost/admin/
    HTTP/1.1 200 OK
    Transfer-Encoding: chunked
    X-Powered-By: PHP/5.2.7-dev
    
    <snip type="more http headers" />
    
    Content-Type: text/html; charset=utf-8
    Date: Mon, 23 Jun 2008 09:12:27 GMT
    Server: lighttpd/1.4.19
    
    <snip content="html source" />
    
    real    0m0.180s
    user    0m0.004s
    sys    0m0.000s

A difference of 6 seconds runtime of a HEAD in contrast to 0.2 seconds for a
GET is quite contrary to the original idea of a HEAD request. HEAD is used to
just receive the headers of an URI instead of receiving the whole contents, to
save bandwidth, memory and execution time.

ext/curl showed the exact same problem. Fiddling a bit with the command line
switches, I found to replace *-i* with *-I* which makes curl print only the
headers, but not the body of the response.

::

    
    dotxp@tango ~ $ time curl -I -X HEAD http://localhost/admin/
    HTTP/1.1 200 OK
    X-Powered-By: PHP/5.2.7-dev
    
    <snip type="more http headers" />
    
    Content-Type: text/html; charset=utf-8
    Date: Mon, 23 Jun 2008 09:19:05 GMT
    Server: lighttpd/1.4.19
    
    real    0m0.044s
    user    0m0.004s
    sys    0m0.000s

0.04 seconds is now even faster than the corresponding GET request, with the
-I switch, which took me 0.09 seconds. Now I just needed to transfer the
command line options to the corresponding ext/curl ones:

::

    
    $c = curl_init();
    curl_setopt( $c, CURLOPT_RETURNTRANSFER, true ); 
    curl_setopt( $c, CURLOPT_CUSTOMREQUEST, 'HEAD' ); 
    curl_setopt( $c, CURLOPT_HEADER, 1 ); 
    curl_setopt( $c, CURLOPT_NOBODY, true ); 
    curl_setopt( $c, CURLOPT_URL, 'http://localhost/admin/' ); 
    
    $res = curl_exec( $c ); 

The *RETURNTRANSFER* makes ext/curl return the HTTP response instead of
printing it. Using the *CUSTOMREQUEST* option you define to send a HEAD
request instead of a standard GET or POST request. The *HEADER* option makes
ext/curl include the response headers in the return value of *curl_exec()*
call and NOBODY avoids the inclusion of the body content here. The *URL*
option as usually sets the URL to request and *curl_exec()* makes ext/curl
execute the request.

The runtime was even a fraction of a second faster here, compared to the
command line version, but that can be subjectively. However, the HEAD request
works as expected now. Maybe it's useful for someone to know this.


..
   Local Variables:
   mode: rst
   fill-column: 79
   End: 
   vim: et syn=rst tw=79

Trackbacks
==========


Comments
========

- adnan. at Mon, 23 Jun 2008 12:43:13 +0200

  if the curl statements were run in succession, is it not possible that
  server caching caused the remaining calls to be faster?

  maybe queries are getting cached at the mysql level? or there's some
  application caching going on?

  maybe on a clean restart (of database and lighttpd), try GET first and then
  HEAD?

-  at Mon, 23 Jun 2008 12:47:02 +0200

  Hi!

  I restarted the server and also tried multiple GET and HEAD requests in a
  sequence on their own. Since I'm requesting a PHP generated page that does
  not send cache headers on its own, caching is also very unlikely.

  I think the problem without the NOBODY option is, that CURL waits for a
  repsonse body, but the server does not send any. Therefore CURL timesout
  somewhen.

  However, the solution above works well.

  Regards, Toby

- Alexandr Egorov at Mon, 23 Jun 2008 17:54:04 +0200

  I had the same problem some time ago. cURL does not "understand" HEAD
  request and thats why he waits for the real body (he receives Content-length
  header and waits for the data). Actually I don't know what is NOBODY =) But
  I solved the problem sending "Connection: close" header in my HEAD requests.
  Seems to work fine.

- Shahar Evron at Mon, 23 Jun 2008 20:31:09 +0200

  With Zend_Http_Client you can do it so:

  <?php require_once 'Zend/Http/Client.php'; $client = new
  Zend_Http_Client('http://localhost/admin'); $response =
  $client->request('HEAD'); ?>

  It does understand the HEAD request so it will not try to read the body. 

  Probably won't time as fast as cURL though, as it's implemented in PHP...

- Toby at Mon, 23 Jun 2008 21:03:54 +0200

  Hi!

  Sorry, your code seems to have been stripped your code example. If possible,
  please post it again without any markup (including the PHP tags).

  Thanks! Toby

- Shahar Evron at Wed, 25 Jun 2008 13:37:15 +0200

  Hi,

  Yep - I enclosed my code in php tags.. here it is without it:

  require_once 'Zend/Http/Client.php'; $client = new
  Zend_Http_Client('http://localhost/admin/'); $response =
  $client->request('HEAD');

- Toby at Wed, 25 Jun 2008 14:02:01 +0200

  Hi Shahar!

  Thanks for posting the code again. :)

  Looks easy. Does Zend_HTTP_Client abstract different methods of sending the
  request or does is just use a manual socket?

  Regards, Toby

- Shahar Evron at Wed, 25 Jun 2008 14:28:26 +0200

  Zend_Http_Client has adapters architecture - so theoretically, one could
  write different adapters that use different methods (cURL, wrappers,
  pecl_http etc.) to do the actual connection.

  In practice, right now we only have a socket based adapter and a proxy
  adapter (which extends the socket one). a cURL  based wrapper was written
  but was never finalized.

- M. van der Klip at Fri, 27 Jun 2008 16:52:48 +0200

  Did you have a look at the manpage for curl? The definition of the '-I'
  option is not to display headers only, but in fact to perform a HEAD
  request. Exactly what you want and no need at all to specify a custom
  request method.

  The reason the custom request method took so long is probably curl waiting
  for the body to arrive. Eventually the server closes the (keepalive)
  connection and curl displays the result.

- Raj at Mon, 03 Aug 2009 07:25:13 +0200

  hey where can i find php curl tutorials

- Tom  at Mon, 14 Dec 2009 23:58:30 +0100

  Thank you for the hints using curl. But this hint in the comments make my
  script work:

  <<But I solved the problem sending "Connection: close" <<header in my HEAD
  requests. Seems to work fine.

- Solar Panels at Wed, 19 Dec 2012 17:43:06 +0100

  This is also a very good post which I really enjoyed reading. It is not
  everyday that I have the possibility to see something like this..

- tapetes para recibo at Sun, 06 Jan 2013 18:43:57 +0100

  I'm no expert, but I think you just made the best point. You obviously know
  a lot about what you're talking about, and I can really get behind that.
  Thanks for being so upfront and so honest about the subject matter. I really
  feel like I have a better understanding now.

- tapetes para recepción at Tue, 08 Jan 2013 05:48:48 +0100

  This really is such a great post, and was thinking considerably exactly the
  same myself. An additional wonderful update.

- Tinnitus Treatment at Fri, 01 Feb 2013 11:20:33 +0100

  very great post i love php and implemented it with wordpress

- makr at Fri, 01 Feb 2013 11:27:30 +0100

  makr

- tapetes con logo at Tue, 19 Mar 2013 14:23:10 +0100

  It is a very good read for me. I have to admit that you are one of the best
  bloggers I read. Thanks for posting this informative article.

- tapetes para gimnasio at Tue, 19 Mar 2013 14:23:36 +0100

  You have an impressive great converter here. Showing this alphabet with
  their other values is very nice idea.

- Consignment Sales Atlanta at Fri, 05 Apr 2013 08:32:26 +0200

  I should tell you that your blog articles and other content is extremely
  great. It truly is not simple to retain this kind of top quality in the
  webpage.

- kswlewe at Mon, 15 Apr 2013 05:14:46 +0200

  more officials to learn from him, and let every official assets billion,
  then again good work to help poor people, only in this way, China can be
  more brilliant and splendid tomorrow. http://guccibeltsuksale.blog.co.uk/

- dslrcamerasguide at Thu, 16 May 2013 12:54:31 +0200

  if you are a beginner and are learning photography, you should opt for
  Nikon's D3100, which actually shows you how changing the settings can affect
  your photograph.

- askcreditsolutions at Thu, 16 May 2013 12:54:49 +0200

  Unlike most investments, annuity purchase is a once-and-forever decision;
  you cannot change your mind after the cooling-off period has expired. So it
  is vital to get it right first time.

- virtualqwertykeyboard at Thu, 16 May 2013 12:55:00 +0200

  The Blackberry Torch 9800 is perhaps one of the most appealing recent
  handsets from the manufacturer. Featuring a touch screen and also a slide
  out QWERTY keyboard, it offers the very best of both.

- music dvd at Thu, 16 May 2013 12:55:11 +0200

  So start your quest to amazingly creative music teachers resources now. Be
  inspired to share your knowledge as well as the joys of music to your
  students.

- interioir design concepts at Thu, 16 May 2013 12:55:25 +0200

  Interior design elements important to successful decorating projects include
  light, color, materials, pattern, texture, proportion and scale