Sending multiple non-blocking cURL requests with PHP.

In this tutorial, I will show you how to send multiple non-blocking / asynchronous cURL requests using PHP. Furthermore, I will also show you how to get the output and HTTP response code for each request.

curl_multi_init.

They key to all of this is a PHP function called curl_multi_init. The curl_multi_init function allows us to initialize a cURL multi handle and send asynchronous HTTP requests.

Essentially, we create a big “multi handle” that consists of multiple smaller handles. You can think of the “multi handle” as a machine that stores and executes multiple requests.

Let’s take a look at the following code:

//The URLs that we want to send cURL requests to.
$urls = array(
    'http://test.com',
    'http://php.net',
    'http://wikipedia.org',
);

//An array that will contain all of the information
//relating to each request.
$requests = array();


//Initiate a multiple cURL handle
$mh = curl_multi_init();

//Loop through each URL.
foreach($urls as $k => $url){
    $requests[$k] = array();
    $requests[$k]['url'] = $url;
    //Create a normal cURL handle for this particular request.
    $requests[$k]['curl_handle'] = curl_init($url);
    //Configure the options for this request.
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_RETURNTRANSFER, true);
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_TIMEOUT, 10);
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_CONNECTTIMEOUT, 10);
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_SSL_VERIFYHOST, false);
    curl_setopt($requests[$k]['curl_handle'], CURLOPT_SSL_VERIFYPEER, false);
    //Add our normal / single cURL handle to the cURL multi handle.
    curl_multi_add_handle($mh, $requests[$k]['curl_handle']);
}

//Execute our requests using curl_multi_exec.
$stillRunning = false;
do {
    curl_multi_exec($mh, $stillRunning);
} while ($stillRunning);

//Loop through the requests that we executed.
foreach($requests as $k => $request){
    //Remove the handle from the multi handle.
    curl_multi_remove_handle($mh, $request['curl_handle']);
    //Get the response content and the HTTP status code.
    $requests[$k]['content'] = curl_multi_getcontent($request['curl_handle']);
    $requests[$k]['http_code'] = curl_getinfo($request['curl_handle'], CURLINFO_HTTP_CODE);
    //Close the handle.
    curl_close($requests[$k]['curl_handle']);
}
//Close the multi handle.
curl_multi_close($mh);

//var_dump the $requests array for example purposes.
var_dump($requests);

In the code above, we:

  1. Created an array of URLs that we want to send a HTTP request to.
  2. Setup an array called $requests. This array will be used to store response content, etc, so that we can access it later.
  3. Looped through each URL in the $url array and configured each request before adding it to our multi handle.
  4. Executed the multi handle using the curl_multi_exec function. Note that this function will set the variable $stillRunning to a boolean FALSE value once it has finished.
  5. Looped through each executed request and compiled the response content and HTTP status code into our $requests array for future use.
  6. We closed the multi cURL handle.
  7. We var_dumped out the $requests array.

If you run the above code, you should get a good idea of how this piece of code works. You should also notice how much faster a multi handle cURL request is than sending off multiple individual requests.

Note that you may have to change some of the options that we set in the first loop to match your own requirements.

At the end, you should be left with an array that contains the URL, the response content and the HTTP status code for each request that you executed.

Hopefully, this helped!