Category Archives: Curl

How to get Locations in a 50 Mile Radius with Google Maps API v.3

In this post, I will demonstrate how to make getting locations within a given radius simple. In order to make everyone understand, I will break it down into smaller, easy to read steps.

The Distance-Matrix

The key to creating a solution that gets all locations within a given distance radius is to use the Google Maps API Distance-Matrix. Let's examine a simple example request to the Distance-Matrix Json API:

Single Destination URL Example:

https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=44311&destinations=45735&key=Your_API_Key

copy and paste the above URL into a web browser and replace the text, Your_API_Key, with your google Maps V.3 API key. If you don't have an API key yet, they are surprisingly easy to obtain from https://developers.google.com/maps/web-services/

The Response:

The URL above will return a Json response of:

{
   "destination_addresses" : [ "Guysville, OH 45735, USA" ],
   "origin_addresses" : [ "Akron, OH 44311, USA" ],
   "rows" : [
      {
         "elements" : [
            {
               "distance" : {
                  "text" : "166 mi",
                  "value" : 267653
               },
               "duration" : {
                  "text" : "2 hours 30 mins",
                  "value" : 8989
               },
               "status" : "OK"
            }
         ]
      }
   ],
   "status" : "OK"
}

Notice that I only used zip codes for orgin and destination in my example URL. You can use any acceptable address format instead of just a zip code of you like, but I find that for getting distances, zip codes work quite efficiently. Examples of acceptable addresses include:

  • Cleveland, OH
  • 593 Brown Street, Akron, Ohio
  • 44319
  • Akron, Ohio 4319
  • 593 Brown Street, Akron, Ohio 44311
  • USA
  • Amsterdam

Those are just a few acceptable addresses that could be used for either the destination or orgin parameters in the URL used for an API request to the Distance-Matrix.

 

Multiple Destination URL Example:

One thing that's important to know about the Distance-Matrix API is that there are limits on it's free usage. At the time of writing this post, the limits were 2500 elements per day. A single request may take up several elements however, so be careful. However it is much more practical to include several destination addresses in a single request because it will make your script run significantly quicker. Here is a simple example that will return a response with two destination distances using just zip codes for addresses again:

https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=44311&destinations=45735|44319&key=Your_API_Key

This the json response will include the distances for both zip codes or addresses passed in the URL. Here is the response from the above URL with two destinations:

{
   "destination_addresses" : [ "Guysville, OH 45735, USA", "Akron, OH 44319, USA" ],
   "origin_addresses" : [ "Akron, OH 44311, USA" ],
   "rows" : [
      {
         "elements" : [
            {
               "distance" : {
                  "text" : "166 mi",
                  "value" : 267653
               },
               "duration" : {
                  "text" : "2 hours 30 mins",
                  "value" : 8989
               },
               "status" : "OK"
            },
            {
               "distance" : {
                  "text" : "8.9 mi",
                  "value" : 14385
               },
               "duration" : {
                  "text" : "14 mins",
                  "value" : 836
               },
               "status" : "OK"
            }
         ]
      }
   ],
   "status" : "OK"
}

Parsing Json data with PHP

The next task is to fetch a response from PHP and parse the Json data in your PHP code. Lets look at a practical example using our first URL example above that has a single zip code as the destination address and also a single zip code for the orgin. Here is the PHP code to make the request, get a Json response and parse the distance from that response:

<?php
$url = "https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=44311&destinations=45735&key=Your-API-Key";

    //fetch json response from googleapis.com:
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $response = json_decode(curl_exec($ch), true);
    //If google responds with a status of OK
    //Extract the distance text:
    if($response['status'] == "OK"){
        $dist = $response['rows'][0]['elements'][0]['distance']['text'];
        echo "<p>Dist: $dist</p>";
    }
?>

Now lets have a look at how we would modify the above code for a request with two destination addresses instead of one:

<?php
$url = "https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=44311&destinations=45735|44319&key=Your-API-Key";

    //fetch json response from googleapis.com:
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $response = json_decode(curl_exec($ch), true);
    //If google responds with a status of OK
    //Extract the distance text:
    if($response['status'] == "OK"){
        $dist = $response['rows'][0]['elements'][0]['distance']['text'];
        echo "<p>Dist: $dist</p>";
        $dist2 = $response['rows'][0]['elements'][1]['distance']['text'];
        echo "<p>Dist2: $dist2</p>";
    }
?>

If you insert your own API key in the above code and run it from your own server, the response would look like this:

Dist: 166 mi

Dist2: 8.9 mi

Notice the difference in the $dist and $dist2 variable values. The key of the elements is different. The first distance is stored in elements[0] while the second is stored in elements[1]. If we had three destinations in the request, the third would be in elements[2] and so on...

How to figure out the values

So, what if you want more than just distance from the returned json element? Lets examine the returned data and figure out how to get it into PHP variables. In the example code above, try placeing this code right after the line that reads "if($response['status'] == "OK"){" and it will show you what the PHP array looks like after the Json response was converted to a PHP object:

        echo "<pre>";
        print_r($response);
        echo "</pre>";

The above code inserted into the last code example would look like:

<?php
$url = "https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=44311&destinations=45735|44319&key=Your-API-Key";

    //fetch json response from googleapis.com:
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $response = json_decode(curl_exec($ch), true);
    //If google responds with a status of OK
    //Extract the distance text:
    if($response['status'] == "OK"){
        echo "<pre>";
        print_r($response);
        echo "</pre>";
        $dist = $response['rows'][0]['elements'][0]['distance']['text'];
        echo "<p>Dist: $dist</p>";
        $dist2 = $response['rows'][0]['elements'][1]['distance']['text'];
        echo "<p>Dist2: $dist2</p>";
    }
?>

Again, replace "Your-API-Key" in the URL variable with your own key and run the code. You should see results like this:

Array
(
    [destination_addresses] => Array
        (
            [0] => Guysville, OH 45735, USA
            [1] => Akron, OH 44319, USA
        )

    [origin_addresses] => Array
        (
            [0] => Akron, OH 44311, USA
        )

    [rows] => Array
        (
            [0] => Array
                (
                    [elements] => Array
                        (
                            [0] => Array
                                (
                                    [distance] => Array
                                        (
                                            [text] => 166 mi
                                            [value] => 267653
                                        )

                                    [duration] => Array
                                        (
                                            [text] => 2 hours 30 mins
                                            [value] => 8989
                                        )

                                    [status] => OK
                                )

                            [1] => Array
                                (
                                    [distance] => Array
                                        (
                                            [text] => 8.9 mi
                                            [value] => 14385
                                        )

                                    [duration] => Array
                                        (
                                            [text] => 14 mins
                                            [value] => 836
                                        )

                                    [status] => OK
                                )

                        )

                )

        )

    [status] => OK
)

Dist: 166 mi

Dist2: 8.9 mi

First you see the array printed out from the response after it's converted from Json to a PHP array, then you see the two distances printed to the screen in the last two lines. Examine the array closely and you can figure out the correct keys to use to pick out specific values from the array in your PHP code.

More info on the Distance-Matrix can be found at https://developers.google.com/maps/documentation/distance-matrix/start

You Can Now Contribute Web Development Content

Jafty Interactive Web Development has decided to start accepting user contributed blog posts! You can now submit any post that is related to out blog theme of web development and it will be published after admin moderation. As long as it is on topic and I think it improves the quality of my blog, it will probably be published. Some topics I would like to encourage users to submit posts about include:

  • PHP
  • JavaScript
  • HTML
  • XML
  • CSS
  • Search Engine Optimization/SEO
  • Freelancing as a web developer
  • Freelance writing
  • working from home
  • Laravel
  • WordPress
  • C Languages
  • Java
  • ASP.net
  • JQUERY
  • Ruby on rails
  • MySQL Database
  • SQL
  • PHP and MySQL
  • Remote server administration
  • Linux Command Line
  • Linux and Windows Servers
  • MAC and web development
  • PC vs MAC
  • Mobile Development
  • Mobile website optimization
  • PERL
  • AJAX
  • Adwords
  • CURL
  • Graphics
  • anything else related to web development!

That's just a list to fuel your mind. Feel free to submit any related topic. I will accept anything related to web development. I'm looking forward to some great blog posts! - Ian L.

Click Here to Sign Up and Begin Writing!

 

Using XMLRPC to get a List of Posts from WordPress

Today I'm going to demonstrate how one can go about remotely fetching a list of posts for any post type including custom post types remotely using XML-RPC in WordPress. What I thought was going to be a simple task ended up taking me the majority of an entire day to work out yesterday, so I am posting my solution here for future reference and to help others solve the same problem.

So during my quest to figure out a solution for listing posts for a remote WordPress blog inside the wp-admin of another WordPress blog, I came across a couple promising functions such as wp.getPosts, blogger_getRecentPosts and similar existing XML-RPC functions that reside within wp-includes/class-IXR.php. None of those did the trick however, so below is the final solution I came up with which involves extending the XML-RPC class noted.

Extending the XML-RPC functions in WordPress

Here's what you can add to your theme's functions.php or plugin file to extend the class-IXR.php class to allow remote listing of post titles for any given registered post type:

function wp_getTits() {
// Get the 'Profiles' post type
/*$args = array(
    'post_type' => 'oils-blends',
);*/
$args = array(
'nopaging' => true,
'numberposts'    => 300,//increase if more than 300 products exist!
'post_type'        => 'oils-blends',
'post_status'   => 'publish'//only exporting published products!
);
$loop = new WP_Query($args);
$tits = "";
while($loop->have_posts()): $loop->the_post();

$titis = get_the_title();
$tits .= $titis."zxz";

endwhile;
wp_reset_query();
return $tits;
}

function wp_new_xmlrpc_methods( $methods ) {
    $methods['wp.getTits'] = 'wp_getTits';
    return $methods;   
}
add_filter( 'xmlrpc_methods', 'wp_new_xmlrpc_methods');

After you add the above code to functions.php or your plugin file, then you can use the following block of code within a plugin file of another WordPress site to remotely list the posts of a WordPress blog on another server. I added the following code inside my plugin which was used to post to a remote WordPress site. I needed to read the existing posts in order to know whether to write or update the posts that I was pushing to the second blog:

<?php
echo "<p>Starting to get posts from remote website...</p><h3>List of Titles:</h3>";

$rpcurl = "http://example.com.au/wordpress_dir/xmlrpc.php";

$request = xmlrpc_encode_request('wp.getTits','1');
$ch = curl_init();
curl_setopt($ch, CURLOPT_POSTFIELDS, $request);
curl_setopt($ch, CURLOPT_URL, $rpcurl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 1);
$results = curl_exec($ch);
//echo "Curl results: $results<hr />";
curl_close($ch);
//separate titles by zxz string:
$titleArray = explode("zxz",$results);
    foreach($titleArray as $curTit){
        echo "$curTit<br />";
    }//END FOREACH TITLE
echo "<p>Finished...</p>";

?>

That's all folks. Now you can remotely fetch posts from one WordPress site to another without any prior knowledge of post IDs etc. Now if you do know the post IDs, then you could use the existing XML-RPC method, wp.getPost which uses the post ID as an argument. I'll share this code with you too even though it wasn't the final solution for me, it could work for someone that happens to know the post IDs for which they need to grab remotely. The following code was tested and working as of WordPress version 4.2.3:

Listing Posts Remotely using wp.getPost

As you will see, it is a lot easier to get post data when you know the post ID. Here is all you have to add to your WordPress plugin file to remotely grab a post using wp.getPost when you know the post's ID:

<?php
echo "<p>Starting to get post data from remote site...</p>";

$rpcurl = "http://example.com.au/wordpress_dir/xmlrpc.php";
$username = 'admin';
$password = 'password';
$post_id = 2820;
include('../wp-includes/class-IXR.php');  
        $client = new IXR_Client($rpcurl);
     
        $USER = $username;
        $PASS = $password;
               
        if (!$client->query('metaWeblog.getPost',$post_id, $USER,$PASS))
        {  
            echo('Error occurred during getting post.' . $client->getErrorCode().":".$client->getErrorMessage());  
        }
        $post = $client->getResponse();
       
        var_dump($post);
echo "<hr /><p>Finished....</p>";
?>

Of course you have to edit the URL, username and password above, but after that it will work fine.

Summary

So I've given you two ways to get post data from WordPress sites remotely. If you know the post IDs, then use the last method which is simpler, otherwise you have to extend the class explained above using the first two code snippets on this page. Good luck!

CodeLobster All-in-One Coding Software

CodeLobster PHP Edition Software Review

Written by Ian Lincicome of Jafty Interactive Web Development - Jafty.com

For the past couple of weeks, I have been evaluating CodeLobster's latest release of "CodeLobster PHP Edition" which I'll refer to as simply "CodeLobster" from here on out in this review. The current edition at the time of writing this review was CodeLobster PHP Edition Professional Version 5.4. I am reviewing CodeLobster because I am looking for a tool I can use for my web development business, Jafty Interactive Web Development. Also I was given a break on purchasing the pro version of CodeLobster in exchange for writing a product review on my site, Jafty.com. Please note that I am under no obligation to write a positive review. I will be as brutally honest as ever, I assure you. This review will include both the pros and cons of using CodeLobster rather than a one-sided review like many company's probably hire writers to write. In fact, if that was my intention, I wouldn't have mentioned that the someone had asked me to write this review. My ulterior motive for writing the review is to find the perfect program to do my work for my business. I am also evaluating other products as well because I want to work with the best.

I'm looking for an application that will allow me to seamlessly edit several types of code on several different servers without having to use a separate code editor, FTP program, web browser and Language reference material. To this date, I have been using three separate applications to accomplish my day-to-day work. I used FileZilla to copy files from a client's web server to my desktop. I used Notepad++ to edit the code files. Finally, before using FileZilla again to upload the files back to the client's server, I viewed the file in Firefox to see how it looks in a browser. That is if it is not a server-side language like PHP which I use the most. If I had to edit a PHP file, I had yet another couple of steps to perform because I'd have to upload the file to the server to view it and then repeat the whole entire process again if I had more changes to make to get it perfect. So I used FileZilla, Notepad++ and Firefox mostly up until now. I figure I could probably almost double my productivity with one good tool that does it all.

Technologies CodeLobster Supports

CodeLobster supports a wide range of technologies including:

  • PHP
  • HTML
  • JavaScript
  • CSS
  • SQL
  • MySQL
  • Version control systems such as SVN, Git etc.
  • CakePHP
  • CodeIgniter
  • WordPress
  • Drupal
  • Joomla
  • Facebook
  • jQuery
  • Smarty
  • Symfony
  • Yii

CodeLobster also has a very wide assortment of features including all the standard components you would expect in a code editor and many extras. I found a helpful built-in Help feature that also links to online help if you can't find what you need in the built-in documentation. The online documentation is surprisingly complete for a newer product.

Installing CodeLobster

The installation went smoothly and was very easy and comprehensive. Simply download and click the set up file and the installer will guide you through the simple set up process. Here is the Download Link: http://www.codelobster.com/download.html

When I first installed CodeLobster, it allowed me to select what languages/technologies I wanted to install. I only deselected two that I knew for sure I would most likely never use. I figured if there was even a small chance that I might use one of the items listed, then I should leave it checked. That way I will be prepared in case I need that particular technology in the future.

Pros and Cons

Once I began using CodeLobster I noticed both good and bad points regarding the program. I loved how versatile the program was right out of the box. I did not like the way the word wrap feature worked(View/Word Wrap). It breaks up words rather than splitting lines by the nearest space. I'd prefer it to not break up words at all or at least do so in a more logical manner(by syllables with hyphens for example). From what I understand from reading the forums and talking with my contact, CodeLobster plans to add an option between soft and hard word-wrap. Hard wrap is how it is now and soft wrap would be a word-wrap technique that doesn't break up words. I do hope they add it soon because it was one of the of the first things I noticed.

One of the first features I noticed and liked was the completely configurable hot-keys. You can configure any hot-key by going to Tools/Preferences/IDE/Hot keys. Another nice feature is the Find and Replace feature that is very much like that of Notepad++ including regular expression search and the ability to search and replace text in files, folders and sub-folders. This can be a huge time-saver for coders.

One of the first things I noticed and did not like was the lack of a decent spell checker. I know notepad++ has the option as a plugin, but not as a built-in feature. Either way would be great, but I think it should be a standard built-in feature. I realize that auto-complete helps with this, but not if you are writing a plain text file such as I am doing right now while writing this review. I talked to my contact at CodeLobster Software about this and he assured me that they plan to add a robust spell-checking feature in the near future. He mentioned that they plan on making several additions to the software and will be releasing future editions with new features and improvements ASAP. So, while CodeLobster may not have everything I would like it to have at the moment, I was assured that they are heading in the right direction with future expansion plans.

Another nice feature I also noticed right away was CodeLobster's tool bars. The tool bars are completely configurable and have a nice drag and drop feature to move tool bar items around as you please. The same can be said for the man different windows that can be added or removed from the work area.

CodeLobster is a code editor first & foremost and it does a wonderful job at editing PHP code which is my code of choice. If you are looking for a word processor, CodeLobster probably won't fit the bill, but it is a very robust code editor that can handle all of the languages I use for web development and several others.

CodeLobster's plugin ability is great to have in a code editing program. The pro version of CodeLobster comes with many plugins that the free version does not include from what I understand, so I think the pro version is definitely worth the cost for those of you who work with frameworks such as WordPress, Joomla, CakePHP, CodeIgniter, etc. I currently own the Pro version and I am happy with its performance so far and will be much happier once some of the additions are made in the near future.

Even the free version would be great for coders who use many different coding languages regularly but don't need the extra plugin abilities that require the pro version. As a full-time web developer, I mostly use PHP, HTML, CSS, JavaScript, WordPress and MySQL. I also occasionally use Joomla, XML, Perl, Python, Drupal, Smarty, Laravel, CakePHP and others, so I require the Pro version of CodeLobster myself. If you only use the most common languages such as PHP, HTML, CSS and JavaScript, you can probably do fine with the free version. Some of the things I require the Pro version for include WordPress support, Smarty, Laravel and CakePHP. Honestly there are to many features to list here. You should visit their web page at http://www.codelobster.com/details_phped.html to figure out which version is best for you. They also have a Lite version which you can read about on their site since I won't be covering CodeLobster Lite here.

As I continue to evaluate the software I am finding there are more and more features that I like. As a web developer, I need a tool that allows me to quickly edit code, test it and upload it to the client's server. So far, CodeLobster seems to be meeting my three main requirements quite well in most cases.

FTP Feature

CodeLobster's FTP ability is a very powerful feature. The FTP feature allows you to set up as many FTP connections as you need. I was able to copy all of my client's FTP connections from FileZilla into CodeLobster. Now I am able to click on a connection, edit a server file, preview it and upload it back to the server seamlessly from within a single application! This is huge for me because I used to use at least three different applications to do the same thing without CodeLobster. I can see that my work will get done a lot faster by using CodeLobster. In fact, I am already getting work done faster after only two days of using it.

The FTP feature is very functional, but for a guy like me who hates to read instructions, it took a little bit of fooling around using trial and error methods to get it set up for the first time with all of my client's 20+ FTP connections and configuring basic settings. Everything I needed to accomplish my FTP tasks is in CodeLobster, but the FTP system does need some improvements. For example it gave me trouble when trying to connect to a SFTP connection. It worked great with all non-secure FTP connections however. As luck would have it, out of over 20 current clients I work with, only one uses SFTP rather than FTP, so I am unable to test it on other secure SFTP servers to see if the problem is just with this one server or not. Unfortunately this is one of my regular clients so it is going to be a problem for me. It did connect using SFTP, but had issues when trying to edit files on the server which I was able to do fine with regular FTP connections. I did bring this issue to the attention of my CodeLobster contact however and he assures me it will be looked into ASAP.

Regarding the GUI set-up for FTP, in my personal opinion, the way the windows are organized in the work area could have been done a little better. For example, when I use FTP, I have to go to Tools/FTP to open the explorer window that lists FTP connections. That alone will allow you to open your FTP connection but won't allow you to see what's going on. If there was an error and the only window you had open in your work area was the explorer window, you wouldn't see that error. This is where I think it could have been done better. I would have programmed it so that when you open the FTP window, the output window opens along with it, right above it, similar to how FileZilla allows you to view console output. Of course this is just my opinion, it may not bother others and actually I am starting to get used to it so it doesn't bother me as much as time passes.

Search Features

The "Find in Files"(Search/Find in Files...) feature is great for finding text in open files. It could be improved to allow file search on the local directory tree files as well, but it is sufficient the way it is. Also I love the regular expression search as I mentioned earlier.

Also, I often use the incremental search feature when I edit code. While I am glad it has this feature, a small nuisance for me is that the incremental search box is not persistent across different views. I think it would be nice if the incremental search box would stay at the bottom of all view tabs once it is opened. I don't think it should close unless you manually close it. Again, this is just my preference, not a bug.

Nice-to-Have Features

One more thing I liked while test driving CodeLobster is that when editing .html and .css files I could hover over class names in the HTML or CSS code and it would show me the associated style declaration. Also when hovering over items in a .css file, it shows me information regarding the browser compatibility of the CSS code.

The HTML preview feature works very well. It is a built-in browser that shows the results of your code live without having to open a separate browser window. All you do is click on the "preview" tab. I did notice that sometimes you have to hit the refresh button to see your changes, but that is to be expected.

I also enjoyed the code formatting features very much. I tested them on an HTML file that I copied from an online web page. At first the HTML code was ran all together with no line breaks or indents. I simply went to the tool bar and clicked "Tools/Code Format/Format" and it formatted it perfectly. I am sure it will work with other supported languages equally well.

When you double-click on a variable name in the PHP editor, it highlights the variable and the dollar sign, so you can copy the whole thing and paste or search for it easily. Notepad++ wouldn't include the dollar sign and it was a bother to me.

CodeLobster has what they call a "Map" feature(View/Windows/Map) that I've never seen before. It is basically a small window with a tiny view of the document you're working on. It allows you to click anywhere in the Map to quickly jump to a section of the file that you need to work on. It is a great feature for working with large files.

Conclusion

I have concluded that CodeLobster will indeed fill a need in my current web development process. It is not yet developed enough to be the only tool I use simply because of that one SFTP connection it will not work correctly with. However I found that I can combine it with another tool, CyberDuck, as a work-around until the developers of CodeLobster work out the final kinks in the software and add some more nice-to-have type features such as spell-check and . In the end it still gets a thumbs up for all the wonderful features it has and for its robustness in general. Eventually I believe it will make the perfect all-in-one coding tool for my company. The other contender for my new all-in-one coding tool was CyberDuck and I can honestly say that CodeLobster has much more potential. CyberDuck is also in its early stages of development and it needs several improvements to be truly useful. CyberDuck fails in comparison to CodeLobster when it comes to Features. CodeLobster is truly made to be an all-in-one coding solution while CyberDuck is missing many of the necessary features to be a real competitor.

If you are interested in how I combined CodeLobster with CyberDuck to make up for CodeLobster's short-coming when connecting by SFTP, I'll quickly go over how to do it. Create a bookmark(that's what CyberDuck calls their FTP/SFTP connections) in CyberDuck to the SFTP server. CyberDuck allows you to set it up so you can use your own code editor with it in a way that allows you to click on a file from a remote server and open it in that editor. Then when you save it in that editor(CodeLobster), it saves it to the server. That way I still have a seamless process even though I have to combine two tools to get what I truly want.

The New Ajax

Okay, as a developer, I've been using Ajax for years. To be honest, it gives me a new ulcer every time I have to use it. Well, it still does lol, but not quite as bad. Today Ajax technology mixed with jQuery makes it almost tolerable to code with. Also, what's important, is that it is a very necessary technique if you want to go anywhere at all with your coding skills. Let me demonstrate how to make it easier for everyone. It is not really that hard if you break it down into the simplest form possible and then expand from there. Here is a simple Ajax call to get the results of a PHP script:

Update - I've decided to expand this article to include just about every way possible to send data to a PHP script and get a response, so I've include the JavaScript/Ajax method, the jQuery/Ajax method and the CURL method of sending data to a PHP script and getting back other data or manipulated data.

Simple JavaScript Ajax call to PHP script:

<!DOCTYPE html>
<html>
<head>
<script>
function get_txt(){
var xmlhttp;
if (window.XMLHttpRequest)
  {// code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }else{// code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
xmlhttp.onreadystatechange=function(){
  if (xmlhttp.readyState==4 && xmlhttp.status==200){
    var returnval=xmlhttp.responseText;
alert(returnval);
    }
  }
xmlhttp.open("GET","phpscript.php",true);
xmlhttp.send();
}
</script>
</head>
<body>
<input type="button" value="TEST" onclick="get_txt()" />
</body>
</html>

The above Ajax example will only retrieve data from the declared file. It could be altered to send data to it and receive a response, but I'm saving that gem for my next example using jQuery and Ajax to send a variable to a PHP file and get back a response.

Simple jQuery Ajax call to PHP Script that Returns Data:

The following jQuery and Ajax script will send the "user" variable holding the text "Ian" to the PHP script which will read the "user" variable with $_POST['user'] and send the results of the PHP code back to the Ajax script!

<script type="text/javascript" src="jquery.js"></script>
<div id="status"></div><hr>
<div id="rez"></div>
<script>
//var $j = jQuery.noConflict();//use j. instead of $ if needed

$.ajax({               
    type: "POST", // the kind of data we are sending
    url: "update_scores.php", // this is the file that processes the form data
    data: {user:'ian'}, // this is our serialized form-like data to pass to php
    success: function(msg){    // this function runs when the data has been successfully processed
        // this sets up our notification area for error / success messages
        $( document ).ajaxComplete(function( event,request, settings ) {
            $( "#status" ).append( "<p>Request Complete.</p>" );
            preresult = msg;
            $( "#rez" ).append( "<p>"+msg+"</p>" );
        });
            

    }
});
</script>

Sending Data to a PHP script and getting a response using CURL

Note that you could use CURL to do the same thing still using:

        $data = array('username' => 'ian');
        $results= curlPost('http://www.jafty.com/job.php', $data);

function curlPost($url, $data){
// check for URL and data:
    if(empty($url)){
        return 'Error: missing URL';
    }
    if(empty($data)){
        return 'Error: missing Data';
    }
//get the data for the POST and put in URL:
$fields_string = '';
foreach($data as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
$fields_string = rtrim($fields_string,'&');
//open connection
$ch = curl_init();
//Set the URL, no. of POST vars and POST data:
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($data));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
//timeout after 10 secs(change as needed):
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,10);
//curl_setopt($ch,CURLOPT_HEADER,false);
//Set to return data instead of printing:
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
//pass user agent:
curl_setopt($ch,  CURLOPT_USERAGENT , "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
//execute post:
$result = curl_exec($ch);
//close connection:
curl_close($ch);
return $result;
}//end curlPost function

...to pass a variable to a PHP script and save the PHP script's results in $results.

Post Form Data with CURL and get Results

Today, I am doing another domain name related project(on of my specialties). I need a fresh PHP function to get a domain name's whois data from. It's time to get dirty with CURL because all my PHP whois functions are failing lately after about 500 queries or less, so let's do this!

PHP CURL POST FUNCTION

The first thing I needed was a decent CURL function to post to a URL and get results. Here is what I found:

<?php
function curlPost($url, $data){
// check for URL and data:
    if(empty($url)){
        return 'Error: missing URL';
    }
    if(empty($data)){
        return 'Error: missing Data';
    }

//get the data for the POST and put in URL:
$fields_string = '';
foreach($data as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
$fields_string = rtrim($fields_string,'&');

//open connection
$ch = curl_init();

//Set the URL, no. of POST vars and POST data:
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($data));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);

//timeout after 10 secs(change as needed):
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,10);
//curl_setopt($ch,CURLOPT_HEADER,false);
//Set to return data instead of printing:
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
//pass user agent:
curl_setopt($ch,  CURLOPT_USERAGENT , "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

//execute post:
$result = curl_exec($ch);

//close connection:
curl_close($ch);
return $result;
}//end curlPost function.

echo "<h3>Whois test</h3>";
$data = array('d43b' => 'jafty.com');
echo curlPost('http://www.secret.net/services/whois.php', $data);

?>

Okay, I know, unfair, I took out the target domain, but find your own lol. I don't want everyone exploiting the poor server. But I was still limited with this technique because the server I queried had a maximum number of queries allowed, so I still need to find out how to overcome this.

cURL & PHP

Scraping with Curl using Cookies

Okay, I got a much needed lesson in scaping today using curl and cookies. I quickly discovered that you can not get the contents of some webpages without using cookies because some pages use cookies to validate requests for pages or other data.

My mission was to scrape the following URL from expireddomains.com:

http://member.expireddomains.net/domains/expiredcom/?start=0&o=changes&r=d

If you knew anything about expireddomains.com, you might know that the URL above is for members only, so the goal of this exercise is to make the web server at expireddomains.com think that you are a logged in user. Here is how:

So, first I tried this:

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
$html = curl_exec($ch);
curl_close($ch);

echo "HTML:<br>$html<hr>";

That returned nothing but a redirect to the login page of expireddomains.com. So then I read online how to use curl while sending a cookie with the page request and found that this worked:

$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
// cookies to be sent
curl_setopt($ch, CURLOPT_COOKIE, "PUT_COOKIE_HERE");
$html = curl_exec($ch);
curl_close($ch);

echo "HTML:<br>$html<hr>";

That worked! so I was home free, but more importantly here are the steps I took to get the value to replace PUT_COOKIE_HERE:

  1. First, I signed up for a free membership at expireddomains.com.
  2. Then I went to the page I wanted to scrape and made a note of the exact URL which was:  http://member.expireddomains.net/domains/expiredcom/?start=8550&o=changes&r=d
  3. So I copied the URL to my clipboard from step two above and pasted it into the Firefox address bar. Don't hit Enter yet.
  4. Then I opened Firebug and allowed the URL to resolve while Firebug was open so I could monitor the HTML headers sent.
  5. Look for a header that reads: "Cookie" and copy the value and paste it in place of PUT_COOKIE_HERE in the code above. Make sure the value is in parentheses.
  6. Now run your code again and you'll get results as long as that's all the webpage is looking for. I have had some that want a referrer set as well which you can also do with curl. Hint, Google curl set referrer for more information.