Mini Bots PHP Class

A powerfull PHP class to perform a lot of small jobs in your PHP programs

This PHP class is a growing collection (from 2010!!) of small spiders and web bots that go out on the web and make some small useful jobs. This software can help to build your spider PHP scripts and easily brings new functionalities to your works.

You can do very nice works with CURL using php and this class, this small web bots can be included in your own works with a few line of code:

Get GPS coordinates from an address (geocode an address)

If you need to show a place on a map you need to convert the address from street, number and country, to a couple of GPS coordinates. You can use this getLatLong method of MiniBots class.

include("minibots.class.php");
$mb=new Minibots();
$gps = $mb->getLatLong("Piazza Duomo, Milan, Italy");
print_r($gps);

And this will be the output, which you can include and use in a Google Map:

Array (
 [lat] => 45.465454000000001
 [long] => 9.1865159999999992
 )

Convert an IP address to a GPS location (Georeferencing IPs)

If you need to perform some action on users coming from a specific geographical location, you can use the MiniBots Class to convert an IP address to a geographical location, so you can for example choose what to show based on the country of the user. There are two methods that performs the same job but using different free services, here is the ipToGeo call:

include("minibots.class.php");
$mb=new Minibots();
$obj = $mb->ipToGeo();
print_r($obj);

And you’ve got this:

stdClass Object
(
    [ip] => 93.51.121.24
    [country_code] => IT
    [country_name] => Italy
    [region_code] => 25
    [region_name] => Lombardy
    [city] => Milan
    [zip_code] => 20121
    [time_zone] => Europe/Rome
    [latitude] => 45.4667
    [longitude] => 9.2
    [metro_code] => 0
)

 

 

Get informations from WikiPedia with PHP

In this class there is this method called wikiDefinition which allows you to retrieve basic informations from WikiPedia, for example you can search a famous people name and find the picture to show its face. Or you can search for a monument and get its GPS latitude and longitude coordinates.

The call is very simple and the results are packed into an object that you can easily parse to get everything you need, for example here is a call to retrieve informations about the small island of Favignana in Mediterranean sea:

include("minibots.class.php");
$mb=new Minibots();
$obj = $mb->wikiDefinition("favignana");
print_r($obj);

And this will be the output object, available in your code:

stdClass Object (
    [title] => Favignana
    [extract] => Favignana is a comune including three islands (Favignana, Marettimo and Levanzo) of the Aegadian Islands, southern Italy. It is situated approximately 7 kilometres (4 miles) west of the coast of Sicily, between Trapani and Marsala, the coastal area where the Stagnone Lagoon and the international airport of Trapani, are sited.
    [thumbnail] => stdClass Object (
             => https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Favignana.jpg/320px-Favignana.jpg
            [width] => 320
            [height] => 240
            [original] => http://upload.wikimedia.org/wikipedia/commons/c/c8/Favignana.jpg
        )
    [originalimage] => stdClass Object (
             => https://upload.wikimedia.org/wikipedia/commons/c/c8/Favignana.jpg
            [width] => 800
            [height] => 600
        )
    [lang] => en
    [dir] => ltr
    [timestamp] => 2017-01-22T21:52:40Z
    [description] => Italian comune
    [coordinates] => stdClass Object (
            [lat] => 37.93505556
            [lon] => 12.27233333
        )
    [url] => https://en.wikipedia.org/wiki/Favignana
)

In the following list there are all the methods included in this class.

List of all features included:

  • get ip address of the user
  • add days to a date
  • functions to easily parse attributes and tags from html
  • get remote pages also with POST data
  • get all links and all emails from a page
  • extract text from a page
  • function to make urls from local to absolute (useful during spidering)
  • get the size of a remote file and check if a remote url exists
  • copy a remote file to your server
  • make smtp validation for an address
  • get informations on any URL (extract meta data, works also with Amazon products)
  • get informations on YouTube and Vimeo URLs
  • read Facebook sharing counters for any URLs with Facebook App
  • read Facebook fan counters for Facebook pages, without any facebook integration
  • get tweet counters for shared counters
  • get Google Plus counters
  • get Twitter user informations (with number of followers)
  • get images from Instagram both with username and with hashtags
  • get user counters for an Instagram user
  • get sharing counters for Pinterest
  • get sharing counters for Linkedin
  • get Google suggest for a bad spelled keyword
  • convert an address to GPS coordinates
  • get informations from WikiPedia
  • geocode an ip address to a GPS coordinates and textual informations (region, country…)
  • get money exchange rates
  • get images for search engine (Picsearch) with any words
  • get Gravatar pic for an email
  • interact with Pingomatic service
  • interact with NotifyMyAndroid service

This is a growing and changing list.

You can play with some interactive demos of MiniBots Class here or you can watch a complete demo.

Documentation, support and limitations

Every method is easy to call and documented. The documentation of the class explicitly tells to pay attention to use this class especially for production systems. It comes as it is and even if I give you support through CodeCanyon, sometimes free services are closed and can’t be scraped anymore.

Moreover, sometimes services automatically blocks spiders that makes many requests and couse heavy traffic. Remember alsways that heavy traffic means heavy costs for servers, so don’t make too many requests and store data locally when possible. Check the services you’re using (in documentations you will find the name of them) and if you have to make heavy usage contact them and buy their services.

If you’re interested on making spiders, here is a tutorial on how to build a spider.

Download MiniBots Class / Price and buy

This class is available for download for just 9$ on CodeCanyon.

Download (9$)

Comments on “Mini Bots PHP Class”

53 thoughts

  1. seeal says:

    love this class thx :D

  2. Matteo says:

    Great class , thanks :)

  3. Greg Avola says:

    Great work dude – way to go.

  4. holyguard says:

    Utilissimo!!!

    Puoi usare l’exchange rate di Google invece di quello della banca d’Italia, che te ne pare?

  5. Tod Wedge says:

    Just landed on this place via Google seek. I love it. This post change my percept and I am taking the RSS feeds. Cheers Up.

  6. webcarlo says:

    complimenti però sappiate che Google non gradisce che si facciano query nei suoi servizi che non prevedono API, potrebbe penalizzare un sito che serve i dati con il grabbing

  7. admin says:

    Credo che l’importante sia non abusarne, anche Google fa grabbing a tutti i livelli, non sono sicuro che penalizzi queste attività, dove l’hai letto?

  8. I just tried updating my twitter status with the mini bot’s twitterSetStatus() function and the functioned returned false which indicates that my status was not updated. I verified on twitter and no update was set to my status.

    Does this have anything to do with the oAuth implementation of twitter?

    Any ideas to get twitter update status to work in the near future?

  9. Giulio Pons says:

    Yes. Since 31, 2010 twitter made basic autenthication deprecated, I will soon make available a different version of the function (I hope).

  10. Thanks a million. I’m currently using the facebook status update and it works great!

  11. Any ideas to add hi5.com to this class?

  12. Ronan says:

    Bel lavoro davvero !

    peccato che tu non abbia scelto una licence per questo script.
    The Free Software Foundation has recommended that the GNU AGPLv3 be considered for any software that will commonly be run over a network

    Saluti dalla Bretagna,
    Ronan

  13. Giulio Pons says:

    Ciao Ronan, non conosco bene le licenze, mi leggerò quella che mi consigli. Ciao!

  14. Douglas Ianitsky says:

    Hey, Thanks for class.

    I modify the googleGetImages function for get a especify quanty.

    public function googleGetImages($k, $quant = 20) {
    		$x = array();
    		$next = 0;	
    		while ($next < $quant) {
    			$url = "http://www.google.com.br/images?q=##query##&gbv=2&sout=1&tbs=isch:1,itp:##query##&sa=N&start=##next##&ndsp=20";
    			$web_page = file_get_contents( str_replace(array("##query##", "##next##"), array(urlencode($k), $next), $url ));
    			$tieni = stristr($web_page,"dyn.setResults(");
    			$tieni = str_replace( "dyn.setResults(","", str_replace(stristr($tieni,");"),"",$tieni) );
    			$tieni = str_replace("[]","",$tieni);
    			$m = preg_split("/[\[\]]/",$tieni);
    			
    			for($i=0;$i<count($m);$i++) {
    				$m[$i] = str_replace("/imgres?imgurl\\x3d","",$m[$i]);
    				$m[$i] = str_replace(stristr($m[$i],"\\x26imgrefurl"),"",$m[$i]);
    				$m[$i] = preg_replace("/^\"/i","",$m[$i]);
    				$m[$i] = preg_replace("/^,/i","",$m[$i]);
    				if ($m[$i]!="") {
    					array_push($x,$m[$i]);
    					if (++$next >= $quant)
    						break;
    				}
    			}
    		}		
    		return $x;
    	}
    
  15. Douglas Ianitsky says:

    Another small change

    // get list of images from google images
    //$k == query to search
    //$quant == total of itens that you need
    //$check == check if image link is broken (depending on the amount of images, the time to check them becomes too long. Use only if absolutely necessary.)
    public function googleGetImages($k, $quant = 20, $check = false) {
    	$x = array();
    	$next = 0;	
    	while ($next < $quant) {
    		$url = "http://www.google.com/images?q=##query##&gbv=2&sout=1&tbs=isch:1&sa=N&start=##next##&ndsp=20";
    		$web_page = file_get_contents( str_replace(array("##query##", "##next##"), array(urlencode($k), $next), $url ));
    		$tieni = stristr($web_page,"dyn.setResults(");
    		$tieni = str_replace( "dyn.setResults(","", str_replace(stristr($tieni,");"),"",$tieni) );
    		$tieni = str_replace("[]","",$tieni);
    		$m = preg_split("/[\[\]]/",$tieni);
    		
    		for($i=0;$i<count($m);$i++) {
    			$m[$i] = str_replace("/imgres?imgurl\\x3d","",$m[$i]);
    			$m[$i] = str_replace(stristr($m[$i],"\\x26imgrefurl"),"",$m[$i]);
    			$m[$i] = preg_replace("/^\"/i","",$m[$i]);
    			$m[$i] = preg_replace("/^,/i","",$m[$i]);
    			if ($m[$i]!="") {
    				if ($check) {
    					if (@getimagesize($m[$i])) {
    						array_push($x,$m[$i]);
    						if (++$next >= $quant)
    							break;
    					}
    				} else {
    					array_push($x,$m[$i]);
    					if (++$next >= $quant)
    						break;
    				}
    			}
    		}
    	}		
    	return $x;
    }
    
  16. Giulio Pons says:

    Thank you Douglas!

  17. Michael says:

    Wow really amazing, very usefull collection! Thanks!!

  18. nina says:

    can you make function for parse facebook wall feeds ?

  19. santiago says:

    Thanks!!

  20. Putranto says:

    Just what i needed, twitter status change without messy oAuth !!! Thanks bro, excellent !!!

  21. Stefano says:

    Sto provando il codice di Douglas per recuperare l’immagine da Google Images, ma ottengo sempre un errore di sintassi T_PUBLIC. Ho tolto il public alla funzione e non restituisce più l’errore ma non ritorna niente, pagina bianca (test effettuato tramite XAMPP).

    prima della classe ho semplicemente inserito una riga così:
    $k = “MC544FD/A”;
    Dopodichè ho modificato quantità in 1 ed abilitato il check dell’esistenza dell’immagine.

    Devo inserire altro codice per visualizzare l’immagine?

    Inoltre è possibile inserire un controllo “di pertinenza”? Mi spiego, se il primo risultato ha un nome estremamente diverso da quello cercato (e lo chiedo perchè io sto cercando dei codici di prodotto) è possibile dire allo script di skippare il grab dell’immagine?

    Grazie mille!

  22. Joseph says:

    I will use this class in my collection of web directories and submission form. Thanks.
    Directory Gist – Directory of Web Directories –
    http://www.directorygist.com
    ———–
    Searcheable database of 7000 free web directories, Search Engines, social BookMarking Sites, Article Directories. Also online submission tools to ease your website promotion.

  23. SAYIM says:

    Great class.
    However, I need the page URL as well in the result.
    For example, I need http://www.example.com/the_page.php as well as http://www.example.com/the_page/imageX.jpg

    Thanks,

  24. Short Links says:

    Great class. Please could you add the short URL service at http://101.gs to “decode any small url to the original long url”

    Thanks

  25. Hung Nguyen says:

    So great,
    Your Bot class so useful.
    Thanks

  26. C. Davis says:

    Hi,

    I am trying to use your dosmtpvalidation function but am not able to build a form to pass variables to that function. Could you demonstrate what the form looks like? I tried to copy example at http://www.barattalo.it/demo/dosmtpvalidation.php including jquery and css file but submit button does nothing when I copy it exactly to my website.

  27. alex says:

    hi

    Demo for minibots.class.php Does not work

  28. Kazamboo says:

    googleGetImages class didnt work

  29. Jack says:

    some of features are not working now.

  30. James says:

    Great Class! Any chance of getting the source to the demo pages? I got the twitter class working, with _post[“u”] etc, but I think what I have can be done better. Plus when I submit it, it just displays a double of my twitter form where the result should say, bad, ok, etc.

    Could I possibly get the code for the twitter demo page?

  31. alper c says:

    googlegetimages not working =(

  32. Renga says:

    This is very helpfull for me.
    Thanks

  33. hi i like your work but how to use it i m new to php, what i whould to do is to get link of first 3 image ( and medium size)

    $k = ‘demo’;
    $images = googleGetImages($k, $quant = 3, $check = false)
    print_r $images;

  34. sampath says:

    Hii,
    I want to get latitude and longitude from address.
    The demo which you have shown is mapping exactly.
    Can you provide me the entire source.
    Im new to PHP and not able to link it.
    Please help me

  35. Giulio Pons says:

    google has changed its code and I have to fix it.

  36. gary says:

    Do you ever tried to curl dropbox.com ?

    Looks impossible but i don’t know why….

    Love your bot class by the way

    thanks

  37. Giulio Pons says:

    I think dropbox has api libraries, probably it’s better use the apis.

  38. Inarus says:

    Awsome class Thanks!
    I notced that in minibots Ver.3.61 since 14.03.2017 getInstagramPicsByTag method does not works (Undefined offset on line 1055). –> http://www.barattalo.it/demo/minibots/test2.php

  39. Giulio Pons says:

    Thanks, unfortunately Instagram has just closed the public access to the tag exploration pages. So now it’s not possible to retrieve the images by tag.

  40. Giulio Pons says:

    The method to get Instagram pics by tag works again!

  41. Seb says:

    Hi, it looks like Instagram methods aren’t working anymore.

  42. Giulio Pons says:

    Hi Seb, I see it working on http://www.barattalo.it/instapostcards

  43. Seb says:

    Ok Giulio, I see it working too in instapostcards. Maybe it is just related to GetInstagramPic method. I have a few scripts based on that method and they don’t work anymore since this morning.

  44. Giulio Pons says:

    Ok, fixed!

  45. Seb says:

    Awesome Giulio!

  46. Seb says:

    Codecanyon shows today’s update but lets me download the previous version. Is it normal?

  47. Giulio Pons says:

    I’ve just uploaded it to codecanyon, they need one or two days to approve it before you will be able to download the new version.

  48. Toby Woby says:

    Hi, I think readFacebookPageCounters is not working properly anymore, I tried even with the last version 3.68!

  49. Giulio Pons says:

    Hi, since August 2016 the readFacebookCounters method needs also to configure an app on Facebook and use appid and app secret. The readFacebookPageCounters (for facebook pages) I see that have big restrictions on usage, so sometimes it works and sometimes it doesn’t. I’m going to investigate a little bit more on it.

  50. Giulio Pons says:

    You can try it here: http://www.barattalo.it/demo/minibots.php
    The method is working, probably there are some limitations.

Leave a Reply

Your email address will not be published. Required fields are marked with an *