This project has moved to Github, now it’s free and open source, you can find it here: https://github.com/giuliopons/minibots-php-class
This PHP class is a growing collection (since 2010!!) of small spiders and web bots that go out on the web and make some small useful jobs. This software can help to build your spider PHP scripts and easily brings new functionalities to your works.
You can do very nice works with CURL using php and this class, this small web bots can be included in your own works with a few line of code:
Get GPS coordinates from an address (geocode an address)
If you need to show a place on a map you need to convert the address from street, number and country, to a couple of GPS coordinates. You can use this getLatLong method of MiniBots class.
include("minibots.class.php"); $mb=new Minibots(); $gps = $mb-getLatLong("Piazza Duomo, Milan, Italy","APIKEYHERE"); print_r($gps);
And this will be the output, which you can include and use in a Google Maps:
Array ( [lat] = 45.465454000000001 [long] = 9.1865159999999992 )
Convert an IP address to a GPS location (Georeferencing IPs)
If you need to perform some action on users coming from a specific geographical location, you can use the MiniBots Class to convert an IP address to a geographical location, so you can for example choose what to show based on the country of the user. There are two methods that performs the same job but using different free services, here is the
include("minibots.class.php"); $mb=new Minibots(); $obj = $mb-ipToGeo(); print_r($obj);
And you’ve got this:
stdClass Object ( [ip] = 220.127.116.11 [country_code] = IT [country_name] = Italy [region_code] = 25 [region_name] = Lombardy [city] = Milan [zip_code] = 20121 [time_zone] = Europe/Rome [latitude] = 45.4667 [longitude] = 9.2 [metro_code] = 0 )
Get informations from WikiPedia with PHP
In this class there is this method called
wikiDefinition which allows you to retrieve basic informations from WikiPedia, for example you can search a famous people name and find the picture to show its face. Or you can search for a monument and get its GPS latitude and longitude coordinates.
The call is very simple and the results are packed into an object that you can easily parse to get everything you need, for example here is a call to retrieve informations about the small island of Favignana in Mediterranean sea:
include("minibots.class.php"); $mb=new Minibots(); $obj = $mb-wikiDefinition("favignana"); print_r($obj);
And this will be the output object, available in your code:
stdClass Object ( [title] = Favignana [extract] = Favignana is a comune including three islands (Favignana, Marettimo and Levanzo) of the Aegadian Islands, southern Italy. It is situated approximately 7 kilometres (4 miles) west of the coast of Sicily, between Trapani and Marsala, the coastal area where the Stagnone Lagoon and the international airport of Trapani, are sited. [thumbnail] = stdClass Object ( [source] = https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Favignana.jpg/320px-Favignana.jpg [width] = 320 [height] = 240 [original] = http://upload.wikimedia.org/wikipedia/commons/c/c8/Favignana.jpg ) [...cutted...] )
In the following list there are all the methods included in this class.
List of all features included:
- get ip address of the user
- add days to a date
- functions to easily parse attributes and tags from html
- get remote pages also with POST data
- get all links and all emails from a page
- extract text from a page
- function to make urls from local to absolute (useful during spidering)
- get the size of a remote file and check if a remote url exists
- copy a remote file to your server
- make SMTP validation for an address
- get informations on any URL (extract meta data, works also with Amazon products)
- get informations on YouTube and Vimeo URLs (also Youtube views count)
- read Facebook sharing counters for any URLs with Facebook App
- read Facebook fan counters for Facebook pages, without any facebook integration
- get tweet counters for shared counters
get Google Plus counters
- get Twitter user informations (with number of followers)
- get images from Instagram both with username and with hashtags
- get user counters for an Instagram user
- get sharing counters for Pinterest
get sharing counters for Linkedin
- get Google suggest for a bad spelled keyword
- convert an address to GPS coordinates
- get informations from Wikipedia
- geocode an ip address to a GPS coordinates and textual informations (region, country…)
- get money exchange rates
- get images for search engine (Picsearch) with any words
- get Gravatar pic for an email
- interact with Pingomatic service
- interact with NotifyMyDevice service
This is a growing and changing list.
Some methods could stop working, so please check if the method you need is working with the interactive demos of MiniBots Class. If the method you’re searching isn’t available in the interactive page you can try the complete demo test page.
Documentation, support and limitations
Every method is easy to call and documented. The documentation of the class explicitly tells to pay attention to use this class especially for production systems. It comes as it is and even if I give you support through CodeCanyon, sometimes free services are closed and can’t be scraped anymore.
Moreover, sometimes services automatically blocks spiders that makes many requests and couse heavy traffic. Remember alsways that heavy traffic means heavy costs for servers, so don’t make too many requests and store data locally when possible. Check the services you’re using (in documentations you will find the name of them) and if you have to make heavy usage contact them and buy their services.
If you’re interested on making spiders, here is a tutorial on how to build a spider.
Download MiniBots Class FREE
Now open source on Github Minibots PHP Class.
love this class thx :D
Great class , thanks :)
Great work dude – way to go.
Puoi usare l’exchange rate di Google invece di quello della banca d’Italia, che te ne pare?
Just landed on this place via Google seek. I love it. This post change my percept and I am taking the RSS feeds. Cheers Up.
complimenti però sappiate che Google non gradisce che si facciano query nei suoi servizi che non prevedono API, potrebbe penalizzare un sito che serve i dati con il grabbing
Credo che l’importante sia non abusarne, anche Google fa grabbing a tutti i livelli, non sono sicuro che penalizzi queste attività, dove l’hai letto?
I just tried updating my twitter status with the mini bot’s twitterSetStatus() function and the functioned returned false which indicates that my status was not updated. I verified on twitter and no update was set to my status.
Does this have anything to do with the oAuth implementation of twitter?
Any ideas to get twitter update status to work in the near future?
Yes. Since 31, 2010 twitter made basic autenthication deprecated, I will soon make available a different version of the function (I hope).
Thanks a million. I’m currently using the facebook status update and it works great!
Any ideas to add hi5.com to this class?
Bel lavoro davvero !
peccato che tu non abbia scelto una licence per questo script.
The Free Software Foundation has recommended that the GNU AGPLv3 be considered for any software that will commonly be run over a network
Saluti dalla Bretagna,
Ciao Ronan, non conosco bene le licenze, mi leggerò quella che mi consigli. Ciao!
Hey, Thanks for class.
I modify the googleGetImages function for get a especify quanty.
Another small change
Thank you Douglas!
Wow really amazing, very usefull collection! Thanks!!
can you make function for parse facebook wall feeds ?
Just what i needed, twitter status change without messy oAuth !!! Thanks bro, excellent !!!
Sto provando il codice di Douglas per recuperare l’immagine da Google Images, ma ottengo sempre un errore di sintassi T_PUBLIC. Ho tolto il public alla funzione e non restituisce più l’errore ma non ritorna niente, pagina bianca (test effettuato tramite XAMPP).
prima della classe ho semplicemente inserito una riga così:
$k = “MC544FD/A”;
Dopodichè ho modificato quantità in 1 ed abilitato il check dell’esistenza dell’immagine.
Devo inserire altro codice per visualizzare l’immagine?
Inoltre è possibile inserire un controllo “di pertinenza”? Mi spiego, se il primo risultato ha un nome estremamente diverso da quello cercato (e lo chiedo perchè io sto cercando dei codici di prodotto) è possibile dire allo script di skippare il grab dell’immagine?
I will use this class in my collection of web directories and submission form. Thanks.
Directory Gist – Directory of Web Directories –
Searcheable database of 7000 free web directories, Search Engines, social BookMarking Sites, Article Directories. Also online submission tools to ease your website promotion.
However, I need the page URL as well in the result.
For example, I need http://www.example.com/the_page.php as well as http://www.example.com/the_page/imageX.jpg
Great class. Please could you add the short URL service at http://101.gs to “decode any small url to the original long url”
Your Bot class so useful.
I am trying to use your dosmtpvalidation function but am not able to build a form to pass variables to that function. Could you demonstrate what the form looks like? I tried to copy example at http://www.barattalo.it/demo/dosmtpvalidation.php including jquery and css file but submit button does nothing when I copy it exactly to my website.
Demo for minibots.class.php Does not work
googleGetImages class didnt work
some of features are not working now.
google get images class not working
minibots.class.php Does not work
Great Class! Any chance of getting the source to the demo pages? I got the twitter class working, with _post[“u”] etc, but I think what I have can be done better. Plus when I submit it, it just displays a double of my twitter form where the result should say, bad, ok, etc.
Could I possibly get the code for the twitter demo page?
googlegetimages not working =(
This is very helpfull for me.
hi i like your work but how to use it i m new to php, what i whould to do is to get link of first 3 image ( and medium size)
$k = ‘demo’;
$images = googleGetImages($k, $quant = 3, $check = false)
I want to get latitude and longitude from address.
The demo which you have shown is mapping exactly.
Can you provide me the entire source.
Im new to PHP and not able to link it.
Please help me
google has changed its code and I have to fix it.
Do you ever tried to curl dropbox.com ?
Looks impossible but i don’t know why….
Love your bot class by the way
I think dropbox has api libraries, probably it’s better use the apis.
Awsome class Thanks!
I notced that in minibots Ver.3.61 since 14.03.2017 getInstagramPicsByTag method does not works (Undefined offset on line 1055). –> http://www.barattalo.it/demo/minibots/test2.php
Thanks, unfortunately Instagram has just closed the public access to the tag exploration pages. So now it’s not possible to retrieve the images by tag.
The method to get Instagram pics by tag works again!
Hi, it looks like Instagram methods aren’t working anymore.
Hi Seb, I see it working on http://www.barattalo.it/instapostcards
Ok Giulio, I see it working too in instapostcards. Maybe it is just related to GetInstagramPic method. I have a few scripts based on that method and they don’t work anymore since this morning.
Codecanyon shows today’s update but lets me download the previous version. Is it normal?
I’ve just uploaded it to codecanyon, they need one or two days to approve it before you will be able to download the new version.
Hi, I think readFacebookPageCounters is not working properly anymore, I tried even with the last version 3.68!
Hi, since August 2016 the readFacebookCounters method needs also to configure an app on Facebook and use appid and app secret. The readFacebookPageCounters (for facebook pages) I see that have big restrictions on usage, so sometimes it works and sometimes it doesn’t. I’m going to investigate a little bit more on it.
You can try it here: http://www.barattalo.it/demo/minibots.php
The method is working, probably there are some limitations.
Thank you eery much ercan zengin
I can with this class download photo from instagram to my FTP and description of same photo to SQL? after looking and take all photos from a #hashtag?
if yes I buy it!
yes, but you have to code it. the class allows you to get filenames and description taking photos by hashtag, but: it doesn’t save file locally (you have to do it), it doesn’t make queries to your db (you have to do it), it doesn’t get all the photos for a tag but only the first 12 pictures of the most recents image and the 12 most recent popular images (if you want to get all future images you have to make multiple queries with a timer and save new pictures everytime you find one).
Thanks for fast answer. I already tested ur script and is rly cool! Big thanks! I have some question:
1. How I can show numbers of count edge_followed_by? because I cant show same way as like $show[“user”].
I mean how to shows this part of code:
2. Did u think about update ur script to looking relation from Instagram where is one special tag?
3. Do you have meybe a some example what exactly you mean with multiple queries with a timer?
Here I need to add ur script download much more than 24 photos form #tag :)
Hey, how can I installed mini boat on server, and how much url copy & save my db in a day.
I don’t understand your question, sorry.
(Hey, how can I installed mini boat on server, and how much url copy & save my db in a day.)
1. How add this class in my php search engine project codding.
2. this class how much url crawl in single day & all data can save my own database directly.
Hi, I want take information about:
– USERNAME, COMMENTS, LIKE, FLOWERS
when I give link for photo like this:
Now we have option only when I give for script $user or $tag
How I need to change script for put WHOLE link for photo and then take information from photo and user?
There is also a method
getInstagramPicto let you get data from a picture url.
It’s up to your coding skills. The Minibots class gives you a few method to search and retrieve data from the last pictures of a user, last pictures for a tag, or from a url of an instgram post. It doesn’t crawl, it doesn’t store in a db.
1) If you have an item of an array which is an object you can get the variables of the object simply with
2) what do you mean with “special tag”
3) I don’t have an example, sorry. You have to make a
cronwhich calls your php script. Your script should perform the search and if it finds new images store them. You can set cron to be called, for example, every hour. This way you have a script that every hour search for new images of a tag and you can monitor all the new images coming from a tag.