Dec 27 2009

Mini Bots PHP Class

Giulio Pons @ 11:49 pm

LAST UPDATE 01/11/2011 This php class is a growing collection of small spiders that go out on the web and make some small usefull works. I’ve decided to raise it to the top level menu beacuse I think it’s a nice project, and I will explain the functions in the blog.

Since this class could violate the rules of other sites, please, use this class properly, be correct: I mean, bots can cause problems to server, and can be used to steal contents, so use those scripts with attention (that is to say “not too much”). This class was build for educational purpose, so check the rules of the called services to decide if you can use those scripts in your commercial products, I don’t know.

Actually the mini bots can do those things:

Version 1.9f includes:

  1. check if a remote mp3 exists and is an mp3
  2. search Myspace Concerts (DEMO)
  3. search Google and get the spelling suggest (DEMO)
  4. search Google and retrieve the meteo information for a place
  5. search Bank Of Italy service to get exchange rates for a specified currency
  6. search GeoIpTool to retrieve geographical information from a specified IP address (DEMO)
  7. contact any mail server to validate email address through smtp protocol (DEMO)
  8. call tinyurl.com to get a tiny url of a specified long url (DEMO)
  9. decode any small url to the original long url (DEMO)
  10. url checker to see if a url exists
  11. get url infos to get data about a page (title, description, favicon, images) (DEMO)
  12. retrieves all the text from a url
  13. convert a url to a pdf and save it to a local file
  14. get twitter users info from web page and from Twitter’s API
  15. set twitter status
  16. get twitter statuses update list
  17. get video list uploaded by a user in Youtube (DEMO)
  18. get first page of url images from Google Images (DEMO)
  19. change facebook status
  20. post to facebook fan page wall
  21. parsing rss flickr feed of a user (DEMO)
  22. sending push alerts to iPhone (DEMO)
  23. google suggest keywords service (DEMO)
  24. find definitions on Wikipedia (DEMO)
  25. convert a street address to latitude/longitude coordinates (DEMO)

DOWNLOAD:

Here is the DOWNLOAD PHP source and the DEMO INDEX.

HISTORY:
ver.1.9g, added check if an mp3 exists and is an mp3, minor updates
ver.1.9e, fixed myspace
ver.1.9d, myspace, latitude longitude and more
ver.1.9, wikipedia definitions
ver.1.8, pushMeTo, google suggest
ver.1.7, various fixes and Flickr Feed parser
ver.1.6a, post to facebook fan page wall
ver.1.6, add twitterInfoApi (like twitterInfo but faster), add twitterSetStatus, add twitterGetStatusList, add youtubeGetVideos, googleGetImages
ver.1.5, modified twitterInfo, fixed private functions, addedd facebook status method.
ver.1.4, added twitterInfo, wepage2txt and url2pdf methods.
ver.1.3, added url_exists and getUrlInfo methods.

Share

43 Responses to “Mini Bots PHP Class”

  1. seeal says:
    January 6th, 2010 7:48 pm

    love this class thx :D

  2. Matteo says:
    January 7th, 2010 7:19 am

    Great class , thanks :)

  3. Greg Avola says:
    January 12th, 2010 8:45 pm

    Great work dude – way to go.

  4. uberVU - social comments says:
    January 14th, 2010 2:01 pm

    Social comments and analytics for this post…

    This post was mentioned on Twitter by deliciobot: #DelicioBot : Mini Bots PHP Class http://bit.ly/5sfw5Q...

  5. holyguard says:
    February 27th, 2010 1:46 am

    Utilissimo!!!

    Puoi usare l’exchange rate di Google invece di quello della banca d’Italia, che te ne pare?

  6. Tod Wedge says:
    March 11th, 2010 8:27 pm

    Just landed on this place via Google seek. I love it. This post change my percept and I am taking the RSS feeds. Cheers Up.

  7. webcarlo says:
    April 1st, 2010 6:11 pm

    complimenti però sappiate che Google non gradisce che si facciano query nei suoi servizi che non prevedono API, potrebbe penalizzare un sito che serve i dati con il grabbing

  8. admin says:
    April 5th, 2010 8:58 am

    Credo che l’importante sia non abusarne, anche Google fa grabbing a tutti i livelli, non sono sicuro che penalizzi queste attività, dove l’hai letto?

  9. PHP Mini bot « My Blog says:
    May 10th, 2010 7:25 am

    [...] there are 17 bots included in this class! Well I’ve updated the mini bots class with some new things: you can now retrive the list of videos from a Youtube user, you can retrieve [...]

  12. LongVNIT » Ping pingomatic.com services with PHP says:
    August 8th, 2010 4:20 pm

    [...] The complex part, for me, was understing XML-RPC rules and build the correct headers to call the service. The call is made with fsockopen php function. Here is the function that will be added to the next version of Mini Bot Class. [...]

  16. Jorge Garifuna says:
    September 6th, 2010 11:16 pm

    I just tried updating my twitter status with the mini bot’s twitterSetStatus() function and the functioned returned false which indicates that my status was not updated. I verified on twitter and no update was set to my status.

    Does this have anything to do with the oAuth implementation of twitter?

    Any ideas to get twitter update status to work in the near future?

  17. Giulio Pons says:
    September 7th, 2010 1:35 pm

    Yes. Since 31, 2010 twitter made basic autenthication deprecated, I will soon make available a different version of the function (I hope).

  18. Jorge Garifuna says:
    September 8th, 2010 11:33 pm

    Thanks a million. I’m currently using the facebook status update and it works great!

  21. Jorge Garifuna says:
    September 16th, 2010 2:42 am

    Any ideas to add hi5.com to this class?

  22. Facebook Status Updates mit PHP - c0de says:
    September 25th, 2010 4:23 pm

    [...] hab auf barattalo.it diese wirklich geniale Funktion entdeckt die er als teil einer Bot Classe [...]

  23. Mini Bot Class Updated - Barattalo says:
    October 6th, 2010 2:49 pm

    [...] Mini Bots PHP Class [...]

  24. Ronan says:
    December 9th, 2010 10:01 pm

    Bel lavoro davvero !

    peccato che tu non abbia scelto una licence per questo script.
    The Free Software Foundation has recommended that the GNU AGPLv3 be considered for any software that will commonly be run over a network

    Saluti dalla Bretagna,
    Ronan

  25. Giulio Pons says:
    December 10th, 2010 6:58 pm

    Ciao Ronan, non conosco bene le licenze, mi leggerò quella che mi consigli. Ciao!

  28. Douglas Ianitsky says:
    February 21st, 2011 6:55 pm

    Hey, Thanks for class.

    I modify the googleGetImages function for get a especify quanty. 

    public function googleGetImages($k, $quant = 20) {
		$x = array();
		$next = 0;
		while ($next < $quant) {
			$url = "http://www.google.com.br/images?q=##query##&gbv=2&sout=1&tbs=isch:1,itp:##query##&sa=N&start=##next##&ndsp=20";
			$web_page = file_get_contents( str_replace(array("##query##", "##next##"), array(urlencode($k), $next), $url ));
			$tieni = stristr($web_page,"dyn.setResults(");
			$tieni = str_replace( "dyn.setResults(","", str_replace(stristr($tieni,");"),"",$tieni) );
			$tieni = str_replace("[]","",$tieni);
			$m = preg_split("/[\[\]]/",$tieni);

			for($i=0;$i<count($m);$i++) {
				$m[$i] = str_replace("/imgres?imgurl\\x3d","",$m[$i]);
				$m[$i] = str_replace(stristr($m[$i],"\\x26imgrefurl"),"",$m[$i]);
				$m[$i] = preg_replace("/^\"/i","",$m[$i]);
				$m[$i] = preg_replace("/^,/i","",$m[$i]);
				if ($m[$i]!="") {
					array_push($x,$m[$i]);
					if (++$next >= $quant)
						break;
				}
			}
		}
		return $x;
	}
  29. Douglas Ianitsky says:
    February 22nd, 2011 7:02 pm

    Another small change

    // get list of images from google images
//$k == query to search
//$quant == total of itens that you need
//$check == check if image link is broken (depending on the amount of images, the time to check them becomes too long. Use only if absolutely necessary.)
public function googleGetImages($k, $quant = 20, $check = false) {
	$x = array();
	$next = 0;
	while ($next < $quant) {
		$url = "http://www.google.com/images?q=##query##&gbv=2&sout=1&tbs=isch:1&sa=N&start=##next##&ndsp=20";
		$web_page = file_get_contents( str_replace(array("##query##", "##next##"), array(urlencode($k), $next), $url ));
		$tieni = stristr($web_page,"dyn.setResults(");
		$tieni = str_replace( "dyn.setResults(","", str_replace(stristr($tieni,");"),"",$tieni) );
		$tieni = str_replace("[]","",$tieni);
		$m = preg_split("/[\[\]]/",$tieni);

		for($i=0;$i<count($m);$i++) {
			$m[$i] = str_replace("/imgres?imgurl\\x3d","",$m[$i]);
			$m[$i] = str_replace(stristr($m[$i],"\\x26imgrefurl"),"",$m[$i]);
			$m[$i] = preg_replace("/^\"/i","",$m[$i]);
			$m[$i] = preg_replace("/^,/i","",$m[$i]);
			if ($m[$i]!="") {
				if ($check) {
					if (@getimagesize($m[$i])) {
						array_push($x,$m[$i]);
						if (++$next >= $quant)
							break;
					}
				} else {
					array_push($x,$m[$i]);
					if (++$next >= $quant)
						break;
				}
			}
		}
	}
	return $x;
}
  30. Giulio Pons says:
    February 22nd, 2011 9:52 pm

    Thank you Douglas!

  31. Michael says:
    February 28th, 2011 1:44 am

    Wow really amazing, very usefull collection! Thanks!!

  32. nina says:
    March 1st, 2011 4:57 am

    can you make function for parse facebook wall feeds ?

  33. santiago says:
    March 25th, 2011 7:58 am

    Thanks!!

  34. Putranto says:
    May 13th, 2011 12:32 pm

    Just what i needed, twitter status change without messy oAuth !!! Thanks bro, excellent !!!

  35. Php – Script to ping pingomatic.com | Linux tutorials says:
    July 2nd, 2011 9:41 am

    [...] The complex part, for me, was understing XML-RPC rules and build the correct headers to call the service. The call is made withfsockopen php function. Here is the function that will be added to the next version of Mini Bot Class. [...]

  36. Stefano says:
    July 9th, 2011 11:45 am

    Sto provando il codice di Douglas per recuperare l’immagine da Google Images, ma ottengo sempre un errore di sintassi T_PUBLIC. Ho tolto il public alla funzione e non restituisce più l’errore ma non ritorna niente, pagina bianca (test effettuato tramite XAMPP).

    prima della classe ho semplicemente inserito una riga così:
    $k = “MC544FD/A”;
    Dopodichè ho modificato quantità in 1 ed abilitato il check dell’esistenza dell’immagine.

    Devo inserire altro codice per visualizzare l’immagine?

    Inoltre è possibile inserire un controllo “di pertinenza”? Mi spiego, se il primo risultato ha un nome estremamente diverso da quello cercato (e lo chiedo perchè io sto cercando dei codici di prodotto) è possibile dire allo script di skippare il grab dell’immagine?

    Grazie mille!

  38. Joseph says:
    July 30th, 2011 5:43 am

    I will use this class in my collection of web directories and submission form. Thanks.
    Directory Gist – Directory of Web Directories –
    http://www.directorygist.com
    ———–
    Searcheable database of 7000 free web directories, Search Engines, social BookMarking Sites, Article Directories. Also online submission tools to ease your website promotion.

  39. SAYIM says:
    July 30th, 2011 5:08 pm

    Great class.
    However, I need the page URL as well in the result.
    For example, I need http://www.example.com/the_page.php as well as http://www.example.com/the_page/imageX.jpg

    Thanks,

  40. Short Links says:
    August 19th, 2011 6:09 am

    Great class. Please could you add the short URL service at http://101.gs to “decode any small url to the original long url”

    Thanks

  42. Hung Nguyen says:
    November 10th, 2011 7:29 pm

    So great,
    Your Bot class so useful.
    Thanks

  43. Derrick Duriga says:
    November 14th, 2011 5:55 pm

    Greetings! I know this is somewhat off topic but I was wondering which blog platform are you using for this site? I’m getting sick and tired of WordPress because I’ve had issues with hackers and I’m looking at alternatives for another platform. I would be great if you could point me in the direction of a good platform.

Leave a Reply