Dec 24 2009

PHP bot to grab meteo information from Google

Category: Php,Spiders & web botsGiulio Pons @ 3:37 pm

Google has many usefull functions that give you data fast, such as cinema infos, or for meteo forecasts. I think that Google grabs those informations from the many sites indexed with his bots.

As I did on a previous post for words spelling you can retrieve those informations with some mini bots. The mini bot I’ve made for meteo retrieves informations from italian Google servers about weather forecast of a specified city (not only italian cities).

Since google gives only 4 days meteo if you ask for a date too much in the future it will return an empty string.

Below you can find the PHP source code, and here is a working demo!

This function is included in the Mini Bots Class.

function dayadd($days,$date=null , $format="d/m/Y"){
	//add days to a date function
	return date($format,strtotime($days." days",strtotime( $date ? $date : date($format) )));
}

function attr($s,$attrname) {
	//get the attribute value of an html tag
	preg_match_all('#\s*('.$attrname.')\s*=\s*["|\']([^"\']*)["|\']\s*#i', $s, $x);
	if (count($x)>=3) return $x[2][0];
	return "";
}

function doGoogleMeteo($q,$date) {
	if ($date>dayadd(3,date("Y-m-d"),"Y-m-d"))return "";

	// grab google page with meteo query
	$web_page = file_get_contents( "http://www.google.it/search?q=meteo+" . urlencode($q) );

	//parse html to find data, and store them in an array
	preg_match_all('#<div class=e>(.*)</table>#Us', $web_page, $m);
	if (count($m)>0) {
		$p = array();
		preg_match_all('#<img([^>]*)?>#Us', $m[0][0], $img);
		for ($i=0;$i<count($img[0]);$i++) {
			$tag = str_replace("src=\"/","src=\"http://www.google.it/",$img[0][$i]);
			$p[dayadd($i,date("Y-m-d"),"Y-m-d")]["title"] = attr($tag,"title");
			$p[dayadd($i,date("Y-m-d"),"Y-m-d")]["img"] = attr($tag,"src");
		}
		preg_match_all('#<nobr>(.*)</nobr>#Uis', $m[0][0], $nobr);
		for ($i=0;$i<count($nobr[1]);$i++) {
			$temp= explode("|",$nobr[1][$i]);
			$p[dayadd($i,date("Y-m-d"),"Y-m-d")]["min"] = trim($temp[1]);
			$p[dayadd($i,date("Y-m-d"),"Y-m-d")]["max"] = trim($temp[0]);
		}
		return $p[$date];
	}

	return "nada.";
}

print_r ( doGoogleMeteo("milano","2009-12-25") );
//Array (
// [title] => Rovesci
// [img] => http://www.google.it/images/weather/rain.gif
// [min] => -4°C
// [max] => 7°C
//)
Share

Tags: , , , , , , , , ,


Dec 23 2009

ASP web bot that get exchange rates

Category: Asp,Spiders & web botsGiulio Pons @ 3:06 pm

Thanks to Bank of Italy (Banca d’Italia), there are free exchange rates data that are easy to retrieve and parse, here is the code of a mini webbot written in ASP that goes out on Bank of Italy site, get the exchanges rates and print them. It’s very easy to customize and use it (since there are also historical data you can also make graphs like this).

exchange rates graph

<%
Dim yesterday
yesterday = dateadd("d",-1,date)

Dim objHttp
Set objHttp = Server.CreateObject("Microsoft.XMLHTTP")
objHttp.Open "GET", "http://uif.bancaditalia.it/UICFEWebroot/QueryOneDateAllCur?lang=en&rate=0&initDay=" & Day(yesterday) & "&initMonth=" & Month(yesterday) & "&initYear=" & Year(yesterday) & "&refCur=euro&R1=csv", False
objHttp.Send
If objHttp.Status = 200 Then
	Dim arrLines
	Dim arrFields

	Dim i
	arrLines = Split(objHttp.ResponseText,Chr(10))
	For i = 0 To UBound(arrLines)-1
		If i >= 4 then
			arrFields = Split(arrLines(i),",")
			response.write arrFields(4) & " " & arrFields(1) & " (" & arrFields(2) & ") per 1 EUR<br/>"
		End if
	Next
Else
	Response.Write(objHttp.Status & " - " & objHttp.StatusText)
End If
Set objHttp = Nothing
%>
Share

Tags: , , ,


Nov 10 2009

Do spelling using google spell checker

Category: Php,Spiders & web botsGiulio Pons @ 11:31 am

If you have a user input that may contains some error you can try to check the spelling using Google Spelling Suggestion service (there is an api and you have to register to have an api key to use their web services).

But you can obtain the same result searching the Google search engine and parsing the html code to find the link after the phrase: “Did you mean“. You can think at this code as a mini web bot spell checker.

This code works in any language, it finds the anchor tag that has the classname set to “spell”:

DEMO

echo doGoogleSpelling("wokipedia");  //returns "wikipedia"

function doGoogleSpelling($q) {

	// grab google page with search
	$web_page = file_get_contents( "http://www.google.it/search?q=" . urlencode($q) );

	// put anchors tag in an array
	preg_match_all('#<a([^>]*)?>(.*)</a>#Us', $web_page, $a_array);
	for($j=0;$j<count($a_array[0]);$j++) {

		// find link with spell suggestion and return it
		if(stristr($a_array[0][$j],"class=spell")) return strip_tags($a_array[0][$j]);

	}

	return "";
}
Share

Tags: , , , , , , , , , ,


« Previous Page