<title> » scraping http://www.adammoro.com/blog Internet Marketing, Web Development and Programming Stuff Fri, 19 Nov 2010 19:58:33 +0000 en hourly 1 http://wordpress.org/?v=3.0.1 Chunk Data for Easier Scraping http://www.adammoro.com/blog/chunk-data-for-easier-scraping.html http://www.adammoro.com/blog/chunk-data-for-easier-scraping.html#comments Sun, 21 Feb 2010 22:41:32 +0000 Adam Moro http://www.adammoro.com/blog/?p=56 Before you spend an hour writing some elaborate regular expression, try chunking the data and matching several expressions to make for a much simpler (and faster) scrape. So, assuming you're using PHP, after you've pulled the data (e.g. with file_get_contents()), use preg_replace with the following regex to chunk the data into a much easier "soup" to work with.

[ \t\r\n]

Here's an example of how to use this with PHP:

<?php
	$data = file_get_contents("http://www.adammoro.com/");
	$data = preg_replace('~[ \t\r\n]~', '', $data);
	print_r($data);
?>
]]> http://www.adammoro.com/blog/chunk-data-for-easier-scraping.html/feed 5