personal / tool / archived
Haunted Scraper
PHP scraper (QueryPath + Tipsy) that crawls theshadowlands.net's haunted-places listings and writes them out to a haunted.csv file.
- BPM
- 136
- GRADE
- B
- LEVEL
- 4
- RELEASES
- 0
- YEARS
- 2015–2015
Scraping The Shadowlands
theshadowlands.net is one of the oldest corners of the web still around: a hand-maintained index of reportedly haunted places, listed state by state in flat HTML that's barely changed since the nineties. No API, no pagination, no JavaScript, just thousands of entries in markup, so it was easy to scrape.
In December 2015 I pointed a PHP script at it. QueryPath for the DOM traversal, Tipsy holding the app together, composer install, php app.php, and out comes haunted.csv. I just liked the idea of turning a big old collection of ghost stories into rows and columns.