Simpler DIY Search Engine and Web Mining
Apparently Alexa, Amazons search company, is offering its index to anyone who wants it (and is willing to pay for it - but its quite cheap and you only pay for CPU hours, storage used and data processed, no setup or licensing fee). From John Batelle's Searchblog:
Anyone can also use Alexa's servers and processing power to mine its index to discover things - perhaps, to outsource the crawl needed to create a vertical search engine, for example. Or maybe to build new kinds of search engines entirely, or ...well, whatever creative folks can dream up. And then, anyone can run that new service on Alexa's (er...Amazon's) platform, should they wish.
Although I currently don't have any great idea how to use it - this sounds great. Maybe one day we'll have the same for structured data, either Google Base going down the same path offering similar apis or Alexa/Amazon will build their own Base/Fremont.
More: Much more information on the Read/Write Web blog.
<< Home