Web Crawling Software With API

[hyeager]

Can anyone recommend any web crawling software to gather information from specified web sites to keep the consolidated information on my site up to date? I'd like for this software to run in the background and gather the latest pricing and product description information from certain suppliers. I host my sites on Windows.

Thanks,

Hans

[paulhiles]

Hey Hans,

I've moved both of your software requests into Web Programming. There was a post quite recently in Content Management Systems that was dealing with spidering software. Take a look!
Spidering Software to Crawl Other Sites

[hyeager]

I don't mind that you repositioned my posts. Sorry I didn't post it in the correc category the first time. Thanks for pointing me to the other post. It is somewhat helpful.

Hans