I Need A Way To Scrape A Site's Content

[morestar]

Hello members and non-members of WebProWorld.com. I need your help today finding a way to scrape/grab all the textual (and possibly image) content and maybe even page name of a website.

Basically I'm going to be moving the new site we created for a client over to their new domain shortly and I would basically like to find a tool that will get the content so I can use it for another time - after we launch.

If anyone knows of a reliable tool that'll do this scraping and is hopefully free that would be fabulous! Thanks everyone.

[crankydave]

You don't have/can't get FTP access to the current site?

[morestar]

Nope, not yet Dave...and I don't want to wait.

I'm seeing programs everywhere, well, scripts but it's 2010, there must be one site that offers a free service to do so.

*still looking

[julien_simon]

this one isn't too bad. It will get you the files for a website. It won't grab any database or anything password protected of course. Let me know what you think: httrack.com

[morestar]

Thank you Julien, that program httrack is more than great and will now change the dynamic of the work I do in different areas too!

[julien_simon]

Thank you Julien, that program httrack is more than great and will now change the dynamic of the work I do in different areas too!

Glad I could help

[kgun]

Seamless for you: http://www.schrenk.com/nostarch/webbots/

[morestar]

With httrack, be weary - there's probably a setting for this - I just went with the default - kgun knows how much I love default settings (re:opera) - httrack seemed to also start scraping content from external links. I may have ticked Google and Youtube off a moment ago...

Some how I was in youtube's bin folder?

[julien_simon]

With httrack, be weary - there's probably a setting for this - I just went with the default - kgun knows how much I love default settings (repera) - httrack seemed to also start scraping content from external links. I may have ticked Google and Youtube off a moment ago...

Some how I was in youtube's bin folder?

did you find anything good?

I have not used it myself actually so I wasn't aware of that setting problem.

[kgun]

This is one of the simpler PHP cURL tasks that is described in the first chapters of the book cited in my last post. You can download the code from the book without buying it.

You will not appreciate cURL before you have learned it. Start with that book and PHP cURL and you don't need a third party site for this specific and similar tasks.

Related WPW thread: http://www.webproworld.com/webmaster...thical-WebBots.