Identifying The duplicate pages

[suthra]

Hi , how does a searchengine like google identify a duplicate page.

For example, i create 2 websites (seperate domains) with contents like:

Same Templates Footer , header for both domains and the content goes like this.
1st Domain :
In todays world everyone searches for money....

2nd Domain

In this world Everyone searches for money.....

Now humans knows this is a duplicate page , but how does a bot knows that it is a duplicate page?




Plz reply.,
Regds.,
sarathy.s

[Mel]

IMO it depends on the search engine involved, but to take google as an example, they parse the text into hitlists records for each page and I would imagine thet the hitlists would be near duplicates.

Another scheme which Google has patented, but which I don't know is in actual use by Google, is a technique to "fingerprint" each page by recording dozens of parameters and combining that into what is a unique id for each page based on its content.