|
|
||||||
|
||||||
| Index Link To US Private Messages Archive FAQ RSS | ||||||
| eCommerce Discussion Forum Ask questions about web hosting, merchant services and ecommerce issues. Topics include shopping carts, security, payment strategies, storefront partnerships, etc. |
Share Thread: & Tags
|
||||
|
![]() |
|
|
LinkBack | Thread Tools | Display Modes |
|
|||
|
We've heard from one of our vendors that a competitor is using a spider to gather pricing information and automatically adjust their prices to beat ours and the other competitors. I have a couple of questions on this.
1) Is this legal? I mean, that would cost us money in bandwidth, CPU time (we've been considering upgrading servers because of peak times, etc.), as well as causing some bottlenecks at other places on the web. 2) How would we go about catching them? I've looked for unusual user agents, and I don't really see much. I know it is a simple matter to cloak a bot to look like a legit browser, so how would I identify it? I have come across a few IP's that are using more bandwidth than others, but they don't seem to resolve to anywhere close to this competitor's IP block. Any suggestions are welcome. Brian.
__________________
ToolBarn.com, an Internet Retailer Top 500 and Inc. 500 Company | Tool Parts | Pet Supplies |
|
|||
|
I don't think it's illegal for them to use bots to scrape your site unless unless you specify somewhere that you don't allow that sort of thing.
I agree that identifying a bot like that would be your biggest challenge. IP will most likely not be a big help; they could run the bot from their web host, their office, their home, etc. If the bot obeys your robots.txt, that might be a start; look for agents that look like humans but pull your robots.txt. But be careful, looking at robots.txt's has become something of a hobby of mine. I'll check it during a site review, or considering a link exchange partners, sometimes I'm just curious; I imagine other developers also do this from time to time. If this bot doesn't grab your robots.txt or intentionally disobeys it, you can set up a bad bot trap such as the one described here. Good luck. |
|
|||
|
You can block IPs that have search a significant number of pages on your site. I wouldn't recommend this as the competitor can use multiple proxy IPs. Any ideas on this topic? I would like to know more about this as well.
Thanks, benc007@excite.com |
|
||||
|
Hi Brian,
They could simply log onto your site and cut the prices anyway. They have simply automated the process. Its been happening in retail for years. If you have a legitimate site and a good customer base and offer integrity then buyers will stick with you. Basically we all shop to penny pinch but there is a difference between saving $1 or $5 or $10 etc. You know what your profit margin is and should have a very good idea of what the general price is for your products. If they are undercutting too much they will go out of business anyway. Annoying as it seems live with and get on. It is legal, as far as I know, because you can sell anything for any price. Whether you make a profit is another matter. Identifying their IP is a waste of time, so go and have a few cold beers and forget about it for a while. The one suggestion is to look at your pages that are indexed well and exclude robots from the other pages. To me not a good suggestion but could possibly work by allowing all the category pages to be linked but exclude all the product pages. I am just having a guess here so think about it before trying anything.
__________________
Keimos - Always learning something new each day www.keimos.co.uk , www.keimos.net , www.selfpacedit.co.uk |
|
|||
|
benc007 is right: it is legal (depending maybe on place of jurisdiction). It is also good business sense. I've always used every means possible to gain competitive data and always been successful ;)
__________________
eSolutionsWork |
|
||||
|
I specifically disallow unapproved bots on my site, my notice is here near the bottom of the page:
http://www.andilinks.com/pol.htm Identifying the bots for banning is a little trickier but you'll find that even when they try to disguise themselves as browsers they usually don't download your images or css files etc., so any domain that is downloading significant bandwith with too few hits per kb can be safely assumed to be a bot. Say if your typical page is 30K with ten extra files called per page and an IP is downloading 90K with just 3 hits it will definitely be a bot. You can then ban the IP with .htaccess. I find many are just one-time visits but if your competitor is scanning for prices they will probably be using a static IP or the same block of IP's every time. Often I will deny the entire range, omitting the last set of 255 of the IP on the .htaccess. That is: Deny from 255.255.255. Instead of: Deny from 255.255.255.255 Andi
__________________
...the Rockies may tumble, Gibralter may crumble... G & I Gershwin, 1937 |
|
|||
|
That is really quite a bad idea - firstly if someone wanys to copy content off your site blocking the IP will have little effect. They will get another IP, use a different ISP etc.
Secondly how can you be sure they are not a bot and not a legitimate search engine / user. Thirdly blocking a whole class C means not only are blocking them but also 250+ other people who may have wanted to view your site. Lastly I would not waste the time - better spending it on marketing etc. |
|
|||
|
I say find a way to redirect the bot to a mirror of your site that only that one particular bot can see (no other spiders or humans) and change your prices on this mirror site to $0.15. This way, if they are not paying attention, they'll get burned. (at least for a short while)
This is probably much easier said then done, and may even step into a gray area of the law. I dunno. But thinking about, it sounds fun. :) |
|
|||
|
Instead of prices, worry about getting more traffic than your competitor. They could be giving their products away for free, but if you get more traffic, you'll be making more money.
__________________
Linketeria Website Directory - Get listed instantly! |
|
|||
|
I don't doubt the legality of this technique, but it's definately not cool: lazy at best, unethical at worst. I don't thing banning bots is a reasonable response, and I doubt a bot like this would accept the order anyway.
I agree with Bgump. It might take some trial and error, and you might need to take some seo risks (redirects and mirror pages, etc) but it should be a simple matter to trick a bot into selling at below cost. It's that or... well... just deal. |
|
|||
|
Quote:
Great post. I agree traffic is king. If you get 10 times more traffic so what if you lose a few percentages who got low balled. Be sticky, have a great site that people like better, pump in quality traffic. Anything else you do can lose you more sales in the long run by losing SE rank or blocking legit customers...or worse yet...god forbid have people buy all of your products at the posted price of $0.15! |
|
|||
|
I can not see it being illegal, because what is to stop them browsing manually and then using the pricing ?
Over here in UK the petrol stations do something similar. Some guarantee to be lower than any within five miles. (they charge too much anyway - but that is another hot topic) Cheers Denis |
|
|||
|
I'm not very techinical, so I am not sure about whether this would work.
Essentially, it would be great if you could find a way to redirect the bot like BGump said, but send it off to your competitor's site! If this worked (unlikely I guess) it would be great watching the bot and the site get into a loop until their prices all dropped to zero...
__________________
Have a Japan or Tattoo-related website? Join my affilate program and make up to 50% per sale! I use and recommend IDevAffliate. |
|
|||
|
There was a big lawsuit a while ago: Ticketmaster vs Tickets.com. It was about deep-linking and prices. Here are some links* you may find interesting:
http://www.gigalaw.com/library/ticke...-08-10-p1.html http://euro.ecom.cmu.edu/program/cou...3/pollstar.htm http://www.lawyeringskills.com/sorki.../appbrief.html http://www.ipcounselors.com/20000508.htm#a2 http://gigalaw.com/library/ticketmas...000-03-27.html http://www.linksandlaw.com/linkingcases-deeplinks.htm *All trademarks belong to their respective companies. |
![]() |
|
| Thread Tools | |
| Display Modes | |
|
|
|
WebProWorld |
Advertise |
Contact Us |
About |
Forum Rules |
MVP's |
Archive |
Newsletter Archive |
Top |
WebProNews
WebProWorld is an iEntry, Inc. ® site - © 2009 All Rights Reserved Privacy Policy and Legal iEntry, Inc. 2549 Richmond Rd. Lexington KY, 40509 |