How to allow other website updating our information

[Corobori]

I am programming a website for real estate's agents. In this website each agent (about 30) can publish the property they are managing from a private section of this common website.
Of course some of those agents have already a website on their own. I have been asked to find out a way so they don't need to reenter each property on this common website. That sounds fair enough.
I want to find a rather easy way to update our database. As you can imagine most of those website have their own way of storing the information, some are in PHP other in ASP using MySql, Access or other programming language or db. The main system, the one I am writing, is in ASP.NET.
Any idea on how to achieve this ?

[wige]

One way would be with XML. Come up with an XML format for listings (or find an existing industry-specific standard) and have the agents send you the URL. Then you would just need to write a program that can parse the file on a set schedule, or even come up with a ping URL that agents can use to tell you when the feed has updates.

[DaveSawers]

Or use RSS in a similar way to what Wige suggested.

Whatever you decide to use, your agents will have to change their websites to publish the necessary information and you will doubtless get push back on that.

Here in Canada there is a national listing service (mls.ca) where all agents put their listings. When they add listings to their own sites, they simply query the national database to get the data and present it any way they like.

[niggles]

XML sounds the best way to go. You'll need to write a little "translator" program for each Agent so you can get the data to your site in a consistent format. Doesn't sound like a fun task, but definitely doable. I'm currently setting up a similar thing - but am leaving that part until last

Cheers,
Nigel

[a53mp]

XML or RSS..or write a script that will scrape the other sites.

Either way it's not very applicable.. you will have to modify your site coding, and you will have to have to possibly hundreds of other users pay someone to convert their site to output xml, and in a specific format to your standards.


Won't happen.

[e-dvertising]

xml - standard,
in germany a standard-xml-scheme for real-eastes (OpenImmo: Startseite) has been set up in 2001, we use this standard to import with cronjobs from data-input from different agents on an austrian real-estate-platform.

getting the updates you have different possibilities:
* delete agents objects and import complete set of objects
* using a "status"-field to remove an re-activate objects

for the image-import there ar at least to ways:
* having the image-information included in an import/zip-file or
* having online the image paths include and download from agents websites to yours

maybe the info and the standard is a useful approach for not having "translate" each agents data ...

[OpticBurst.com]

Originally Posted by Corobori

I am programming a website for real estate's agents. In this website each agent (about 30) can publish the property they are managing from a private section of this common website.
Of course some of those agents have already a website on their own. I have been asked to find out a way so they don't need to reenter each property on this common website. That sounds fair enough.
I want to find a rather easy way to update our database. As you can imagine most of those website have their own way of storing the information, some are in PHP other in ASP using MySql, Access or other programming language or db. The main system, the one I am writing, is in ASP.NET.
Any idea on how to achieve this ?

Very informative thread. I think from my own opinion XML would be the best choice. Now the hardest part is putting this all together to work for you correctly. Can be done I am sure, just with an abundance of time!

[seopo]

This is exactly what the company I work for does. If you have not much time but some budget contact me offlist and I can advise you on costs.

[a53mp]

A better way would be to offer them a service so that they would put all of their listings into YOUR site, and from there it will output xml, rss, csv, etc.. so that they can use that for THEIR site.

This will do a few things..

1. make users go to your site a lot more
2. provide a service that they may not have
3. provide an easy to use interface to add/upload their properties into a database (not manually editing code)
4. give other features down the road like google map integration
5. provide an easy and flexible method for users to download their properties via xml, mysql dump, excel, csv, text, etc
6. and most importantly, charge them to make money.

[Corobori]

Budget is low and time is very tight !

wige
It's sounds good to me.

a53mp
I don't want to "scrap" the other site: this would mean a lot of work and sometimes impossible because of the websites are in Flash.
Update: I am not the owner of the website we're just building it so we don't need to worry about the business side of it.

Nigel
The translation part of some coding is worrying me I wish the other side would take care of this. Again a matter of time and money.

DaveSawers
Sure the agents will have to make changes to their coding and they might argue on this but I reckon they would be rather doing this than entering data twice.

e-dvertising
Your 2nd option about image handling sounds good, I also could "import" the image too. The 1st one would be better but it might asking too much for the other side to send accros the images.

[a53mp]

Good luck trying to convince them!

[jimkelly7777]

If you need a quick fix, maybe you can use an iframe. XML, of course, is the long term solution to this.

[Keimos]

Hi Corobori,

The only way to update your database is to give access to those that need it. You would need to give restricted priviledges to the people to do it so that they could update from within the admin side or of have the necessary priviledges to add content. XML and iframe will not update your database, users need to be able to access the site admin content directly or indirectly so that that content is then added to your database.

If you have a remote access facility then that may be an option?

keimos

[Corobori]

Originally Posted by Keimos

The only way to update your database is to give access to those that need it. You would need to give restricted priviledges to the people to do it so that they could update from within the admin side or of have the necessary priviledges to add content. XML and iframe will not update your database, users need to be able to access the site admin content directly or indirectly so that that content is then added to your database.

Perhaps I didn't make myself understood: actually all agents can already update the db from the admin section, the thing is some of them do already have their own db and admin system and of course their own website. That's for them that I am looking a way to automate as much as possible this task. XML won't update my db ? I wouldn't bet on it, I guess I could parse the XML.

Anyway I'll keep you guys updated on this topic and will definitely read your comments.

[wige]

What you might be able to do is convince the agents to offer the XML feeds as a service to their customers. Develop a windows sidebar/google desktop/iGoogle/Yahoo 360 etc widget (or all of them) that these agents can offer to their clients that will let clients see listings from that broker updated almost live. In exchange, you incorporate those listings into your site database. This gives an added value for the extra work the agents would need to do to make the data file exportable as XML.

You could also check with other aggregation sites to find out if they have an XML format that they use to accept data from agents, and match that existing format. This would reduce the workload of agents, allowing them to upload to multiple aggregators using a single feed.

[seopo]

There's two options:

1) Allow agents to push data to your portal. This would involve you a) creating a function to process either an XML file or text file b) receiving this file would either require them having ftp access or you providing some sort of webservice ideally SOAP.

2) Get data from the agents. Again this would either involve you downloading data from them invariably it wont' be in the same format so you would need to create a user agent or robot to visit these pages and extract out the key data. Very tricky to get right.

Essentially option one is the only one open to you but as it sounds like there is no industry standard for sending property data you could be the first and this can give you quite a bit of power.

The other gotcha you will need to look out for is the transfer of images as this can mount up to quite a lot of data transfer.

You may also like to keep an eye on this thread here as it is purely for property professionals doing just what you are talking about. RightMove BLM V3 - Juicy Red Apple

[imvain2]

Originally Posted by wige

What you might be able to do is convince the agents to offer the XML feeds as a service to their customers. Develop a windows sidebar/google desktop/iGoogle/Yahoo 360 etc widget (or all of them) that these agents can offer to their clients that will let clients see listings from that broker updated almost live. In exchange, you incorporate those listings into your site database. This gives an added value for the extra work the agents would need to do to make the data file exportable as XML.

You could also check with other aggregation sites to find out if they have an XML format that they use to accept data from agents, and match that existing format. This would reduce the workload of agents, allowing them to upload to multiple aggregators using a single feed.

Another idea that Wige made me think of, is a flash "widget" that each agent can add to their website. This flash widget would be hosted at your website. The flash widget would have all of the typical features: Search Homes, View Information, etc.. Then each agent would just have to maintain the listings at your website.

Then you can even add additional features, like auto submission to Trulia, Google Base etc..

Although, one draw back is the search engines won't list each of the properties.

[BluePlanet]

It already has been done. OpenRealty. There are standard formats already accepted and in use for exchanging real estate data. You really should do your homework first unless you just like writing code and billing clients for stuff that is otherwise free. There are all sorts of add-ons, both free and paid that will do everything a real estate site would need.

[texxs]

If anyone here EVER doubted my intellectual superiority, be prepared to willfully stand in line and kiss my lilly white ass:

Come up with an XML format for listings (or find an existing industry-specific standard) and have the agents send you the URL

Seriously you think real estate agents are going to hand write some XML's every week? Maybe you think their also programmers and they'll re-write their web applications to send you some xmls.

Get real estate agents to use SOAP programming

Come on man. all of you here. Listen to the smart guy in the room (me). There only one feasible way to get this done. Make a web app that allows them to put their site and their domain "inside" it. Then everything is under one roof and only needs inputted once. the drawback, websites have to be moved once.

Yes, you may kiss my ass now, morons.

Using dotnetnuke if you really must use MS technology or Dolphin or Drupal, etc. if you actually know what you are doing.


Open Realty won't do it, dude you guys are usually so smart here are acting SOOOOO stupid today.

A flash "widget" won't do it. I could on for days about why that won't work.

Did I seriously see someone say to use SOAP? come on! what's wrong with you!

Direct Database access? come on! I'm going to stop reading this.

What is your clients name and contact info please. I'll hook them up, you should just stand aside.

And someone please slap a53mp for his lil evilness there.

[Peopleunit]

ODBC - Open Database Connectivity

It won't be easy, and might not be worth the effort. On the other hand, I imagine others have been approached with the same problem. Maybe some existing open source projects, if not an outright solution, could get you started in the right direction.

That Open Realty looks interesting. See what add-ons are available for it.

[BluePlanet]

I suggested Open Realty earlier in the thread. There is also another open source realty program although the name escapes me. There must also be commercial programs available.

Personally I do not understand why people want to re-invent the wheel when they have not even tested the wheels available and have neither the skills, talent, time or the money to reinvent it anyway. This is all just a exercise in wasted time for all involved.