How to avoid a dynamic e-commerce site

[wmrobwl]

Hey everyone! We currently use FrontPage for web development of our e-commerce site. We have a custom made shopping cart that integrates with our database. The problem is that we are hand making each and every individual page on our site and we have over 80,000 SKUs in our database and we have over 5,000 pages on our site. We need to figure out a better way of updating and maintaining our site.

I've been researching a way to avoid using dynamic pages. I have many reasons why I don't want to use dynamic pages and I won't even list them here because I don't want anyone to preach about how well they work

This is about the best thing I found so far: internet-soft .com/ dataques.htm.

Has anyone ever used this program or anything similar to it? My problem with it is that I am having an initialization error in Win XP when I try to run it as a limited user. I even tried playing with permissions, but that didn't really help. I can't seem to get in touch with anyone at the company either, which doesn't really motivate me to spend 50 bucks to register the software.

My other issue is that I dont know how I would handle a page that has multiple item numbers on it. It uses a template type of system that kind of does a crude find and replace, but I don't know how to handle multiple item numbers on the same page.

Can anyone please help??

Thanks!

[dumarest]

I've used a product in the past that takes the database and creates webpages.

You use the field names in the HTML of your template which you supply to the software and it will create the pages - to fix ALL your pages - redo the template and then recreate all your pages - load them up - and Bob's your uncle.

If you cannot find something like this - I'll go root around and find the one I used in the past.

D

[kylegetson]

to me, thousands of SKUs and thousands of pages sounds way worse on a front-page system then any dynamic system out there that I've ever used. Ive heard lots and lots of reasons not to use a dynamic system, but I tend to ignore them once you get over a few hundred items to manage.

I would suggest migrating to something simpler to maintain. Ive worked with alot of shopping carts with thousands of products, and I wouldnt have been able to handle it if I had to deal with pages individually.

I know, this isnt what you want to hear, but I would take a serious look at Magento Commerce. Ive built 5 sites using it, and Im a big fan.

[amnesia]

You're wanting to make a static e-commerce site, and you want to do it in Front Page. I can't wait to tell the guys at lunch about this, once I am able to pick myself off the floor from laughing so hard.

This isn't 1995 man. Fire your web developer for even suggesting such retarded nonsense and get some real developers involved. If you have 80,000 products you need a real CMS.

[wmrobwl]

Dumarest, if you know the name of the program you used in the past I would love to know. It sounds very similar to the program that I already found.

How would your program handle multiple item numbers being on the same page? If you didn't know how many items would be on the page. If I knew how many items would be on the page I could do something like item1, item2, item3, etc. but I really don't know how I could handle having multiple item numbers without knowing exactly how many items would be on the page.

[wmrobwl]

Originally Posted by kylegetson

to me, thousands of SKUs and thousands of pages sounds way worse on a front-page system then any dynamic system out there that I've ever used. Ive heard lots and lots of reasons not to use a dynamic system, but I tend to ignore them once you get over a few hundred items to manage.

I would suggest migrating to something simpler to maintain. Ive worked with alot of shopping carts with thousands of products, and I wouldnt have been able to handle it if I had to deal with pages individually.

I know, this isnt what you want to hear, but I would take a serious look at Magento Commerce. Ive built 5 sites using it, and Im a big fan.

The issue with using a cookie cutter shopping cart is that each of our product categories or manufacturers use a slightly different layout.

Originally Posted by amnesia

You're wanting to make a static e-commerce site, and you want to do it in Front Page. I can't wait to tell the guys at lunch about this, once I am able to pick myself off the floor from laughing so hard.

This isn't 1995 man. Fire your web developer for even suggesting such retarded nonsense and get some real developers involved. If you have 80,000 products you need a real CMS.

NO, please re-read the post. I am not WANTING to make a static e-commerce site in FrontPage. This is WHAT WE ALREADY HAVE. This is how we started the site with it's very first product back in the 90's and it just so happened to have grown into what it is now.

[texxs]

OMG! 80,000 sku's, Win XP, FrontPage, Static pages????

The cost of updating prices alone has got to be astronomical!
You 110% need a data driven site, don't worry SE's are fine with them, if you do it right.

It sounds like you're in way over your head.

Disaster is looming.

You're the IT Director, then direct a big overhaul.

Where are you?

If you are anywhere within 24 hours drive of me (Florida), I'll come right away and help you save you're job.

I know it would be easy to dismiss this post as someone who is just "hating" on you, but I'm really sending this with love: THIS IS YOUR WAKE UP CALL. Don't sleep through it Dude or someone is just going to come in and take your spot.

[kylegetson]

Originally Posted by wmrobwl

The issue with using a cookie cutter shopping cart is that each of our product categories or manufacturers use a slightly different layout.

please look at Magento before calling it a 'cookie cutter' shopping cart. its the most advanced and indepth system Ive ever worked with. See their showcase: Magento - Showcase - eCommerce Software for Growth
personally, I think its tough to call that it 'cookie cutter' cart.

[Jon-WebsiteCM]

Originally Posted by amnesia

You're wanting to make a static e-commerce site, and you want to do it in Front Page. I can't wait to tell the guys at lunch about this, once I am able to pick myself off the floor from laughing so hard.

This isn't 1995 man. Fire your web developer for even suggesting such retarded nonsense and get some real developers involved. If you have 80,000 products you need a real CMS.

This pretty much sums it up. There's no advantage to an html site that can't be overcome on a dynamic site. This is like building a Walmart sized retail building from sticks and leaves.

[Jon-WebsiteCM]

Originally Posted by wmrobwl

The issue with using a cookie cutter shopping cart is that each of our product categories or manufacturers use a slightly different layout.

You can modify any open-source dynamic system so you can have multiple templates (if it isn't included in the product by default). In fact we provide such modules for x-cart.

[kylegetson]

Originally Posted by Jon-WebsiteCM

This pretty much sums it up. There's no advantage to an html site that can't be overcome on a dynamic site. This is like building a Walmart sized retail building from sticks and leaves.

ha ha ha. I agree.

[texxs]

The issue with using a cookie cutter shopping cart is that each product category or manufacturer uses a slightly different layout.

Ya, a CMS can handle that.

A web site without dynamic (data driven, server generated)) pages is a static site.

I'm NOT trying to make fun of you, but it really sounds like you're over your head.

My offer for legit help is on the table, or hire someone you know. Scratch that, you better hire someone you haven't already talked to about this, unless they told you were going about this the wrong way.

[astro]

Hmmmm

Front page.

Harumph!

Frontpage does not like to work with anything non Microsoft. The Borg collective.....sorry guys at microsoft will all disagree, but I doubt any experienced webbies will.

Besides Front page templates are so predictable and so easily recognised. It is like running your Internet business from a public phone box. It creates a site like a high street flower shop keeper has built by his 14 year old son, who knows everything about computers. Ever looked at a Front page page with an HTML web editor? Amazing!

Time for a rethink on design software methinks.

/astro

[wmrobwl]

Why am I getting so much heat about this? Had we known back when we first started the site that we would have over 5,000 pages and over 80,000 SKUs we probably wouldn't have gone this route. In fact...why don't you try to think about what options were really available in the late 90's. The only pre-fab cart that was available at the time was MIVA and that totally SUCKED.

I'm not averse to a complete overhaul. In fact, that's what I've been planning for about the past two years. I just never really figured out a viable solution. The way I see it, you can have the server generate pages for you automatically ie.: dynamic pages or you can create static pages. Why can't the pages be automatically generated for you on your local computer the way that dynamic pages are generated on the server and then just uploaded to the server? Why should a page be regenerated by the server everytime someone requests it? It's not like each of our 5,000 pages changes every single day. We might have a page that doesnt need to be updated for 3+ years.

Yes I do believe NJ is just about 24 hours or less of a drive away from FL.

[Mark.M]

Hello,

Sorry, I can't offer any info about the software mentioned..

If you have that many products would you consider using a database to build for the static pages on-the_fly?

A Database:
1) loop through products
2) add product information into an html template
3) copy the temple as each page name to your site structure

Do this on your server, and you'd have what you need while still being manageable.

With so many items, you might want to break the products up in to sections or groups. That way you can easily build a group of pages when you change a few instead of rebuilding the entire site.

Without being disrespectful to your original post, how do you know for sure if a site is dynamically generated or static?

Miva was not the problem, the way the application was created had issues.

You could run the same idea on your hard drive, then FTP all of your updates.. but then I'd ask why add the extra step?


Hope you post any new findings..

[Orion]

Late 90's we were making dynamic php / mySQL carts .. yes miva sucked big time

the nice thing is about 5 years ago or more we were able to add a couple lines to the .htaccess and create static URLs (not pages) for those same carts (you can change any dynamic URL to a static one petty easily now)...

there are many static carts (which is what you want to search for) static cart with database.. so the data is in there but the php (or asp) creates static pages for you..

there are many solutions out there like that.. The database part gives you a simple back end to be able to add update edit and delete the products...

one downfall to such a system is every time you make a change you then have to update the site and with 5000 pages that can be HUGE server load and time for each update. Not that they can't work for your site...

Don't want to give any heat, but explain a bit.. they style of cart you are looking for is actually a shortcut that a programmer takes on the way to creating a really viable solution. They're awesome little projects. I used to get my students to make similar carts, but they are often poorly coded and rarely secure solutions.

It might help if you were able to list a number of the reasons you feel you require the static HTML pages..

Any dynamic solution (take the three most popular wordpress, joomla, dupral) allow you to make multiple templates (very easily) to handle multiple styles of product display... Also they handle multiple sku's per page. They also give you static URLs, auto updates on sitemaps.

Whichever solution you do end up going with, I'd really like to know which one you go with and the reasons.. It may be of great help down the road for others...

thanks,

[wmrobwl]

OK, I'm sorry let me repost with some additional informaton. I'm open to all suggestions.

1. Obvious SEO is an important consideration
2. How can we transition from a live production site to a new setup?
3. We run our own servers on multiple T1 lines
4. Performance aspects are also an important consideration
5. I think when a developer is working on the site it would be preferable to work at native speeds rather than to use a web browser to access an interface on the web server.
6. I think it would be beneficial to do all of the work on a local machine and then upload the files to the server at a later time.
7. We use FrontPage solely as an HTML editor. We don't use their publishing or forms mechanisms. We upload to our servers using WinSCP.
8. We have an existing DB that is integrated with our web site. Hence the reason I don't think it's necessary to start from scratch with a brand new CMS or shopping cart.

Is there a way that we can automatically generate static pages, locally, the way that a CMS or shopping cart automatically generates dynamic pages on the server?

Thanks for your suggestions!

[Kevnn]

Well, I won't argue with someone who "knows better", but if you won't use a page that is created dynamically at the users request, why not write a script that builds all the pages for you so you can just run it when you add new products to the system.

Have your script access your db and pull product data out, then it can create and save all the pages to whatever folder you want and presto, you have 54000 new pages to index.

Have fun!

[Jon-WebsiteCM]

The route you went is understandable in 1994, but it seems unheard of to be using a 15-year-old solution given the evolution of the internet since then (I'm sure your not still running windows 3.1.11 on a 186 and a 1200 baud modem) which is why your taking a lot of heat. To be manually generating the amount of pages you are are manually in this day and age has lost you insane amounts of revenue, which is why your 'getting so much heat.'

Like said above, at very minimum its pretty simple to write a script that uses a designed template, pulls from the database to insert into the template, and outputs an .html file.

Are you manually processing orders too?

[writerguy]

Okay, you've mentioned the idea of creating the pages on your computer and uploading them to your server.

The only software I'm familiar with that might do something like this is a package called WebMerge. (If you go to the ABW affiliate marketing forums, there's a WebMerge subforum there.)

The WebMerge software may even be something spoken of in this thread already. I haven't read all this thread carefully, but your comment about sticking with a static software that will build pages on your computer then allow you to upload them to a server makes me think of WebMerge.

I haven't used it for a couple of years, and I don't have any idea what the current cost of the software is. You can find it via their subforum at ABW or probably just Google WebMerge. I think their current version is 2.4 or 2.5. Maybe it's what you're looking for.

Gary

[wmrobwl]

Originally Posted by Jon-WebsiteCM

The route you went is understandable in 1994, but it seems unheard of to be using a 15-year-old solution given the evolution of the internet since then (I'm sure your not still running windows 3.1.11 on a 186 and a 1200 baud modem) which is why your taking a lot of heat. To be manually generating the amount of pages you are are manually in this day and age has lost you insane amounts of revenue, which is why your 'getting so much heat.'

Like said above, at very minimum its pretty simple to write a script that uses a designed template, pulls from the database to insert into the template, and outputs an .html file.

Are you manually processing orders too?

Maybe it's just that FrontPage gives people a certain impression. I'm not using their cheesy templates or any of their other stuff. Has anyone ever used FP as strictly an HTML editor? If I had said Dreamweaver or that we hand code HTML in Notepad I don't think I would be getting this reaction. I personally like FP as an HTML editor because it's very simple. You have WISYWIG mode and pure HTML mode. No big bells and whistles. The only thing purer than FP would be to hand code directly in Notepad. Anyway, another reason I like static pages is because you can customize any page just by editing it's HTML. No, we don't manually process our orders...we hand write them and then send snail mail to our vendors to order the products lol j/k

Could anyone recommend a script that could pull from the DB and automatically generate multiple HTML files? I'm also open to any other suggestions. I hope I didn't give the impression that I'm dead set on any particular direction.

[mikefromvt]

We have a system, built with Open Source Components:

Python
VIM
SED
AWK

And publish thousands of static web pages at a pass from tab delimited .CSV files created with OpenOffice. I would be happy to discuss this system with you.

[Jon-WebsiteCM]

Lol - at least you have a sense of humor about it. Indeed frontpage has a stigma to it that hasn't helped, but for me personally running e-commerce stores, the thought of generating html for 80,000 items is mind-boggling.

What type of database are you using?

[wmrobwl]

We're using Filemaker as our DB / CMS. We can export in various file formats, like xml, csv, tab, xls, etc.

[Kevnn]

After reading some of the other posts, maybe I should be a bit more specific for you.

While I haven't done what you are asking for the same reason you want it, I have written asp pages or vbs files in the past that generated multiple static pages for products in my clients database (they had approx 94k products). My solution requested information from the user as to which product(s) they wanted static pages for (by sku or category), then created and saved each page to a folder on an attached storage array that the requesting user had access to. It was part of a CD I created for them. The CD showed "special" products like School Supplies. It ran a flash file that just opened html pages for the products it was pitching and the static page creation was so they could create new cd's each year with changing products. We did a couple of different themes and they created their own CD's from that point forward.

If you have over 50k products, then you will probably want a couple of versions. One for product pages and one to list products by category at least.

It can be written as a .vbs file(s) and you could run it on your personal computer to save processor time from your server (with the exception of the db connection) if you like. It's very easy to do. And you can save a little money/time if you already have the html template(s) created.

[mikefromvt]

Hi WMROBWL:

Yes, as long as you can output a tab delimited .csv file from FileMaker, then our system would work very well. We have a client with over 6.5K static pages on his site and have been maintaining same since Oceober 2005. We would

1) Create a template from your FrontPage web page
2) Customize the template for each of your product categories with our 'buildtemplate' SED script
3) Format our datafile with our 'awkscr' AWK script
4) Process the static pages with the Walk plugin for VIM

We have run as many as 404 SKUs at one pass and produced the pages in less than 3 minutes per run.

Our client enjoys SERP on either page 1 or 2 for 17 of 31 Keywords tracked with Advanced Web Rankings software.

I would be happy to elaborate if you have interest in hearing further.

[1centwiz]

From my limited comprehension of what you want, first question is what format is your existing DB in? (I have a CFM DB site that was built to be used on a site created with tables using Frontpage and converted it to CSS last year) Second question, do you have CSS already established on this site? If not, then that might be the first place to start.

With Frontpage(you may want to upgrade to Expression Web 2.0) you will be able to create static Dynamic Web Templates(DWT). By using Div tags instead of the tables, tr's and td's and with corresponding Cascading Style Sheets(CSS) that replace the attributes shown on the pages (font style etc.) you can then include an internal div id 'view page' that includes a number of options for a number of items. Create a javascript(we use a CFM 'if' tag) so the user can choose 10/20/50/100(for example you can set your own min and max) so that the 'view page' div id reloads with more items. You would only need to build a few CSS pages(depending on your choices) to accommodate this along with your Main DWT and your 'view page' DWT's.

If you do it correctly, then you will have a static page with the DB page inside of it which re-sizes as needed. For visual, imagine a C as your main DWT template, and inside the C is where the DB 'view page' div id DWT resides and changes accordingly. Make sure you use a footer on your main DWT that always resides at the bottom and has a clear:both div so that your pages can expand.

By using a CSS Meta tag for each category page, you can have separate meta tags for SEO and link them to the appropriate pages. Again, those CSS meta pages can be changed when needed according to the serps when ever you need to.

Basically, you then have an easy to build fluid website with columns (see AListApartdotcom/topics/design/layout/ for some extra help) with a number of page specific CSS pages, the Inventory entry of your DB, and your DWT pages. All of which once created, are easily adjustable when necessary with an easy one touch for all pages on your site.

If I'm off base here, then I guess I've misunderstood with the rest of them exactly what you want and need. If for some reason I've not explained myself well, hey it happens, then you can PM me at my user name @gmail.com and I will gladly share the link with the site I manage that uses this format with the ColdFusion Database(CFM).

I'm just a little guy among the big fishes here!

[wmrobwl]

Thank you everyone for your suggestions and responses. It really gives me a lot to think about. I'm going to check into Magento Commerce and some other shopping cart systems even though we already have a shopping cart/database integrated. I will also check into Web Merge. Texxs, Jon-WebsiteCM, Kevnn, and Mikefromvt I am sending you guys PM's.

As soon as I decide on a solution I will post to let everyone know why I chose it and how well it works out. In the meantime, I would also be open to any new suggestions as well.

Thanks!

[Kevnn]

One thing to understand is that any type of dynamically built page means that all pages will look exactly the same with the exception of differences in the data itself.

The data IS the key. I've worked with lots of manufacturers and distributors databases. The most recent were United Stationers and SP Richards. They are large e-commerce oriented databases and each has it's own shortcomings as to the end product. It's not too terribly difficult to get all you need from their regular datasets, but if you pay for the "enhanced" version, you have more to work with. More actual text, more complete category structure, etc. If your current 5000 pages were hand created, then chances are they use data that wasn't in the db in at least some cases. That would be lost, unless you can take the energies you've been directing at page upkeep and creation and put it into building out your database. That is a big job in itself with 54k changing products so you will want limit that type of thing to a minimum or use other tools to collect data for other places to fill in whatever blanks your data set has.

[wige]

Just to open with some background, I am a web programmer from the old school so to speak; I have been developing sites both for personal use and for business since the early nineties, since before there even was such a thing as PHP or ASP. Yes, I wrote entire sites in Perl. I have also built entire CMS from the ground up. You can see a few examples at Philadelphia Phillies Tickets and Eagles Tickets - Philadelphia Ticket Agency Ticket Warehouse and TicketWarehouse.us - these are the same database, but different CMS systems to display data to the user.

From an external perspective, a dynamic site that is properly designed will appear to be static - users should not be able to tell the difference. However, by sticking with a static approach, I think you will miss out on a lot of opportunities to really utilize the data you have on your products and even integrate data you get from customers.

To address the points you made above...

Originally Posted by wmrobwl

1. Obvious SEO is an important consideration

Search engines can not tell the difference between a static site and a properly designed dynamic site. If you can optimize a static site, you should be able to even more effectively optimize a dynamic site.

2. How can we transition from a live production site to a new setup?

This can be tricky, but it should be possible to have a seamless transition.

3. We run our own servers on multiple T1 lines

Thats fine. You would probably need a custom CMS to properly handle and distribute the workload. Are your workstations on the same network or VPN with the servers, or at least at the same campus?

4. Performance aspects are also an important consideration

Dynamic sites have distinct advantages over static sites because they allow you to distribute the workload over multiple servers. However, server overhead for all but the most heavily trafficked sites is often minimal. Most of the processing is done natively and so users should not have a negative experience due to the use of dynamic components.

5. I think when a developer is working on the site it would be preferable to work at native speeds rather than to use a web browser to access an interface on the web server.

In most cases, delays when working on the server through a web interface are due to latency - network slowdowns rather than server capability. If the developers are at the same location as the server, there are several options to allow them to work natively, either using a custom desktop application (C++, Java, etc. depending on the database) or an AJAX powered web interface.

6. I think it would be beneficial to do all of the work on a local machine and then upload the files to the server at a later time.

I don't believe this is always the case. In fact, for certain types of data, this can be illegal. Yes, test data should be seperate, but only developing locally then uploading can cause unexpected issues unless every development environment is identical.

7. We use FrontPage solely as an HTML editor. We don't use their publishing or forms mechanisms. We upload to our servers using WinSCP.

There are better HTML editors. Free ones too.

8. We have an existing DB that is integrated with our web site. Hence the reason I don't think it's necessary to start from scratch with a brand new CMS or shopping cart.

Well, in this case, your site is already dynamic. However, depending on the age, the benefits of switching to something that is a better fit for your data needs might outweigh the costs. By a lot.

Is there a way that we can automatically generate static pages, locally, the way that a CMS or shopping cart automatically generates dynamic pages on the server?

Again being a programmer, I tend to lean toward custom solutions. I don't specifically know of anything that will do this automatically, or particularly believe this is the best option for your needs, but it would probably be reasonably easy to develop such a solution if one can't be found.

[subsystems]

Sounds to me like a bit of reading about server content caching is in order. This assumes you have a dedicated server. Mostly due to RAM needed.

One basic idea is the site is dynamically generated as pages are requested. Then the HTML is cached in the server's RAM. The database isn't accessed for that webpage for subsequent requests. Given enough server RAM it is much faster than even static pages. Figure based on the number of web pages cached in RAM and their average size you can get an idea how much RAM you need for server cache. With 2GB of server RAM we had 2000 products pages, all the search results, drill down sub-categories each showing the number of products based on previously chosen categories etc. Like newegg.com and many others. e.g. Newegg.com - Memory, DDR Memory, SDRAM Memory, Computer Memory, Computer RAM, DDR RAM, Memory Stick, Memory Upgrades

Also look into partial caching of page content. Lets say your header, footer and navigation remains constant yet many areas change. Then cache the static content and insert it into the page template on the fly. Also think about caching the advanced search dropdowns like the ones on the page listed above.

Both Apache and IIS have means for caching content. I know IIS can flush the cache based on whether or not an item in the database changes that was used to generate the content (.NET). So if you change a price or a description, the cache for the related cached content is flushed and reloaded and cached dynamically when requested. Lets say you wanted to move a product from one category to another. All the search results would change, so having the cache flush automatically based on database changes is pretty useful.

Now imagine a search that looks up in all products that were also purchased with the currently shown item. Query only active products group and count them then sort that list decending and display the top 3 or 5 on a product's details page. That search is no small feat and can really load the server. So a trick I have used is to pre-cache some query results. Setup a page that steps through the product pages etc caches certain things you think visitors will need or that take too long to query on the fly.

You can cache at different levels. Direct caching of query results to direct caching of HTML pages or HTML snippets. One of my favorite methods is to use XML as the go-between to the database. Then cache the XML. The advantage is you have isolated your database structure from your page script. You sent XML formatted requests to a processing script that takes the request sees if the results are already in XML cache and if not it sends an SQL request to the database and the results are cached as XML and returned to the page script. As far as the page is concerned it is dealing only with XML. The database is only dealing with XML. The middle-tier is handling the conversion and the caching. One last thing, when the server needs to be rebooted, the cache is cleared so the site will be slowest then. You can speed it up a bit by having the caching/processing script write the XML data to disk as it is requested from the database. If the item is not in cache, look for the XML file first before going to the database. By doing this even a server reboot may not trigger a database query. So the site stays fairly fast at all times. You still get the advantages of handling your product data in traditional SQL. No more SQL calls from a search page that hang the server in an infinite loop. You have multiple validation layers to help protect against that.

The good part in all of this is you have only 1 product page and 1 search page to get going. Most of your focus will be on the XML caching/processing script that talks to the database. Once you get it returning and caching the data how you want it the product and search pages fall into place quickly.

Also looking into URL rewriting so the browser URLs for all the products are friendly URLs. If you rewrite URLs into querystring parameters like I did, VALIDATE, VALIDATE, VALIDATE. Strip out any URL data that doesn't belong there. Crawlers are bad about throwing extra junk in the requests.

[JoomlaBear]

Definitely look at Magento - very, very impressed with this software.

Check out Magento's CEO giving a bit of background and listing some of the big players that are adopting it, worth watching. You can see the video bottom right of Magento's home page - I can't post a link because I'm relatively new here....

I do agree with the sentiments of most saying you should go dynamic - It's not such a headache really, and you don't need to worry about pages being generated constantly - that's what server side caching is all about. Rebuildign the pages at intervals set by you - 10 mins, 10 hours, 10 days, It's all really quite straightforward.

[Peter (IMC)]

Originally Posted by wmrobwl

Hey everyone! We currently use FrontPage for web development of our e-commerce site. We have a custom made shopping cart that integrates with our database. The problem is that we are hand making each and every individual page on our site and we have over 80,000 SKUs in our database and we have over 5,000 pages on our site. We need to figure out a better way of updating and maintaining our site.

I've been researching a way to avoid using dynamic pages. I have many reasons why I don't want to use dynamic pages and I won't even list them here because I don't want anyone to preach about how well they work

This is about the best thing I found so far: internet-soft .com/ dataques.htm.

Has anyone ever used this program or anything similar to it? My problem with it is that I am having an initialization error in Win XP when I try to run it as a limited user. I even tried playing with permissions, but that didn't really help. I can't seem to get in touch with anyone at the company either, which doesn't really motivate me to spend 50 bucks to register the software.

My other issue is that I dont know how I would handle a page that has multiple item numbers on it. It uses a template type of system that kind of does a crude find and replace, but I don't know how to handle multiple item numbers on the same page.

Can anyone please help??

Thanks!

I'd go with Merchant Metrix Ecommerce Shopping Carts & Search Engine Marketing by Merchant Metrix

Here's 2 sites that uses the Merchant Metrix software: Wholesale Keychains | Key Ring Chain
Harmonica Store Blues harmonicas Hohner Suzuki Lee Oskar

Visit those sites and see if you can tell that they´re 100 % dynamic. Not a single hand made page in those sites. You'll have a hard time figuring that out, even their check out pages use static URL's.

[case510]

Originally Posted by wmrobwl

Hey everyone! We currently use FrontPage for web development of our e-commerce site. We have a custom made shopping cart that integrates with our database. The problem is that we are hand making each and every individual page on our site and we have over 80,000 SKUs in our ......

Try a shopping cart that can either utilize mode_rewrite (for example Magento - Home - eCommerce Software for Growth - free of charge) or generate static pages .html (for example x-cart.com - commercial license)

[yorkshirewoman]

Hi there

I know you're in the USA but you might like to look at Actinic.
actinic dot co dot uk
There are various products available - Actinic Catalogue creates static pages from a database. I haven't used it myself but know of a number of companies using it very happily.

Hope you find a good solution!

Cheers.

[Clarrie]

Originally Posted by wmrobwl

OK, I'm sorry let me repost with some additional informaton. I'm open to all suggestions.

1. Obvious SEO is an important consideration
2. How can we transition from a live production site to a new setup?
3. We run our own servers on multiple T1 lines
4. Performance aspects are also an important consideration
5. I think when a developer is working on the site it would be preferable to work at native speeds rather than to use a web browser to access an interface on the web server.
6. I think it would be beneficial to do all of the work on a local machine and then upload the files to the server at a later time.
7. We use FrontPage solely as an HTML editor. We don't use their publishing or forms mechanisms. We upload to our servers using WinSCP.
8. We have an existing DB that is integrated with our web site. Hence the reason I don't think it's necessary to start from scratch with a brand new CMS or shopping cart.

Is there a way that we can automatically generate static pages, locally, the way that a CMS or shopping cart automatically generates dynamic pages on the server?

Thanks for your suggestions!

I've not personally had experience of it but I think the Actinic cart does something like this - the software runs on a local machine and then publishes the product pages as html pages.

[B0B]

Wow that hit a hot button. An 80,000 item static catalog!

I wrote a program, about 15 years ago, that spit out static catalogs, it was really cool and better than most out there today. But that was 15 years ago, things have changed. Modern catalogs have summary listings based searches and sorts, making it easy to find what you want. Without some kind of smarts on the server you can't do it. Related items, like batteries and gloves for that garden hoe, etc. are much harder to accomplish and require a lot more updating and uploading on static pages. My advice is if you plan on having more than a few thousand items, use a dynamic catalog.

SEO isn’t a problem as long as you are sane about doing it. AJAX is a harder problem but solvable.

I use FP for much of my work, go ahead and throw stones but it’s easy to use.

[effisk]

Originally Posted by wmrobwl

I have many reasons why I don't want to use dynamic pages and I won't even list them here because I don't want anyone to preach about how well they work

I have many reasons why I won't suggest a system with static pages and I won't even list them because everyone knows them.

Now seriously, do list your reasons, because everyone in this thread is interested. What possible reasons could you find?

Frontpage for an e-commerce website? What a waste of time and money!

Ok, I know why you don't want to use a CMS like magento or oscommerce: You don't know them, and you're scared because it's new stuff to you and you're going to be lost.

Oh, also: don't share your website URL with us, because I'm sure there's a few of us ready to get in touch with your boss about some
dynamic solutions that will save time and money.

As some already said above: switch to dynamic or lose your job, choice is yours.

[PPIS]

Hi WMROBWL,

Here's another vote for Actinic (Actinic Shopping Cart Software Ecommerce Solutions. E-Commerce UK. Sell Online). Yorkshirewoman above mentioned it earlier. They do produce a USA version and it seems to cover everything that you are looking for.

You can import your existing database, create as many templates as you like (it integrates directly with Dreamweaver) and it then creates very search engine friendly HTML page before you upload them. Once you have made the initial upload of the new site, any edits are uploaded incrementally by the CMS so you don't then have to upload the whole site again. It also means that you can preview the site locally without having to set it up on a local server first.

I guess that the only drawback is going to be the sheer volume of products that you are selling. Have you considered splitting the product base over several sites and cross linking them internally? Just a thought...

Certainly worth a look!

Cheers,

Duncan

[NJ]

Originally Posted by wmrobwl

I've been researching a way to avoid using dynamic pages. I have many reasons why I don't want to use dynamic pages and I won't even list them here because I don't want anyone to preach about how well they work
Thanks!

I'd like to know why dynamic pages are a problem, too! I'm also wondering if the various responses in this thread don't represent a wide variety of definitions of "dynamic" pages. For example, if your system automatically generates a new page every time you add a new product instead of when a viewer requests the page, isn't that "dynamic?" It's just a different trigger to generate a page; so, you don't have to do the clicking, dragging, and posting yourself!

[wmrobwl]

I guess one of the main reasons I don't want to change over to a shopping cart / CMS / dynamic site is because we already have a great CMS / database / shopping cart setup already. We just need to figure out a way to automatically generate the pages so less manual labor is necessary. We run a lot of open source on our servers and a lot of our applications in the office tend to be custom made (either in house or outsourced). I guess the answer to my question then is to extend the functionality of our CMS / database / shopping cart since it seems like that's the line of thinking that everyone is accustomed to.

My original question was very micro in nature. I know we have a great CMS, a great database, and a great shopping cart. Yes we have to hand code a lot of our pages individually and most of it is copying and pasting, but it's obviously very repetitive. I was looking for a way to automate the one aspect of our system that has obvious flaws. I knew what this could have potentially turned into and it turned into just that. Through your probing some of you are making it seem like we built a site with a FrontPage template and we are using FrontPage because we have no idea that other options exist. I really wish I would have originally said "HTML editor" rather than the brand name "FrontPage". I guess it just has that big of a stigma attached to it. I appreciate all of your good intentions and I will post an update to let you know exactly what we chose to do.

Thanks!

[wmrobwl]

Now you guys have got me thinking Rather than an off the shelf cart I decided to check into some open source options. I found OScommerce and zen-cart. Does anyone have any experience with either cart?

I like some aspects of each of the carts on these sites: jewelrycentral dot com and kneedraggers dot com. Does anyone have any idea what kind of carts they might be running on their sites? I'm sure we can export our DB and import it into the new cart's DB fairly easily.

[Jon-WebsiteCM]

I don't have experience with Zen-Cart. I used osCommerce some time ago and found it terrible so I wrote it off long ago. We specialize in x-cart, the pros and cons of which can be found at Why or Why Not X-Cart? - X-Cart Tutorials - WebsiteCM. Magento as mentioned above is the up-and-comer, and you may also want to checkout cs-cart.com

[gembiz]

We use zen-cart and it is OK. You get all the source so you can mod it if you want to but there are numerous plugins available for such things as credit card handling, seo rewriting and so forth. There are even ways to upload existing products into the database. For 80,000 products the trick is to get the categories / subcategories right.

With ZenCart you need to learn to use the layout boxes and turn off all the ones you do not want. There is even a book on using ZenCart and it is not that expensive.

Download it, install and play with it - it costs you nothing except a bit of time - and then you will soon have an idea as to how it matches up to your requirements. The default colour scheme is awful, but the CSS is easily amended or you can buy in themes and apply them. It will even handle multi-lingual sites.

[texxs]

wmrobwl your being a great sport and opening your mind. good for you you and your company will benefit from it greatly1

don't get discouraged by the plethera of options available to you.

Think of the future when making changes, not just the present.

[neonexus]

I've worked on a number of websites in the past using FrontPage and other Microsoft products. Each became difficult to manage, less cost effective and ultimately too much of a burden to continue down that path.

As comments above have suggested, check out the public use eCommerce solutions. I would strongly recommend Magento eCommerce due to its "out of the box" ability to handle customizations at the category and product levels which you have indicated as an issue.

[wmrobwl]

OK so here's an update...most of you will probably be very disappointed because you were all so excited about converting us over to the dynamic world.

I carefully considered each and every one of your suggestions with equal weight. I chose Web Merge and I chose it for the following reasons:

1. I thought about using a dynamic shopping cart long before I came up with the idea for an HTML page creator and for various reasons could never really prove that it would be a great solution for our situation.

2. Although I am not averse to drastic overhauls, I didn't think it was necessary to tear down a perfectly fine infrastructure when there was only one piece of the puzzle missing. Web Merge perfectly completes that missing piece and it really is an amazing piece of software when you sit down and play with it.

3. Richard from Web Merge was very helpful when I called and is actually the creator of the software.

4. Using Web Merge is equally the same amount of work as having a dynamic shopping cart when adding new products and updating and maintaining existing products [*FOR US*]. However, a dynamic shopping cart would take much longer to implement than using Web Merge in our situation because I would have to make way too many accommodations to get it up and running. We run multiple servers in a cluster and to even incorporate a different shopping cart to run on the cluster would require X amount of work.

I don't feel it's very necessary for me to continue listing my reasons why I chose it because my situation is probably completely different than most other people. Most other people looking at a dynamic shopping cart probably haven't spent as much time and money in an infrastructure that is completely run in-house.

I won't say that we were on the cutting edge of web development, but when you're running a small business with no outside funding you have to keep a watchful eye on your bottom line and you have to wear many hats which means that every area of the business cannot possibly be at peak performance at every second of everyday.

The bottom line is that if anyone is reading this thread and wants to know the best way to go if you're starting a new site or if you're in a simple situation where you outsource your web hosting to a hosting company (or even if you run your own hosting on a single server) and you don't have any shopping cart or CMS system already in place then you should definitely go with one of the shopping cart's mentioned in this thread.

My question was very specific. I wanted to achieve a certain outcome and I believe that I found the solution to that. I was deliberately ambiguous because I knew that the debate between static and dynamic would overshadow my real question. I didn’t foresee that my use of the brand name of my HTML editor would have sparked such a fierce response. I will continue using FrontPage to edit occasional HTML pages, but I think that most people still think of FrontPage as that program that creates those cheesy template sites. When we use it we don’t use themes or any of their built-in features. We use it STRICTLY AS AN HTML EDITOR.

Finally, I would like to thank each and every one of you who contributed ideas. Through the research of each of those ideas I was able to confidently arrive at a viable solution.

[sujancsecuet]

Maybe you need to decide to use any dynamic shopping cart immediately and transfer your all inventory there. If you are concerned about SEO performance, there are lots of shopping cart solution with SEO friendly modules to overcome. So, best is to move soon. In future when you will have more products then it will be more tough for you to manually create pages.

[1centwiz]

Originally Posted by wige

Just to open with some background, I am a web programmer from the old school so to speak; I have been developing sites both for personal use and for business since the early nineties, since before there even was such a thing as PHP or ASP. Yes, I wrote entire sites in Perl. I have also built entire CMS from the ground up. You can see a few examples at Philadelphia Phillies Tickets and Eagles Tickets - Philadelphia Ticket Agency Ticket Warehouse and TicketWarehouse.us - these are the same database, but different CMS systems to display data to the user.

From an external perspective, a dynamic site that is properly designed will appear to be static - users should not be able to tell the difference. However, by sticking with a static approach, I think you will miss out on a lot of opportunities to really utilize the data you have on your products and even integrate data you get from customers.

To address the points you made above...

Search engines can not tell the difference between a static site and a properly designed dynamic site. If you can optimize a static site, you should be able to even more effectively optimize a dynamic site.

This can be tricky, but it should be possible to have a seamless transition.

Thats fine. You would probably need a custom CMS to properly handle and distribute the workload. Are your workstations on the same network or VPN with the servers, or at least at the same campus?

Dynamic sites have distinct advantages over static sites because they allow you to distribute the workload over multiple servers. However, server overhead for all but the most heavily trafficked sites is often minimal. Most of the processing is done natively and so users should not have a negative experience due to the use of dynamic components.

In most cases, delays when working on the server through a web interface are due to latency - network slowdowns rather than server capability. If the developers are at the same location as the server, there are several options to allow them to work natively, either using a custom desktop application (C++, Java, etc. depending on the database) or an AJAX powered web interface.

I don't believe this is always the case. In fact, for certain types of data, this can be illegal. Yes, test data should be seperate, but only developing locally then uploading can cause unexpected issues unless every development environment is identical.

There are better HTML editors. Free ones too.

Well, in this case, your site is already dynamic. However, depending on the age, the benefits of switching to something that is a better fit for your data needs might outweigh the costs. By a lot.


Again being a programmer, I tend to lean toward custom solutions. I don't specifically know of anything that will do this automatically, or particularly believe this is the best option for your needs, but it would probably be reasonably easy to develop such a solution if one can't be found.

AWESOME FEEDBACK! I learned something here... thanks!

[JoomlaBear]

WebMerge looks interesting - I'll keep an eye on that.

If anyone is considering a new ecom venture I can't recommend Magento highly enough though - - Magento is the cutting edge.

Also free and open source.