What shall we do with the W3C DOM?

Posted on November 12, 2002

in IA/Usability

by Peter-Paul Koch (ppk)

Rated 3.97 (Ratings: 12) (Add your rating)

Want more?

More articles in IA/Usability
More articles by ppk

Peter-Paul Koch

Member info | Full bio

User since: September 12, 1999

Last login: September 03, 2005

Articles written: 8

Recently I was writing one short article and one far longer chapter on the JavaScript DOM. Much to my surprise, I seem to have thought of something new during this writing. For the article I created a simple example script that allows the user to add or remove form fields at wish. Concluding the article, I concluded that the W3C DOM allows the user far more control on his web pages.

The idea

After some more thought I decided that this idea might lead to a radically new way of creating web pages and that I urgently needed the opinions of my peers. So I wrote a mail to Evolt and my own WDF-DOM list with the following theory:

Basically, since the W3C DOM allows us to completely rewrite the page according to the wishes of the user, we should design web pages in a new way. We no longer need to take serious decisions about how the site will work, how the navigation, the forms and the other elements interact with the users. Instead, we can offer the user a way to create his/her own web page, with exactly those elements and that interaction he/she wants, likes or needs. Thus one web page can look completely different for two users.

At its most extreme, usability specialists no longer need to decide how the web page will work and how the user would like to interact with it. Instead, they should decide how the "control panel" of the page should work, how the users can create their own page with a minimum of fuss.

On the whole the reactions were positive. Some members brought up practical problems (how to store the user's private interface, that sites might start to look alike if users can redesign them at will). While these considerations are not unimportant, they should be deferred to the future. The first question is: should we give the user such control, exactly what will this control consist of, and what will this control be good for?

Interestingly, no one referred to any article or website about this theory. It seems that no one has thought of it before (or, at least, has written about it), which in turn might mean that I'm the first one to think of it. Good for my ego, but it also means that we're groping around in the dark. (And if anyone does know about an article that discusses this idea, I stand ready to be corrected).

So we've got a new idea. Is the idea worthwhile?

What's it about?

First of all, let's make the idea itself somewhat clearer. The reactions showed that people thought I was talking about users influencing the design of the site or calling up new data. I meant neither.

Sure, we can write a script that allows the user to pick a design of his choice or even to upload his own background images (and incidentally, we don't need the W3C DOM to do this). Sure, ideas like the Semantic Web might allow us to request and control data flows in the future, but structures like it aren't operational yet.

What I meant is the users influencing the presentation of the data, not design-wise but interaction-wise. Let's return to my simple example script. The idea is the following:

Users can enter short CD reviews. Each review consists of a set of form fields.
How many reviews does the form allow? In the past we web developers had to decide on a certain number. Maybe we could add some DHTML to make the form more usable, but the users couldn't change the maximum number of reviews.
In this example, though, the user can choose exactly how many reviews he wants to enter. If he needs more form fields, he clicks on the button and gets them. If he wants to remove a review, he clicks on another button and the review is completely thrown away.
Submitting the form sends all reviews to the server.

In this example the user cannot influence the design of the site. He doesn't need to, either. Nor can he influence which data he views. Instead, the script allows him to personalize the form itself, so that entering data becomes a much easier job and he can choose how many form fields he needs. So the user is allowed to control the interaction, but nothing else, of the page.

This is what I meant.

Problems

Members saw some problems with the new idea. Some of the more interesting are:

The idea will meet with resistance, especially from marketing departments, but also from users themselves. It's a new idea, and thus it's viewed with suspicion.
Reaction: True, but unavoidable. It will take a few years for mainstream sites to use this concept. So what?
Personalization through server side applications is already commonplace.
Reaction: True, but the W3C DOM is not a server side application. For instance, when a user personalizes the page through the W3C DOM he doesn't have to wait for the page to reload. Is this enough difference to choose the W3C DOM over server side applications?
It's a good idea, but only for portal-like sites.
Reaction: True. We have to find out which sites this idea will work best for.
Only people who have experience with building UI's will actually use the scripts.
Reaction: I'm not so sure if this is true. Many users don't want to be bothered with this kind of functionality, but some of them will. And if it becomes commonplace to have such personalization, more and more people will start using it.
For users who don't want to personalize, we need a good default UI.
Reaction: True, of course.
All possible combinations of UI elements have to be rigidly tested.
Reaction: I don't agree. We need to design the individual UI elements, but we leave it to the user to combine them. If a certain combination turns out to be unworkable, the user will change the UI.
Sites may lose their unique look-and-feel.
Reaction: I don't know. If the users can influence the graphical design of the site it might happen, but my idea isn't about the graphical design.
The result may be something else than a website.
Reaction: It's too early to decide on this. But if websites are not quite website-like any more, won't that be a fascinating development?
There will be flaws in the implementation of the scripts, and the user has to deal with them.
Reaction: Possible, of course. But it all depends on how the scripts are implemented. As long as the data (and maybe the design) are divided into comprehensible blocks, there will not be such problems.

None of these problems strikes me as unsolvable.

What kind of site?

Of course a lot depends on what kind of site you're making. I made the following division:

Sites it will work for:

Portals
Intranets
Library/reference site
Newspaper sites
Intranets

Sites it won't work for:

Corporate sites, since the exact design is too important to allow users to change it
Entertainment sites, for the same reason

In short, it will work on sites that offer a lot of information. It won't work on sites where user experience is more important than information.

Ideas

So what shall we do with the W3C DOM? It all depends on good ideas. It might be possible to open a new chapter in user interaction, but we need some good, solid, above all practical ideas.

The IHT site, the best practical implementation of the unique possibilities of the W3C DOM so far, offers the users limited control over their environment. They can adjust the font size, and can switch between a 1 and 3 column layout.

How could this interface be extended? We could think of offering the user however many columns he'd like, or to view several articles simultaneously. Maybe someone else has more ideas...

Another good idea is to use the W3C DOM to program a wysiwyg-like application that allows people to create their own websites.

But we need more ideas. We need, say, a dozen scripts that implement this idea in a small way. Then we could start evaluating them, see which functionalities are unique and which aren't, which functionalities the users like, which functionalities offer a true chance to build websites in a radically new way.

Please use your imagination and write a practical W3C DOM script that allows the user control of his environment.

Peter-Paul Koch is a freelance browser expert and JavaScript guru living in Amsterdam, the Netherlands. He has been an Internet professional only since 1998, so he's definitely second generation.

His personal site is www.quirksmode.org. It includes the W3C DOM Compatibility Tables, currently the best resource on the Internet for this subject. Because of this research, he has been asked to co-edited chapters 17 to 19 of Flanagan's "JavaScript, the Definitive Guide", O'Reilly, 4th edition.

He is an administrator of the WDF-DOM mailing list, that counts most international JavaScript gurus among its members.

He has written the "Keep it Simple" column on Digital Web Magazine, as well as articles on A List Apart, Apple Developer Connection, and O'Reilly's Web Dev Center, in addition to Evolt.

22 comments on this article. Log in to add your comment | Rate this article:

XML and XSLT

Submitted by starvingartist on November 14, 2002 - 22:21.

What about just using XML? You can use a client side XSLT (XSL Transformation) to transform an online XML page into the presentation you want. With XML and XSL you separate structure and presentation.

Isn't this already the direction W3C and the semantic web is going?

Re: XML and XSLT

Submitted by ppk on November 17, 2002 - 08:27.

You can use a client side XSLT (XSL Transformation) to transform an online XML page into the presentation you want.

True, but you cannot change the page once it has been displayed. Besides, client side XSL is badly supported and generally not important.

Previous Work like this

Submitted by tim.parkin on November 19, 2002 - 10:46.

Take a look at www.xopus.org and more directly http://www.q42.nl/demos/annapa/ This is an idea thats been floating around for a while. Rebuilding the page within a certain context can save on a load of time. Lots of modules reorder html tables on the fly by clicking on the column you want ordering. In the larger view, it's a bad thing to do to much of this for the same reasons that Flash disables the 'back' button. I may have misunderstood what you saw as 'new' about the approach, if it's doing it with the W3C Dom then I understand but it's hardly earth shattering and falls in the category of implementations rather than inventions.

test of the example script

Submitted by dusoft on November 28, 2002 - 05:17.

I tested the script recommended and found out that it is veeery slow. I have AMD Duron 850, 128 MB RAM etc... Internet explorer v6 and it was so slow - the whole collapsing, not very usable if it was supposed to run on older computers, too.

How about something like this, PPK?

Submitted by MiArDe on November 29, 2002 - 22:50.

http://www.miarde.com/DOMTest/domFormEx.php

Good example!

Submitted by ppk on November 30, 2002 - 03:34.

How about something like this?

Yes, that's the best example I've seen so far.

Tim's Example

Submitted by MiArDe on November 30, 2002 - 10:29.

Tim - I tested the site you posted above ( http://www.q42.nl/demos/annapa/ ) but it crashed my browser every time. IE5.5 on WinME. It causes a failure in DXTMSFT.DLL. Any ideas?

q42 example

Submitted by tim.parkin on December 1, 2002 - 03:45.

Not really, its only a technology test so I preume they've not done any extensive testing. The dll is a direct x file (I just checked, it'sinstalled by DX8) so it is a rendering bug. I don't know why this should crash but I would guess it's an early IE5.5 release. the miarde test is a more stable example (after a quick test by me) and is also more 'self explanatory'.

Another Example...

Submitted by MiArDe on December 1, 2002 - 14:36.

Thanks Tim! I just wanted to view more examples but I'll leave that one be.

I threw together another example that may be used to let users resize content areas. There are probably more elegant ways to do this and I haven't looked into all of the problems that may arise by using this method but hey, it's just an example.

Resizing Example

Idea

Submitted by MiArDe on December 3, 2002 - 08:50.

As far as ideas go, what about giving the user the ability to customize the site's menu? Let them remove items from the menu that they do not use, like, etc. Of course items that are critical to the site's functionality would have to be "read-only" but other, less important ones, could be removed. Do I really need to add that the site would have to have a menu in the first place?

Has anyone checked out the links being posted to the wdf-dom mailing list? It looks like a lot of people are/have been doing things mentioned in this article.

Always Behind

Submitted by MiArDe on December 5, 2002 - 07:18.

After lots of reading and searching, my examples seem infantile. As far as what can be done with DOM, I've started looking into using DOM combined with XML-RPC ( a-la Scott Andrew's javaScript client and Brent Ashley's RS lib ). What about letting users pick the types of components they want on their interface - components created by different authors, distributed across the web? Using XML-RPC to grab either the component code (or engine or both), then using DOM to take the info and put it to use. I'm sure this isn't a new idea but I don't see it being done. Thoughts?

XML RPC?

Submitted by ppk on December 6, 2002 - 04:49.

What is XML RPC? Never heard of it.

A spec...

Submitted by MiArDe on December 6, 2002 - 07:07.

...that allows you to make remote method/function calls over the internet (via http). Basically, you send an XML-encoded message over HTTP to a waiting RPC server ( CGI script, PHP script, etc. ) and get an XML encoded response. You can read more about it here

If you're interested, check out Scott Andrew Lepera's javaScript client in the "Implementations" section.

It all depends on good ideas.

Submitted by Oleksa on December 11, 2002 - 15:20.

"We need to design the individual UI elements, but we leave it to the user to combine them. If a certain combination turns out to be unworkable, the user will change the UI."

One another example

Driving a nail with a screwdriver

Submitted by ttfkam on December 22, 2002 - 18:46.

Allowing the user to manipulate the DOM is a very bad idea. Aside from complexity issues, it's the wrong technology. If people are going to go through the trouble of re-working their pages into something more maleable, why not just write clean, simple XHTML and allow for customized stylesheets? In the beginning, you could simply offer a few pre-made skins in CSS which you could programmatically switch between (and persist state). As time goes on, you could serve a custom stylesheet to the user.

DOM manipulation breaks easily if the page structure changes at all. CSS is far more forgiving and flexible.

...of course, this may be a moot point as most users really won't care to take advantage of the flexibility. One site does it, people will twist and tweak for a while. Dozens of sites do it, and people will figure that it's not worth the time -- there are too many sites to tweak and far too few hours in the day. This seems like a "problem" only a webmaster would want solved. This is ironic considering that a webmaster is indeed the only person who doesn't need this functionality -- they can already reformat to their heart's content on the server-side.

Re: Driving a nail with a screwdriver

Submitted by ppk on December 26, 2002 - 06:11.

Because my idea is not about changing styles but about adding or removing blocks of information or functionality (like fomr fields) to or from the page. Without reloading the page, only the W3C DOM can do that.

DOM manipulation breaks easily if the page structure changes at all.

Not true. If you write your script correctly it will continue to function even if the page changes radically.

of course, this may be a moot point as most users really won't care to take advantage of the flexibility.

That, of course, might be a real problem, as I said in the original article. But it's no reason not to write such scripts.

One site does it, people will twist and tweak for a while. Dozens of sites do it, and people will figure that it's not worth the time

I disagree. Sure, people won't tweak every single site they encounter. But if many sites use such scripts the general idea of being able to change the interaction of the site will become more and more accepted and people will use the scripts on sites that matter to them.

Re: Driving a nail with a screwdriver

Submitted by MiArDe on December 29, 2002 - 18:36.

Manipulating the DOM IS a bad idea! That's why I'll leave it up to the W3C ;-) As far as manipulating the document, I disagree. I like the fact that I can:

Load an XML doc into memory
Allow the user to modify the document in memory via the DOM API
POST once - when the user is done their "session"

A recent project I've just completed required that users be able to "complete" tasks assigned to them. That involved gathering input from the user and showing the progress immediately. I was able to create the input fields, update the tree and show the progress all using DOM and client-side XML transforms WITHOUT making another trip to the server to show the progress.

There are still a lot of areas which need improvement but this is all new. I wish more people posted examples of how they're using DOM so we could all learn a bit more...

Re: Driving a nail with a screwdriver

Submitted by ttfkam on January 4, 2003 - 04:04.

Because my idea is not about changing styles but about adding or removing blocks of information or functionality (like fomr fields) to or from the page. Without reloading the page, only the W3C DOM can do that.

display: none; and the programmatic equivalent. I guess you could call it DOM, but hiding is definitely different from removal. Of course there's also the functionality coming into newer clients (like Mozilla/Netscape) that allow you to select the stylesheet used for display.

A recent project I've just completed required that users be able to "complete" tasks assigned to them. That involved gathering input from the user and showing the progress immediately. I was able to create the input fields, update the tree and show the progress all using DOM and client-side XML transforms WITHOUT making another trip to the server to show the progress.

What you are talking about here is not a web page; It is an application. There is a fundamental difference here even though the clients may be the same. A web page as commonly encountered is very much akin to a magazine article. What I'm afraid of is giving the reader a pair of scissors so that they can cut up the article and paste it back together as they see fit. What you have is a case of indecision on the part of the web designer. Take Mac UIs versus Gnome on Linux. The latter is far more customizable, but is that a good thing? With a Mac, just about any user can go to just about any Mac and be able to use it effectively. This is because Apple -- more than just about any other company I think -- actually engineers its user interfaces. Gnome tried to abdicate their responsibility for good UI design by pushing all decisions to the user. Strangely, I have yet to see a Gnome customization that approached the usability of OS X.

Web applications, on the other hand, use dynamism in order to enhance accessibility. In an attempt to be clearer, take a play by Shakespeare. Can you imagine it as an application? No. It is content. It may be delivered by an application, but on the web it would be a page. There may be different layout choices (definitely the jurisdiction of CSS) and you may want to toggle external references and footnotes (display: none/display: inline), but this is still the responsibility of the content author to make intelligent and informed accessibility decisions.

Now let's do our taxes online. The tax code may be content (a page), but the interaction, the give and take, the decision tree, etc. all point to an application model. Page = content. Application = process. Both require user interface engineering. Applications are well suited to document manipulation through the DOM, but content is a layout domain. That layout may be driven by scripting and anchored to elements accessible through the DOM, but this is not the same as picking up element A and dropping it behind element Z.

I know what you're thinking now. "Isn't that arrogance and stating that you know better how someone should access the information?" No, I don't think it's arrogance. And yes, you should know better. If you cannot make prudent decisions regarding the information you are providing, what business do you have providing that information? Could you imagine if O'reilly started publishing books to each reader's whims? Aside from the publishing costs -- which largely don't apply to web content -- what happens to the poor sap who borrows my book because he lost his copy? O'reilly is extremely consistent with regard to presentation and one O'reilly book can be accessed much like another. What about the case where the user makes a change that hinders access to what would normally be prominent information? Are you going to check every possible configuration to make sure that your site announcements are still available? Are you going to check every possible configuration to make sure that your site still conforms to web accessibility guidelines? Open-ended configuration options are not a good method for development. Your primary responsibility isn't about giving a socket wrench to someone when you sell them a car; It's about selling them a safe car that doesn't need a lot of maintenance. Selling them the car and explaining to them that the radiator could be shifted over to make room for a larger engine is asking for trouble (and weekend calls asking for help when the car won't start).

Your job as a page/app author is to provide a service in the most straightforward and intuitive method possible. If you are making an application (and I could be convinced that portals are stradling the line between page and application), I could see the DOM used by the page/app author -- but this is not giving scripting capability to the client. Giving the user direct access to a page's DOM is a security hole waiting to drop open or at the very least a tech support nightmare. "Static" web pages and applications have enough usability issues without adding a host of "I accidentally deleted the login box when I access your site. How do I get it back?"

Whew! That was longer than I expected. In summation, I don't think the DOM is evil. I actually think that ppk's thoughts are valid...for applications.

My tuppence worth

Submitted by rhundt on February 22, 2003 - 22:19.

Imagine a booking engine - it makes hotel reservations - it has a calendar displaying two months and the means to select then next two months for making reservations up to 12 months ahead. Not only are the calendar cells are clickable, thereby allowing you to interactively select a date range for your stay - it has to automagically selecting dates in - between - checking room availability, and applying rate plans to work out pricing (which is displayed in it's appropriate place in the interface and is dynamically updated as the user clicks around). It also then needs to provide descriptions for each room type (chosen from a list) and give users the ability to control the number of Guests which could be Adults, Children or Infants. Suppose also that you want to do this using visual (GUI-like) components (instead of Forms - unless it is for the mandatory credit card information) - so a plus `button' is clicked and a little dude appears in a box (he/she's of the taller variety being adult). Moreover, there are tooltips (with stylesheets, applied) that can have the `time-out' customisable by the user - put one on each interface element - you get the idea. And it has to do it all in a single page transition. With the old DHTML methods, you end up writing a lot of stuff like:

for (i=0; i ';
}

// you end up having to build your own data structures and iterating 
// over arrays that you have to manage in `user space'
// wheras the DOM API gives you a tree to work with as a data structure,
// and all the methods you could wish to have to dynamically manipulate
// your document strucure (including XML documents - interstingly, the DOM API was
// designed with mainly Java and C++ implementations in mind - JavaScript
// `implements' the DOM API, but it is certainly not the only implementation - this is why
// it has uses for manipulating XML documents equally well as is seen in the way 
// Mozilla is put together using XML languages and a kick a-rse JavaScript engine

for adding little thumbnails, for instance - representing room numbers in your as an HTML string. This invokes the HTML parser, which has to parse that string. This is slow, and compounded by the fact you end up have to carry arrays of references to interface elements around and render them. You can imagine hand coding all of the HTML for two months, adding onlcick event handlers to each, giving each it's own div tag with an `id', and then somehow iterating over the HTML elements hardcoded into the document. If you try to capture this sort of complexity in a single page using non-DOM - legacy - DHTML methods, you soon end up with bloated unmanagable code that is incredibly expensive to run (in terms of CPU resng a host of "I accidentally deleted the login box when I ources).

With DOM methods, one can build interfaces that provide the sort of dynamism that Macromedia's MX products have. This is because the DOM API defines everything as a Node - and discribes a STANDARDS compliant way for manipulating - and they really mean `manipulating' that tree.

I think this is more or less what PPK is saying. The power is in the modularity the DOM API provides, because you can create your own web pages with an advanced `software-like' interface is revolutionary. The DOM API lets you say things like:

var my_div =  document.createElement('div');
var my_table = docment.createElement('table');

my_div.appendChild(my_table); // and voila - it appears;

my_div.getChildNodes[0].style.backgroundColor = '#426942'

// or any attribute:
my_table.width = "100%";
// etc.

any changes you then make to my_table, or any of it's child nodes via methods such as `my_table.insertRow(arg)' are immediately updated in the browser - without the need for invoking the HTML parser.

Moreover, the DOM API allows you to get at parent, sibling and child nodes, and even further up the ancestor and down the decendant tree.

With the advent of the DOM standard, the ball is more in the browser vendors' courts - and the open source browsers are beating the proprietary ones hands down - it is just a matter of time - the internet is changing from a publishing medium to a distributed applications medium - to echo what `ttfkam' said. It us just a matter of time until all major browsers become fully compliant with the DOM standard.

In funishing off my $~ cat dev random to std out [loosly translated as `contribution to this.subject], I believe that we web developers are facing a major change in how we approach design - web sites will start to look and feel more like software (and indeed they already are) - after all why do something in three server transactions and as many pages if you can do it all in one - and still in a `thin client' way (no trojans - like Flash). All you need to do is to start adding stuff that is really close to the GUI's that people are accustomed to - hell, why not add a "File", "Edit", and "Help" button to a bar at the top, and make it drop down with a menu for which you don't have to write a single line of HTML, complete with CSS rollovers and event handlers...aw don't get me started - DOM doesn't only make it possible, it makes it easy.

The latest version (`4th Edition') of O'REILLY's "Definitive Guide to JavaScript" has IMHO a good section on DOM (and its use with JavaScript)

oh, and ...psssst XML will change the world ;-)

THIS shall we do...

Submitted by ppk on May 30, 2003 - 23:23.

In a new article I've taken this idea one step further: Forms, usability, and the W3C DOM.

Good..

Submitted by Putte_5 on June 30, 2003 - 03:45.

Alot of good questions raised here. /Putte

I'll do it if you'll help...

Submitted by HillsCap on December 20, 2003 - 16:56.

Hi, I've been wanting to create an investing website for futures trading. My idea is that it would look exactly like TradeStation 2000i (the best futures charting/trading platform, in my opinion), with user-configurable tabbed pages that show charts of futures contracts that the user can specify. For instance, the first tab would be a quotes page... it'd be a simple spreadsheet-like page in which people could enter the futures contract symbol, and the page would interface with our Clearing Firm's data feed for OAK II (their online trading platform) to fetch those price quotes (high, low, open, close, last). Then, they could add additional tabs and chart whatever symbols they wanted, the charts being taken from the same data feed. Another tab would allow a person to interface with their account and keep track of their profit/loss in real-time... this would work if they're doing live "real-money" trading or just paper trading, again interfacing with the OAK computers for the information. The OAK system supports paper trading as well as live trading. Yet another tab would allow a person to specify what RSS news feeds they wanted to receive... either from an existing list, or they could type in the URL of a news feed if they wanted. Included in the news feeds would be market commentary from various brokers around the U.S./Canada/whichever other country brokers wanted to participate, in text format. The last tab would be a multimedia tab... allowing the person to chat in real time with other traders and with the brokers of their choice. This tab would also have a user-configurable list of brokers whom they would like to listen to real-time trading advice from. Each broker would be sending out a UDP voice feed that could be 'subscribed' to. The user could add any of these tabs, or hide them if they choose not to use them. The tabbed interface would allow a person to quickly switch back and forth between quotes, charts, text commentary, chat, etc., and would allow a person to do one-click trading from either the quotes tab or any of the charts tabs. But, I need help with the programming...

Start of page header

Other Fine Evolt.org Sites

Navigation Starts

Submit

Article Categories

Highest rated articles

Recent comments

Main Page Content

What shall we do with the W3C DOM?

Want more?

Peter-Paul Koch

The idea

What's it about?

Problems

What kind of site?

Ideas

XML and XSLT

Re: XML and XSLT

Previous Work like this

test of the example script

How about something like this, PPK?

Good example!

Tim's Example

q42 example

Another Example...

Idea

Always Behind

XML RPC?

A spec...

It all depends on good ideas.

Driving a nail with a screwdriver

Re: Driving a nail with a screwdriver

Re: Driving a nail with a screwdriver

Re: Driving a nail with a screwdriver

My tuppence worth

THIS shall we do...

Good..

I'll do it if you'll help...