Dogfooding your CMS
Recently Janus Boye posted a quick test of leading CMS vendor sites and how they faired using a mobile browser to view the site. I must say I was quite proud reading this list. Most of the vendors failed the test, but two out of the three CMSes that passed were the CMSes I use almost every day; Sitecore and WordPress. Now, the author didn’t detail what the test constituted, but from what I read I’d say it was probably more of a general overview of how usable the site was when using a mobile browser. Much the same as what a non-technical user might do if they were evaluating the CMSes themselves.
Now the body of the post made me quite proud, but the conversations that ensued in the comments below the post were quite interesting. Obviously this post attracted quite a lot of attention from those that failed the test. One in particular caught my attention, not because of any close alliance I have to that product (or baiting I dish out to a particular member of the CMS’s core team :) ), but because of the reasonings and defenses from the comment author.
The comments came from Niels Hartvig, one of the creators of Umbraco which is a popular open source .net CMS. The Umbraco site had of course failed the mobile test and Niels was saying the Umbraco site didn’t support mobile browsers because the amount of mobile traffic the site received was negligable. And when trying to divide your precious time among the mountain of tasks one has, I can understand why the good people that make up the Umbraco development team decided not / didn’t have time to ensure mobile browser support for the website. Believe me guys, you have my sympathies re: spare time budget.
In fact, another commenter mentioned they are the cobbler’s daughter who has no shoes. Again, I can sympathise with that.
The issue here however is that I think Niels may have interpretted his website analytics incorrectly. I’ll take his word for the amount of mobile traffic the Umbraco website receives, but what single numbers don’t tell you is what the users intent on the site was. (Could turn this into an OMS post but won’t…) For any CMS vendor (or even IT services company) I suspect majority of the mobile traffic visiting the site are not people wanting information on the CMS or company, but people checking to see how well the CMS can handle a mobile browser. These are the people with just enough knowledge to make them dangerous :) .
I have no doubt that Umbraco is capable of producing a very good mobile experience. But the fact that the official website doesn’t provide that experience to prove it doesn’t play well for them. The same goes for XHTML compliancy. Businesses evaluating CMSes that have been spooked by someone fear mongering about XHTML compliance on your website will also run the source of the CMS vendors website through the W3C validator to see if the CMS is capable of outputting XHTML compliant markup. This test doesn’t prove anything other than if the current website is compliant. It says nothing about the CMS.
So I ran the CMSes that passed the mobile test (Sitecore, WordPress and Plone) through the W3C validator at validator.w3.org as a potential business user might, and found the only one that passed was Plone. I also ran the Umbraco site through the validator and found it contains the least validations errors (3 errors) compared to Sitecore (17 errors) and WordPress (9 errors). Checking this blog through the validator I also has errors but the errors were only in the content I’d written (maybe I shouldn’t be using tools but cutting HTML of posts myself). The theme I’m using is compliant.
Now, I know that just because a website doesn’t contain 100% compliant XHTML markup it doesn’t mean that the CMS running the site is not capable of outputting 100% compliant XHTML. Same goes for mobile support of the site. But as I was once told by someone working for a major CMS company, “Business users don’t get that a single website doesn’t portray the entire capabilities of the CMS. They will choose a CMS just because it has a pretty flash component on the homepage”. Something to keep in mind the next time you’re trying to decide if you should add a mobile version of the company website.