Experiences with W3C Compliance for Mobile Websites

Introduction

Mobile technology has taken the world by storm, and as all of us are aware the next decade belongs to the hand held device. Corporations are increasingly trying to reach out to their customers on the hand held devices platform as well. One of the easiest ways to make a presence on the mobile space is to develop a web presence. This approach is considerably simpler than the dedicated native app due to the following
•    There are far too many platforms to write native apps for and all of them are different from each other
•    There is a considerable overhead of maintaining apps in different technologies
•    Platforms keep upgrading, sometimes rendering the previous versions incompatible.
Though developing a mobile website seems to be the preferred option, there are a number of careful considerations to be made while developing a mobile site. The compliance to W3C recommendations is one key aspect of a well designed mobile web site.

What is not covered!

This document is NOT an attempt to redefine or describe the W3C best practices for Mobile Web sites, which can be found here: http://www.w3.org/TR/mobile-bp/. But with the number of websites we have done for mobile enablement, there are bits and pieces of information I want to share which have enabled the sites to pass the W3C compliance checks. Each section below takes up a recommendations and tries to make practical suggestions on what it meant while developing a mobile website.

Thematic Consistency of URI
Content should be accessible on multiple devices. Bookmarks made on one device should be portable to other devices. Bookmarks could be decorated with session information, if the page needs the session information which is not available on the second device, the bookmark should take the user to the login page – This is true even for a normal website – like implementing a Security Filter for the Controller.

Exploit Device Capabilities
Making the page work on all the devices should not be done by compromising the capabilities of an advanced device. So typically you go with a default delivery style and add on to it based on the device capabilities based on run time. 

For e.g., some of the phones support JavaScript, others do not. A Simple example of using this capability only on the devices that support the same is shown below. Code is written in Velocity.

#set($ua = $request.getHeader("User-Agent").toLowerCase())
#if($session.getAttribute("res") == "hi")
#set($hiGuiUA = true)
#set($cssFile = "hi_res_CSS")
#elseif($session.getAttribute("res") == "low")
#set($hiGuiUA = false)
#set($cssFile = "Low_Res_CSS")

The user-Agent header gives detailed information about the Client device from which the request originates. For e.g. the user-agent header for a Iphone is something like the following

Mozilla/5.0 (iPhone; U; XXXXX like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A477d Safari/419.3

Also the user-agent value which would come from IE6 is as follows

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

Work around Deficient Implementations
The device capabilities vary very widely as of date. We need to support different devices whether they are low or high capability. On low capability devices, we should smartly trim down our implementation dynamically. The above code is an example for the same.

I have given another case below where you can even check the specific device names and then decide on what to do – this filters for android, Iphone, Ipod, few blackberries and the Palm Pre.

#elseif($ua.matches(".*(android|ip(hone|od)|blackberry(9630|8530|9530|9550)|pre/1.).*"))
#set($hiGuiUA = true)
#set($cssFile = " hi_res_CSS ")
#else
#set($hiGuiUA = false)
#set($cssFile = "low_res_CSS")
#end

Please note the following for some additional information on other manufacturers, other than the ones which are included on my code snippet.
•    The word "nokia" can be found in the User-Agent header of Nokia's cell phones.
•    The word "motorola" or "mot-" can be found in the User-Agent header of Motorola's cell phones.
•    The word "samsung" or "sec-" can be found in the User-Agent header of Samsung's cell phones.
•    The word "lg-" can be found in the User-Agent header of LG's cell phones.
•    The word "sonyericsson" can be found in the User-Agent header of Sony Ericsson's cell phones.
•    The word "sie-" can be found in the User-Agent header of Siemens' cell phones.

Carry out testing on actual devices as well as emulators.
This is a very important aspect as a lot of the emulators behave different than the real device. Some good examples is that a lot of wrapping issues seen on the blackberry emulator cannot be reproduced on the device

In my experience (I have not used the Iphone emulator – have used Blackberry, Android, Symbian) the emulator which very closely mirrors the device is Android. In our testing we found the device behaved 98% similar to the way the emulator did.

Site Entry Points
As typing URLs on the mobile is always difficult, allow users to access the pages with shorter URLs. This is especially true for front page URLs

For e.g. we use http://m.myconpany.com/ rather than http://m.myconpany.com/index.html

Enabling this is very simple, most web servers will automatically redirect to “index.html”. Else this can be added in your Welcome file list in the Web.xml – See the example shown below

<welcome-file-list>
    <welcome-file>index.html</welcome-file>
    <welcome-file>index.htm</welcome-file>
    <welcome-file>index.jsp</welcome-file>
</welcome-file-list>

Balanced Structure for the page
Take into account the trade-off between having too many links on a page and asking the user to follow too many links to reach what they are looking for. The approach should be to minimize the number of links on a page. As the device will have limited screen real estate, attempt should be made to minimize the amount of scrolling to be done

Link Target Identification
The target of a link should be identified according to its size and time it takes to be downloaded. As most mobile devices work a very low bandwidth, this is an important consideration to make. The resources should not be hidden from low end devices, as the user might want to download on this device and use on another.
Providing clear, concise, descriptive link text will help users identify each link’s purpose. An example of this is shown below
<a href="/server/mysong.mp3">
   <img src="Song.gif" alt="Download the Song as MP3" />
</a>

Image Maps
DO NOT use Image Maps. Let me explain what an Image Map is.
In HTML and XHTML , an image map is a list of coordinates relating to a specific image, created in order to hyperlink areas of the image to various destinations (as opposed to a normal image link, in which the entire area of the image links to a single destination). For example, a map of the world may have each country hyperlinked to further information about that country. The intention of an image map is to provide an easy way of linking various parts of an image without dividing the image into separate image files.
Most of the devices do not support image maps properly. Avoid using them.
Simplest solution is split the image in to multiple images and link each image to the different location.

Externally Linked Resources
Keep them to the minimum. The problem with external links is that they are a separate request to a different domain. Good for reducing the bandwidth foot print.

Page Content
Page content needs to be generic to be displayed on all devices. One tip here is the page layout. We need to make sure that the page can span correctly on all the devices
One simple tip to use is to use the device’s width to display the page. An example of this code is shown below. This meta tab allows you to span your website according to the device width you are supporting.
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no" />

Page Size
Page size consideration we made was that the page should not be more than 10KB to be downloaded in one shot. We significantly trimmed our images and used small GIF images. One of he simplest options to control the page size is to split the page in to multiple screens when there is too much data to show. One such example is a summary page, where we can choose to show only 10 accounts per page.
There are many other ways to limit page size to the minimum. If you are sending JS files minify them. Also compact your HTML by removing all the unnecessary spaces. One another approach is to use the GZIP compression of pages, but this is not recommended as of now as there are wide disparities among the device browsers as to how they handle GZIP compressed pages.

Content Size Limitations
Limit the content to what the user has requested for. Most current mobile web sites use REST based services to fetch the data from the backend. This means that you need to carefully control the granularity of your service. Let us take the following example to demonstrate this situation.
Consider we need to develop a Employee Listing Service. The service is exposed like the following URL: http://myserver.com/employees. A simplest implementation you can think is to map this URL to a Servlet in your Web.xml and send back an XML response from the service method.

Here is the sample response the Server sends back

<?xml version="1.0"?>
<e:Emps xmlns:p="http://myserver.com"
xmlns:xlink="http://www.w3.org/1999/xlink">
<Emp id="00345" xlink:href="http://myserver.com/employees/00345"/>
<Emp id="00346" xlink:href="http://myserver.com/employees/00346"/>
<Emp id="00347" xlink:href="http://myserver.com/employees/00347"/>
<Emp id="00348" xlink:href="http://myserver.com/employees/00348"/>
</e:Emps
So now assume that the Client needs more details on a specific employee. The Client would invoke the URL: http://myserver.com/employees/00345 - This URL also would have a servlet mapping and this servlet will fetch the details of the employee and send back the detail response as shown below

<?xml version="1.0"?>
<e:Emp xmlns:p="http://myserver.com"
xmlns:xlink="http://www.w3.org/1999/xlink">
<Emp-ID>00345</Emp-ID>
<Name>John Doe</Name>
<Specification xlink:href="http://myserver.com/employees/00345/specification"/>
</e:Emp>
</messages>
It is very important, especially in the case of a mobile application to decide on what detail you should return to the Client. The simplest approach is, return only what the client asked for.

Scrolling
Avoid horizontal scrolling as much as possible. The page should be aligned in such a way that user can access all the links by using vertical scrollbar only. Please see an implementation sample below

Images and graphics
Avoid usage of big images. All the images we used were png files which were roughly 1-2KB size
One another way of compressing the image is to use loss-less compression techniques like the Yahoo Smush.it™

Tables
Avoid usage of tables to define the layout of the page. Some devices do not support them. Even if they are used, use them to display only on High Resolution device using conditional logic we described earlier. One more important thing to notice here is that never set a “Fixed Width” for any cell on the table.
Also, do NOT set the “nowrap” attribute, like shown below
<tr>
    <td nowrap="nowrap"> 

Page Markup should be valid
Test your page markup for validity using http://www.w3.org/QA/Tools/#validators

Pixel Measures for elements
DO NOT use them. Always use code with % spans like the following
<body>
<table width="100%" cellpadding="0" cellspacing="0" border="0">
<tr>
<td width="100%">
<!-- start header -->
<table width="100%" cellpadding="0" cellspacing="0" border="0">

Character Set determination
Some devices do not support some character sets. The supported character encodings for a device may be obtained either from a device profile or by examining the value of the HTTP Accept-Charset header. Read the header and make sure that you are setting the correct character set for your response.

User Input
We discussed that we want to keep the number of key strokes by a user to the minimum. So the following things need to be taken care

Where possible use previous entries as defaults – For e.g. if the user entered a specific account # for billing before, try to save this information in a user profile structure and pre-populate this information for the next time. Another example of this is to remember user preferences like his last searched Items.

Try to avoid free format entry as much as possible – Try to convert free format fields to picker fields as much as possible. Designing drop downs for values is a very good option as most devices support picking values from them.

Java Source Reflection - some manifestations

Java Reflection is a great tool, but a lot of cases like source code generation tools, there is a big problem that the code has to be compiled before it reflects on the same. Java Source Reflection comes to the rescue. This is a great way of analyzing the Java Source File without the need for compilation. A more detailed explanation can be found here

Tools like Eclipse make use of Source Reflection as it makes the tool much faster with its tooling. There is one interesting behavior though:

Say you have Class A in the package : com.company.product.Package

and I import the Class A like this in another class : import com.company.product.Package.A;

Now in the Eclipse Source Folder I create this directory structure as : com/company/product/package/A.java - Notice that the directory structure has a "p" there (May be developer mistyped the directory name) 


Your Ant Build will work fine, because when the class file is created, it will create the classes directory according to the package declared with the "P" and the import also would be recognized properly.

But eclipse will show you an error as the source path does not match the import or the Package declared. You might not see this issue with newly developed code using eclipse as you get the error instantly, but if the code is existing and imported to eclipse, this is a very good possibility


Interesting manifestation of Java Source Reflection Vs. Reflection (Or rather compilation)

GWT and JQuery - my musings

I have been programming with GWT and JQuery for about 8-10 months now. I started with GWT initially, and loved it (I was a POJD - Plain Old Java Developer till some time back). GWT is an amazing take on combining the powers of a Structured, Strictly Typed Java to the absolute power of the JavaScript. I really think the ideas of JSNI invocation and the seamless RPC interface is nothing more than wonderful. It makes a good Java Developer instantly productive on the Script based GUI. We predominantly had Java Developers and it made life so easy for them to be doing a Complex AJAX based front end in no time with GWT.

Lately I made amends to my knowledge base on JavaScript and started using the JQuery Framework. It has gradually grown on me. Below is some of my thoughts on both of the frameworks. I agree that I am more influenced by JQuery and might be favoring usage of JQuery in most cases.

1) JQuery is built upon a simple principle of "progressive improvement".
   - You start from a basic design and keep enhancing it. This approach is very productive as you start with a basic page design and then add functionality by using the selectors and defining the behaviors
  
2) JQuery Code is simpler and easier to understand. We shrank some of our 10K line script files to just a 1K or so. Significant amount of modularity is achieved using the plugins.

3) JQuery Supports encapsulation, though (I would say) imperfectly. All the plugin code can be thought as encapsulated. it is not in the strictest sense as you see with GWT (As it is written in Java), but I believe that JQuery enhances the JavaScript by providing a lot more scope for encapsulation.

4) Service invocations: One of the biggest advantages I saw with the GWT was that making REST based calls to the Server became so easy. GWT-RPC makes those rally easy and straight forward, the only rider being the GWT needs a server piece to do it with GWT-RPC (Your Project is not all Script I mean). At least in some cases we had a bad taste with this as the Existing Server application was written on JDK 1.4, and to get RPC running we had to make it run on JDK 1.5. Do not think this is a big problem, it is high time we upgraded. JQuery provides a robust AJAX call framework, gives very good options of hooking up to most of the callbacks. One thing I am seeing is that as I use completely Script, the server side is decoupled of how I provide REST based services. This is not to say GWT is not capable of doing this, it can very well do the same using the HTTP Request Infrastructure. But rather than writing GWT RPC Services, we are now preferring to write our REST Services using JAX-RS or Grails, which decouples the Server side from what Scripting framework is used on the Client.

5) JQuery has very good templating support now. Most templating engines are available as plugins and can be used out of the Box, with JSON data fed in to them. Makes life really simpler for complex objects presented on Repeating Divs, Grids etc.

6) I found it kind of difficult to debug the generated code from GWT (Even in the detailed version). The Java debugger on Eclipse Helps, but my firebug or my server logs did not give me much of a clear picture when an RPC call failed. May be I am not smart enough for that. JQuery code was mine, debugging turns out to be simpler.

7) Plugins for JQuery has increased many fold int eh last year, we could virtually find ready made functionality for whet ever crazy stuff we wanted to do, Grids, Grids within Grids, Slideshows, Graphs (Bar, Pie, Line -- what not ????)

8) Editors : Google launched the GWT Designer recently which I believe should make a world of difference to the way the GWT pages are coded. This should make building the UI and even positioning of elements much more simpler with GWT. JQuery Visual elements being nothing more than HTML Markup, will always have advantage of using of any of the HTML WYSIWYG Editors out there.

Conclusions :

I definitely think both GWT and JQuery are really powerful tools. GWT might be the best choice for people coming from a Java background and want to still program with the Strongly Typed language with good compile time checking. But this requires a lot of unlearning of Java as well as to what cannot be used on the Client side code

JQuery on the other hand, expects you to know HTML, DOM Structure and Event handling but is simpler as a framework. I strongly believe that unless you have a huge pool of Java (Only) Developers who need to be used for Web GUI development, all other cases it might be worthwhile to consider using JQuery instead.

Committed data does not appear on the new AJAX call with Hibernate

We had a scenario where we commit some records during one AJAX call in the DB using Hibernate, but the immediate next AJAX call could not show it on the same page. We had committed the data properly, but the record refuses to show up in the list.

We did not have any second level cache, so we were debugging it based on the Session closure and opening. Issue turned out to be that the proper transaction isolation level was not set in hibernate.cfg.xml file.

I am not going to go in depth on Transaction Isolation levels here, which you can read from here

The issue was resolved after we set the Isolation level with the following line in the config XML :
<property name="hibernate.connection.isolation">2</property>

An explanation on the numbers to be used for your level can be found here (look for isolation)

Issues with SVN - CheckSum Error

You might have seen this weird Error that SVN throws up when a file checkin is aborted in while in progress. This leaves the ".svn" folder in your local setup in an inconsistent state. Faced this a couple of times in the last few days and this is what I did to get rid of the Error

On SVN Repository exploring (Eclipse) choose the immediate parent directory, do a checkout
This folder will now get created as an Eclipse Project in your Work space
Copy the directory path of this folder, "cd" to the folder.
Copy the ".svn" folder to the original folder which was corrupted
Restart Eclipse - Error got resolved.

Hope this helps !