Case Study: Testing a Responsive Website

Recently we were in a position to build our first attempt at a responsive website that will be managed through our CMS, TerminalFour Site Manager. For our users, as of yet nothing is different, but for the visitors things were about to change. Whether for better or worse, that is something we can only guess at.

As part of the process of planning our responsive site, we collaborated with Keith Doyle  – a consultant in responsive user experience and in information architecture from Navopia – and through his contacts we were able to have a usability study carried out by WhatUsersDo.

Background

This was the first responsive website that we’ve done at the University of Bradford, and as such we were on unfamiliar ground. We have read the articles, seen the websites and tested various other responsive sites; however we have not seen what we class as a ‘content’ website with deep navigation before so we do not know if our ideas will work or not.

Just by looking at our basic set of analytics, we have measured the growth in mobile use across all of our sites and determined that this is something that we simply have to move with. To give our growing mobile visitors a better, tailored experience is something that as an institution we need to do now. A recent study by HEERA, higher education external relations association, showed that across a range of HE websites the numbers were following the same trends as commercial websites.

Various other pieces of research predict that mobile users will overtake desktop users some point in the very near future, some point in 2014 actually.

The Study

For information about how the user tests themselves were carried out, please view Keith’s post on the WhatUsersDo website. There he will outline more about the users, how the test was carried out and some of his conclusions based on a usability perspective.

I on the other hand, shall mention our findings based on a HE perspective. The test itself was a small one covering desktop, mobile (iPhone/Android) and tablet (iPad) based on a set of tasks to perform on our online Undergraduate Prospectus.

Our Findings

Overall, I have to say that I was pleased with the results of the videos from these tests. The site seemed to be easily usable, readable and better received than the non-responsive sites that the testers occasionally came across during the tests.

In general all looks good, but there was a lot of small issues that the tests did highlight that we can learn from.

Flash

Although we are trying to not include flash on our mobile sites, this test did highlight just why. We missed one area which pulls in a Flickr feed of images and yes, it did not load up on iOS and instead presented a large black area – this will be rectified.

Video

This is similar to the Flash issue, in that of course iOS won’t see them so you have to make sure that the video uses a format that is viewable. For some, the decision to not have video embedded may be used here, however our stats initially showed that most of our visitors were using WiFi or at least 3G and so we left them embedded. It is therefore the users’ decision whether they want to stream it or not. We are actually in the middle now of creating a new template to allow YouTube (Iframe) that will make the most of HTML5 and fallback to other supported videos dependent on what can then be displayed.

The fact that the stats showed WiFi and 3G also factored into how we treated our content.

Content

We class our site as a ‘content heavy’ site and as such have chosen to not hide any text – that would make it a mobile version which is not our aim – however we do still need to think about the structure of our content; which becomes evident when you see people scrolling up and down on each page (actually, most users did this as soon as page loaded almost as if they were giving themselves the overall picture of the page before then looking at the content).

The opening paragraph of the page needs to be more of a leading paragraph full of keywords relating to the page content (added bonus for SEO I’m sure) so that the user can make up their mind if this is the right page for them or not.

If the page is a ‘signpost’ page linking to other areas of the site, then this information should be right at the top to save unnecessary scrolling.

From the videos we saw, no one commented about the content being too much, in fact quite the opposite as when viewing the course information in particular there was comments made that they expected more information.

Banners

I have seen the stats on our analytics for our desktop sites and I am aware that users do click on banner links and graphics, yet for some reason I did not expect the same for the mobile view.

However I was quite surprised when two of our testers actually used the banner as a source of main navigation to areas within the site. I was contemplating removing this feature for mobile view, but now on hindsight I think more emphasis on the content in here needs to be made and it should be treated more like a navigation area instead of just a ‘fancy’ article area.

Navigation / Menu / links

This for me is the one main thing that I have learned from watching these videos, the naming of our links and how we set up the menu areas is vital.

With desktop, you see everything (usually at least the navigation and leading content) when the page loads. This helps give context as to where you are, what the other pages in this section/parent sections are and also by good designs and layout you can convey some kind of order and structure by feel alone.

With mobile views, this is much harder to achieve.

For example, our site as a whole is very large, and the UG section is but one area albeit the size of most sites itself. So we tried to get the feel that once yo are in the UG site on mobile, everything you see link wise is relating to this site. By this we got rid of the ‘top level’ menu items and presented them in a drop down menu titled ‘main university links’ to show that if these are clicked you are leaving the UG site and so if the layout and style changed it was not a major shock.

To continue this feeling of being in the UG site we chose to have the breadcrumb more prominent to act as a ‘you are here’ signpost on all pages to immediately show the user where they were. No-one commented on this, but also no-one commented that they were lost either so I think that it did its job.

However, when we asked our testers to find specific information instead of just letting them browse the main navigation issues became apparent.

For example:-

Task: “Can you tell us about specific research going on at the University”

Results: Some were unable to find this information even though in the navigation there is a link titled “Inspiring Lecturers and Researchers”

It seems that as the title does not explicitly say ‘Research’, when scanned on a mobile it can be overlooked, in fact most then clicked on the aforementioned main university links and went to the University Research pages and not the targeted UG ones.

This happened also with a few other tasks.

Plans: based on this we are now going to look at how we do our main navigation. Instead of a list of links, we are going to group them under headings, effectively making 3 or 4 groups of links. With a proper descriptive title, and better wording on the links this will hopefully help the users find exactly what they want.

Conclusion

There are many more little titbits that the test has highlighted to us, and for that reason I highly recommend you doing something like this on your own sites. Just to be given the opportunity to see how other people browse your site on these different devices is fascinating, and the little things that you are ok/not sure of may or may not be the same for other users. The only way to find this out, is to do something like this.

I will end this post with one comment from a user, which for me shows just why we put the time into a responsive design in the first place:

“…very attractive as a student to see it working on my Iphone, and its an institution that relates to me as it can work in my format and does appreciate what I want out of a university and how it can benefit me.”