05. Summary of Portal Users Pretest

Pre-test (usability test) of Pronett version 1.0 by Ferrie Wolff

december 2002

This report describes the prodecures and results of the Pronett Portal PreTest. The target of this inquiry is to find out where the main problems are in using the portal facilities so big issues can be solved before its first use in Cardiff. It is also intended to make clear what kinds of activities within Pronett are difficult so the developers can focus on those problems.

There is little information about doing usability tests on WebPages. In this research various techniques have been used, such as observation, interviews and questionnaires. The collective information thus acquired gives an indication of the problems in using this webportal.

Government webpages have to be "Bobby approved". That means that users with visual disabilities can use this site also. W3C has made a guideline with priorities.....

Before the test there have been made 5 categories; composition (design), tasks, functionalities (of Pronett), global problems, and other (such as missing things). Also a list has been created with tasks that a selected group of people in Utrecht, The Netherlands have made. In carrying out these tasks testees perform the most common actions in the Pronett Portal. The persons who joined this test all have different backgrounds. Some are students, some are teachers, and they all had different experiences with computer use.

During the test, there were 3 observers who wrote down all the observations they made, and assigned them to the categories mentioned earlier. After the test, the results were discussed with testees and observers and conclusions were collectively drawn made. With the use of this method there were three different views and the real problems where made clear.

Most problems were in the design category because everybody is confronted with itsees. There were problems with resolutions and webbrowers, buttons that were not seen and lists that weren’t understood.

A huge problem in the tasks was the capital sensitivity in the registration process. A 0 (zero) and O weren’t distinguished, neither were 1 (one) and l (L). Another big issue was the publishing process of documents. The meaning of this was unclear and a lot had to be done before the publishing was a fact. Projects have to be published separately to become visible in the Project Catalogue.

It’s also possible for members to invite people to their workspace. It’s not clear that this is possible within Pronett, and the working of this process is also difficult. These problems are resolved by the use of text on the specific page and explanation has been putted in the user guide), but must be made better version 2 of Pronett.

The visibility of the Roundtable was terrible, the structure of the messages was inconsistent and some messages couldn’t be found back. For version 2 there will be used a forumtool that has been developed to fit Plone. For now surrounding the messages with a border solves the problem.

At this moment Pronett has a "Bobby A-status" which means that the essential elements are correct for user with disabilities. But for version 2 this has to be improved, to attain at least a Bobby AA-status. This is a government ruling.

The state of a document wasn’t clear. So additional information is being given that tells the user what kind of object he/she is working on, and in what kind of state it’s in.

Problems with permissions have been resolved. Each different role within Pronett has now its own permissions and functionalities as described in the specifications.

Also there have been some colour changes to give more accent to some important elements, such as the working space and the favourites box.

There are some issues that have been resolved by the use of some text on the specific page. We have not been able to resolve every problem. Therefore we want to collect the opinion and experiences from the users in Cardiff and how they see the problems must be resolved, because they are the first users and they will have worked with Pronett for a while. A questionnaire is one of the instruments that will be used for datacollection.