NB 6.0 Download Page Usability Study Report
Author: Jakub Franc
Study facilitators: Jakub Franc, Jan Rojcek
Study date: 21 March 2007
Table of contents
- Executive Summary
Usability study of NetBeans 6.0 download page mock-up revealed that user experience of NetBeans IDE download improved significantly. Several minor problems associated with wording, page layout and new packs concept were identified.
Early prototype of NetBeans 6.0 download page was tested in this usability study. Research sample consisted of 6 participants (see detailed description of participants) representing several strategic target groups:
- current NetBeans user
- competitive IDE user
- C developer
The only task (see the original wording) for the participants was to download NetBeans 6.0 to find out whether this IDE could substitute their current IDE well in their current working conditions. Participants were interviewed after each test session.
The main goal of the study was to evaluate whether the new concept of NetBeans IDE download is comprehensible for the (potential) users.
New concept of download was perceived generally very positively by our study participants ("very good", "well arranged", "much better", "standard", "just two pages"). The concept of two main download options was understood almost immediately by all participants.
The standard version seemed to be less downloaded than the custom one. The standard version was chosen only by a novice to Java. All others chose custom version regarding technologies they use.
Two participants downloading custom version were slightly slowed down because of confusion caused by highlighted download button in standard version column. They tended to click on this button instead of clicking on download button associated to custom version. It took them few moments to realize that. Standard version button is highlighted although it is used less than custom one. A visual link between columns of provided functionalities and download button is missing.
A participant did not notice that the checkboxes in custom version column are active. Thus he left the undesired functionalities checked.
A participant complains that he misses screenshots of particular components on the download page. He finds screenshots much more informative than words.
Couple of participants complained that functional dependencies between particular components were not signalized well. They would prefer to see both dependent components highlighted (not only the one) or marked by star. They were slowed down when finding out what is wrong and what other component do they need.
Several participants did not expect enterprise functionality to be a part of Web pack. Although they were enterprise developers they did not download Web pack expecting it does not provide functionality they need. Web is usually associated with deployment. Several participants associated Web pack with visual tools.
Profiler is not expected to be part of "Java IDE". Thus we could hear several times: "And where is Profiler? I would like to download it, too!"
WTK and Glassfish are not understood by newbies to NetBeans. They ask for explanation of these terms. They tend not to download them (even when needed) on the first try. Glassfish was confused with Looking Glass twice.
A participant reported problems in compatibility of GlassFish 9 with JES. He would like to know what version of GlassFish he is going to download. It is not explicitly mentioned there.
A C developer downloaded besides C/C++ all parts of standard version ("to be sure it will works" - he perceives them all as necessary) including Mobility pack he does not need.
Several participants (especially those experienced with Eclipse) call packs "plug-ins".
PHP support in NetBeans was perceived rather ambiguously. A participant finds PHP as a "quick and non-reliable technology for students of second grade" and is not interested in. However another participant appraises PHP support in NetBeans very positively. Part of his working team is using PHP and he likes the idea that the whole company could use just one common tool. He hopes that PHP support will be on sufficient level that his colleagues could switch from their current tools.
Ruby seems to be not widely use among our study participants. However Ruby is perceived as promising technology. Several participants know Ruby on Rail, mainly because its easy prototyping.
All participants would like to try the IDE to evaluate it, provided description is not sufficient for them. A participant (team leader) describes his principles of IDE evaluation: "First of all I need to play with that a bit to see how it behaves, if it is easy to use. Supported technologies? That is the last thing I take in consideration."
Our participants reported that size of downloaded software and duration does not usually influence their decision whether to download it or not. Internet connection in their offices was described as "good" or "sufficient enough", "I do not even know. It takes couple minutes every time I download some software" (Most participants do not download job related software at home).
Even worsened performance is not generally taken as an essential factor in deciding what functionality of the IDE to download. However all participants expect an "easy and quick" way of switching functionality they are not eventually interested in. Some participants missed the information this option would be provided.
|Programing experience||IDE||Applications developed||Status|
|P1||Java - 5 years||Eclipse||web||professional (team leader)|
|P2||Java - 6 years||NetBeans 5.5||enterprise, web, ME||professional (what we call "decission maker")|
|P3||Java - 6 years||NetBeans 5.5, Eclipse 3.2.1||enterprise, web, database||profesional|
|P4||C - 8 years||K Develop||mathematical||student|
|P5||Java - 8 years||Eclipse, NetBeans, J Edit||enterprise, desktop||profesional|
|P6||PHP - 1year, Java - 6 months||NetBeans 5.5||web||student|