NB 6.0 Download Page Usability Study Report

Author: Jakub Franc
Study facilitators: Jakub Franc, Jan Rojcek
Study date: 21 March 2007

Table of contents

• Executive Summary
• Introduction
• Findings
  • Overall evaluation
  • Names of packs regarding to their content
  • Comments on couple particular technologies

• Process of IDE evaluation
• File size / Download duration

Executive Summary

Usability study of NetBeans 6.0 download page mock-up revealed that user experience of NetBeans IDE download improved significantly. Several minor problems associated with wording, page layout and new packs concept were identified.

Introduction

Early prototype of NetBeans 6.0 download page was tested in this usability study. Research sample consisted of 6 participants (see detailed description of participants) representing several strategic target groups:

• current NetBeans user
• competitive IDE user
• C developer

The only task (see the original wording) for the participants was to download NetBeans 6.0 to find out whether this IDE could substitute their current IDE well in their current working conditions. Participants were interviewed after each test session.

The main goal of the study was to evaluate whether the new concept of NetBeans IDE download is comprehensible for the (potential) users.

Findings

Overall evaluation

New concept of download was perceived generally very positively by our study participants ("very good", "well arranged", "much better", "standard", "just two pages"). The concept of two main download options was understood almost immediately by all participants.

The standard version seemed to be less downloaded than the custom one. The standard version was chosen only by a novice to Java. All others chose custom version regarding technologies they use.

Two participants downloading custom version were slightly slowed down because of confusion caused by highlighted download button in standard version column. They tended to click on this button instead of clicking on download button associated to custom version. It took them few moments to realize that. Standard version button is highlighted although it is used less than custom one. A visual link between columns of provided functionalities and download button is missing.

A participant did not notice that the checkboxes in custom version column are active. Thus he left the undesired functionalities checked.

A participant complains that he misses screenshots of particular components on the download page. He finds screenshots much more informative than words.

Couple of participants complained that functional dependencies between particular components were not signalized well. They would prefer to see both dependent components highlighted (not only the one) or marked by star. They were slowed down when finding out what is wrong and what other component do they need.

Names of packs regarding to their content

Several participants did not expect enterprise functionality to be a part of Web pack. Although they were enterprise developers they did not download Web pack expecting it does not provide functionality they need. Web is usually associated with deployment. Several participants associated Web pack with visual tools.

Profiler is not expected to be part of "Java IDE". Thus we could hear several times: "And where is Profiler? I would like to download it, too!"

WTK and Glassfish are not understood by newbies to NetBeans. They ask for explanation of these terms. They tend not to download them (even when needed) on the first try. Glassfish was confused with Looking Glass twice.

A participant reported problems in compatibility of GlassFish 9 with JES. He would like to know what version of GlassFish he is going to download. It is not explicitly mentioned there.

A C developer downloaded besides C/C++ all parts of standard version ("to be sure it will works" - he perceives them all as necessary) including Mobility pack he does not need.

Several participants (especially those experienced with Eclipse) call packs "plug-ins".

Comments on couple particular technologies

PHP
PHP support in NetBeans was perceived rather ambiguously. A participant finds PHP as a "quick and non-reliable technology for students of second grade" and is not interested in. However another participant appraises PHP support in NetBeans very positively. Part of his working team is using PHP and he likes the idea that the whole company could use just one common tool. He hopes that PHP support will be on sufficient level that his colleagues could switch from their current tools.

Ruby
Ruby seems to be not widely use among our study participants. However Ruby is perceived as promising technology. Several participants know Ruby on Rail, mainly because its easy prototyping.

Process of IDE evaluation

All participants would like to try the IDE to evaluate it, provided description is not sufficient for them. A participant (team leader) describes his principles of IDE evaluation: "First of all I need to play with that a bit to see how it behaves, if it is easy to use. Supported technologies? That is the last thing I take in consideration."

File size / Download duration

Our participants reported that size of downloaded software and duration does not usually influence their decision whether to download it or not. Internet connection in their offices was described as "good" or "sufficient enough", "I do not even know. It takes couple minutes every time I download some software" (Most participants do not download job related software at home).

Even worsened performance is not generally taken as an essential factor in deciding what functionality of the IDE to download. However all participants expect an "easy and quick" way of switching functionality they are not eventually interested in. Some participants missed the information this option would be provided.

Appendix

Participants table