WebApps Project Usability Study Report

Author: Jan Rojcek

Study Dates: May 20th - 21st, 2004

Table of Contents:

Executive Summary

Usability study of NetBeans IDE based on new build system with WebApps project type showed that users were able to easily and quickly create simple web application. Participants positively commented the code completion in JSP files and possibility to debug JSPs. The most important problems found were related to creating new project from existing sources and library management.

The most visible problems included:

  • Participants didn't know how to add a library to project.
  • Participants didn't know how to use existing sources in NetBeans.

Introduction

The study was focused on observing the first hour experience when using NetBeans IDE based on new build system with WebApps project type. Eight participants were asked to try to perform 7 tasks using NetBeans IDE based on new build system. All usability issues observed while participants were performing the tasks were noted down.

Participants

We could categorize participants according to JSP/Servlet programming experience as:

  • 1 non-JSP/Servlet programmer - P8
  • 2 junior programmers - P4, P7
  • 5 experienced programmers - P1, P2, P3, P5, P6

The full participant characteristics can be found here: Participants[Fix the link].

Tasks Summary

Eight tasks were focused on testing following:

  • Task 1: Browse the IDE
  • Task 2: Create and launch a Hello World web application
  • Task 3: Create and launch a more complicated web application
  • Task 4: Create a web project from existing sources
  • Task 5: Modify the source code of web application
  • Task 6: Use custom tag library
  • Task 7: Describe how you would use NB with your current project

The full task's script can be found here: Task Script.

Task Analysis

Task 1: Browse the IDE

Objectives

  • Observe participants' behavior when browsing the IDE for the first time.

Notes

  • P1 clicked links in the welcome screen and created and ran a new Java Application without problems.
  • P2 created a new Web Application but didn't launch it.
  • P3 clicked links in the welcome screen and noticed that this NB already has projects.
  • P4 clicked links in the the welcome screen and observed the menus thoroughly.
  • P5 invoked New Project wizard and looked into the menus.
  • P6 created and ran new Java Application without problems. He also ran the project in debugger.
  • P7 created a new Java Application and browsed the menus.
  • P8 browsed the main menus.

Task 2: Create and launch a Hello World web application

Objectives

  • Test creating and launching a simple web application.

Expected Steps

  • Invokes File | New Project
  • Selects "Web Application" template
  • Optionally modifies the project name
  • Optionally modifies the project location
  • Optionally modifies the context path and J2EE specification level.
  • Finishes the wizard
  • Modifies the generated index.jsp
  • Invokes Run | Run Main Project (or through the main toolbar)

Goal

  • G2.1: 80% of participants would use the expected steps to complete the task.
  • G2.2: 100% of participants would complete the task without facilitator's help.

Results

  • G2.1: Failed (62.5% passed - P1, P4, P8 failed)
  • G2.2: Passed (100% passed)

Notes

  • P1, P4, P8 Failed - participants didn't notice generated index.jsp and created a new start page. [WP-1]
  • P1 was creating a new JSP document and in the new file wizard checked the "Create as JSP Document (XML Syntax) even though he didn't understand the XML syntax and had to create another JSP afterwards. [WP-2]
  • P1 didn't know how to run his own web page which wasn't set as the main page of project. So he supplied the page in browser's address field. [WP-3]
  • P1 compiled JSP with wrong tag and the output window showed a link to build-impl.xml. [WP-4]
  • P2 and P5 couldn't find a way how to turn off Tomcat. [WP-5]
  • P3 mentioned that port 8084 was "non-standard". [???]
  • P4 created index.html.html. [WP-6]

Task 3: Create and run a more complicated web application

Objectives

  • Test creating of JSP file.

Expected Steps

  • Modifies index.jsp using code completion.
  • Invokes File | New File (or contextual alternative)
  • Selects "JSP" template
  • Fills in a JSP file name.
  • Finishes the wizard.
  • Modifies the JSP file using code completion.
  • Invokes Run | Run Main Project (or through the main toolbar)
  • Optionally iterates between modifying files and running project.

Goal

  • G3.1: 80% of participants would use the expected steps to complete the task.
  • G3.2: 100% of participants would complete the task without facilitator's help.

Results

  • G3.1: Failed (37.5% passed - P1, P2, P3, P4, P5 failed)
  • G3.3: Passed (100% passed)

Notes

  • P1, P4 Failed - participants didn't have the main page set, so they modified the address value in web browser. [WP-3]
  • P2, P5 Failed - participants created JSP with double extension something.jsp.jsp. [WP-6]
  • P2 Failed - participant created JSP with XML syntax, but he wanted just plain JSP, so he deleted the file and created new one. [WP-2]
  • P3 Failed - participant created JSP with XML syntax, but he wanted just plain JSP. He thought the problem was in the J2EE level of project, so he closed this project and created new one. [WP-2]
  • P6 had to redeploy the project, because it didn't appear correctly in the web browser after Run Project. [???]
  • P1, P4, P7, P8 created only static pages without passing the name parameter. [???]

Task 4: Create a web project from existing sources

Objectives

  • Test creating a web project from existing sources.

Expected Steps

  • Invokes File | New Project
  • Selects "Import Web Module" template
  • Fills in "C:\Work\FlipCoin" location
  • Optionally modifies the project name
  • Finishes the wizard
  • Invokes File | Project Properties (or contextual alternative)
  • Clicks Add Library button in Compiling Sources category
  • Selects JSTL1.1 library and adds it to classpath
  • Closes the Project Properties
  • Optionally invokes Build | Build Main Project (or through the main toolbar)
  • Invokes Run | Run Main Project (or through the main toolbar)

Goal

  • G4.1: 80% of participants would use the expected steps for importing web module.
  • G4.2: 80% of participants would use the expected steps for adding the library to project.
  • G4.3: 100% of participants would complete the task without facilitator's help.

Results

  • G4.1: Failed (50% passed - P2, P4, P6, P8 failed)
  • G4.2: Failed (37.5% passed - P3, P4, P5, P6, P8 failed)
  • G4.3: Failed (25% passed - P2, P3, P4, P5, P6, P8 failed)

Notes

  • P1 invoked Properties from the Files window and it opened the Property Sheet.
  • P2, P4, P6, P8 Failed - participants created new Web Application and wanted to "add directory" into this project. [WP-7]
  • P2, P4, P6, P8 Failed - facilitator had to point participant to correct project template. [WP-7]
  • P2 had to use Redeploy because the web app didn't work after adding the library. [???]
  • P3 selected FlipCoin/web folder as web module root in Import Web Module wizard. [WP-8]
  • P3, P5, P6, P8 Failed - facilitator had to explain how to add the library to project. [WP-9]
  • P3 was confused from "Edit Library" button in Project Properties. [???]
  • P4 selected WEB-INF folder as web module root in Import Web Module wizard. [WP-8]
  • P4, P6 Failed - participant added library to Packaging list. [WP-10]
  • P6 wanted to change Project Location for imported web module, but Finish was disabled without notifying why. [WP-11]
  • P7 selected Web Application template first, but returned from the second panel and picked a correct template. [???]

Task 5: Modify the source code of web application

Objectives

  • Test the code completion usage and regular editor usage.
  • Test repeated project launching.

Expected Steps

  • Modifies the source code using code completion
  • Invokes Run | Run Main Project (or through the main toolbar)
  • Optionally iterates between modifying files and running project.

Goal

  • G5.1: 80% of participants would use the expected steps to complete the task.
  • G5.2: 100% of participants would complete the task without facilitator's help.

Results

  • G5.1: Passed (100% passed)
  • G5.2: Passed (100% passed)

Notes

  • P1 used debugger because Run didn't work at first (because of a bug in Run?). [???]
  • P3 wanted to stop "container" but couldn't find how. [???]
  • Run didn't work for P3, P4 (because of a bug?). [???]
  • P4, P6 used debugger. [???]
  • P5 was looking for "hot deploy" feature - compiling the jsp should upload it? [???]
  • P5 wasn't clear in what was really being done by Run vs. Redeploy. [???]
  • Code completion didn't work in jsp for P6. [???]

Task 6: Use custom tag library

Objectives

  • Test using a custom tag library in project.
  • Test using the code completion with custom tag library.

Expected Steps

  • Invokes File | Project Properties (or contextual alternative)
  • Clicks Add JAR button in Compiling Sources category
  • Locates the JAR file and adds it to the classpath
  • Closes the Project Properties
  • Modifies the source code using the code completion
  • Runs the project

Goal

  • G6.1: 80% of participants would use the expected steps to complete the task.
  • G6.2: 100% of participants would complete the task without facilitator's help.

Results

  • G6.1: Failed (75% passed - P4, P6 failed)
  • G6.2: Failed (75% passed - P4, P6 failed)

Notes

  • P1 wanted to backup index.jsp and copied it into project root what resulted in exception dialog. [???]
  • P1 created double extension jsp index.jsp.jsp when he wanted to backup the file. [WP-6]
  • P1 wanted to browse the library to read the URI and found it in All Files window. [???]
  • P3, P5, P7 wanted to browse the library to read the URI but couldn't find it. [WP-12]
  • P4, P6 Failed - participant added JAR to the Packaging list. [WP-10]
  • P4, P6 Failed - facilitator had to explain where to add the JAR file. [WP-10]

Task 7: Describe how you would use NB with your current project

???

Findings and Recommendations

[WP-1] Automatically created JSP wasn't expanded in Projects window nor opened in the editor

3 participants hadn't noticed that index.jsp was created together with new web application and created additional start pages.

Severity

Because of this problem:

  • 3 participants were significantly slowed down in completing Task 2

Recommendation

Expand the automatically generated index.jsp in the editor.

[WP-2] "Create as JSP Document (XML Syntax)" check box wasn't understood

3 participants checked the check box even though they wanted to create only plain JSP. Participant didn't understand the XML syntax.

Severity

Because of this problem:

  • 1 participant was significantly slowed down in completing Task 2
  • 2 participants were significantly slowed down in completing Task 3

Recommendation

Reword the checkbox label and move it to less prominent location.

[WP-3] Run file was hard to find

3 participant couldn't find how to launch one file in browser, so they filled the address field in browser manually.

Severity

Because of this problem:

  • 1 participant was significantly slowed down in completing Task 2
  • 2 participants were significantly slowed down in completing Task 3

Recommendation

???

[WP-4] Output window showed link to build-imp.xml

Output window showed a link to build-impl.xml when participant compiled the source code with wrong tag.

Severity

Because of this problem:

  • 1 participant was confused when performing Task 2

Recommendation

Don't show the links to metadata files in Output window.

[WP-5] Hard to find how to turn off Tomcat

2 participants were looking for a way how to turn off Tomcat but couldn't find any.

Severity

Because of this problem:

  • 2 participants were confused when performing Task 2

Recommendation

???

[WP-6] New file wizard created files with double extensions

4 participants created file with double extensions.

Severity

Because of this problem:

  • 1 participant was slowed down in completing Task 2
  • 2 participants were slowed down in completing Task 3
  • 1 participant was slowed down in completing Task 6

Recommendation

If the user supplies extension in the file name then don't append another one automatically.

[WP-7] Web Application project was the first choice when creating a project for existing sources

4 participants created Web Application and afterwards were looking for a way how to add existing sources into the project.

Severity

Because of this problem:

  • 4 participants couldn't finish Task 4

Recommendation

???

[WP-8] "Web Module Root" button was confusing when locating existing web module

1 participant specified "web" folder and 1 participant "WEB-INF" folder as web module root in Import Web Module wizard.

Severity

Because of this problem:

  • 2 participants were significantly slowed down in completing Task 4

Recommendation

???

[WP-9] Adding library to project is confusing

4 participants couldn't find how to add a library to project.

Severity

Because of this problem:

  • 4 participants couldn't finish Task 4

Recommendation

???

[WP-10] Packaging project was preferred choice when adding library to classpath.

4 participants added library to packaging list instead of compiling classes.

Severity

Because of this problem:

  • 2 participants couldn't finished Task 4
  • 2 participants couldn't finished Task 6

Recommendation

???

[WP-11] Import Web Module wizard didn't show why wizard buttons were disabled

Wizard button didn't show inline error messages explaining why the button were disabled.

Severity

Because of this problem:

  • 2 participants were slowed down in completing Task 4

Recommendation

[WP-12] Impossible to browse library contents

3 participants wanted to browse library contents but couldn't find where.

Severity

Because of this problem:

  • 3 participant were slowed down in completing Task 6

Recommendation

???

Project Features

About this Project

ui was started in November 2009, is owned by Jiří Kovalský, and has 43 members.
By use of this website, you agree to the NetBeans Policies and Terms of Use (revision 20160708.bf2ac18). © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo
 
 
Close
loading
Please Confirm
Close