Java EE 5 Usability Study Report

Author: Jan Rojcek, Jakub Franc
Study Facilitators: Jakub Franc, Jan Rojcek
Study Dates: 8 - 10 March 2006

Table of Contents:

Executive Summary

The usability study was focused on testing the new Java EE 5 functionality in NetBeans targeted for Promotion G (NetBeans 5.5). Our goal was to observe how the IDE helps users try and learn new Java EE 5 features (EJB 3.0 Session Beans and Java Persistence) using the NetBeans IDE. In the study we asked 8 participants to solve simple tasks (pdf) using Java EE 5 and NetBeans IDE.

As the study required basic knowledge of the upcoming Java EE 5 technology, the study facilitator explained the basics of Java EE 5 needed to complete the tasks to all participants. Six participants (P3-P8) were given a Java EE 5 one pager (pdf) about Java EE 5 and 2 participants (P1, P2) were provided with direct help from facilitator when they didn't know how to proceed because of not knowing Java EE 5.

The positive result from the study was that we could see some progress with participants getting familiar with Java EE 5 during solving the study tasks. In the task #3, we observed the participants were more confident and effective in using Java EE 5 features and editor features like Fix Imports, Encapsulate Fields and Editor Hints.

On the negative side, participants completed on average 2 coding tasks (task #2 and #3) in about 1 hour and 15 minutes. One of the tasks was to create a hello world enterprise application, the other task was to store 5 instances of entity class into the database. The hello world task took about 40 minutes on average. In addition, participants were stuck and couldn't proceed with the tasks without facilitator's help about 20 times during the whole study (this doesn't count failures due to not knowing Java EE 5). There were multiple reasons why participants were struggling so much:

  • NetBeans didn't provide the same UI for Java EE 5 as it had for J2EE 1.4 applications. Participants wanted to use features they were used to use in J2EE 1.4: Add Business Method dialog (see findings[EE-2], [EE-7]) and Call EJB dialog [EE-9]. Instead of using the Add Business Method dialog, participants either used the Add Method dialog [EE-8], which didn't add the method declaration to the interface, or they wrote the methods manually without using features like Pull Up [EE-11] or Hint for Overriding Methods. Instead of using the Call EJB dialog, participants wrote the code manually, which didn't work because the EJB module wasn't on Web module's classpath [EE-4].
  • Old usability problems accumulate. Participants hit old usability problems in Java EE modules but also in other modules. They encountered problems in editor that were already recorded in the editor usability study (see editor study findings [ED-12], [ED-13], [ED-15], [ED-18], [ED-24]). They had significant problems with adding project to classpath [EE-14]. And finally, they experienced problems in Java EE modules when connecting to database [EE-24]; they didn't know how to undeploy application [EE-23]; and the deploy action didn't open up the web browser for them [EE-5].
  • NetBeans and Application Server provided the user with very poor help to recover from errors in the source code that were not captured with compilation. In many cases participants built the application successfully, but then faced errors during deployment in the build output window [EE-32], [EE-20], [EE-22], or during run-time in the web browser and server log [EE-12], [EE-29]. In general participants didn't understand those errors because the error description poorly described the root cause, the error was poorly formatted in the build output window, or the root cause was hidden in the server log participants didn't even look into. A separate category of problems were exceptions shown in the web browser window. Those exceptions either didn't provide any description or provided very poor description that didn't lead participants to the root cause. There was a note below the exceptions shown in the web browser to look into the server log for the root cause. Participants either didn't see the note or they looked into the server log which provided a huge list of messages and exceptions which was very hard to scan and understand. In many cases it looked like a team exercise where participant together with facilitators and observers (developers) tried to solve the problem. In same cases we just failed.

After the study we briefly interviewed participants. This is the summary of the interview:

  • Code completion and automatic generation of the code when creating new project were identified as the most helpful actions in supporting user in learning Java EE 5.
  • Most of the participant were not satisfied with the number of completed tasks, they considered themselves as too slow and unprofessional. They reported that Java EE 5 was too difficult for them to learn it and use it just with help of the IDE.
  • Reading tutorials and then practicing within the IDE seemed to be the common way of learning new technologies.
  • Current NetBeans users were complaining that it was not possible to create business methods "by the right click" as they were used to.

To fix the usability problems revealed in this study, we suggest to use the same UI for Java EE 5 applications that was provided for J2EE 1.4 applications when it makes sense. We also suggest to focus very closely on deployment and runtime error notifications as this seems to be a very big usability problem. And finally, we suggest to start fixing the old usability problems as they really accumulate and significantly affect user experience.

For the good learning experience of Java EE 5 in NetBeans IDE, we suggest to bundle Java EE 5 tutorial and sample projects in the IDE and put it on a prominent place (welcome screen, help menu, etc.). As part of the 5.5 adoption, we suggest to put "Java EE 5 in NetBeans" tutorials on the main developer sites (java.sun.com, etc.) and Google if possible as more participants said they "just search for the tutorial on the web".

Findings and Recommendations

[EE-1] New session bean isn't selected in Project window

Finding
  • Participant created a new session bean but it wasn't selected in Project window.
Severity
  • All participants had to manually expand the tree to locate the session bean.
Recommendation
  • Select the session bean in project window after creating it. This should work like for all other created files.

[EE-2] No way to add business method using contextual menu on file node

Finding
  • Participant wanted to add a business method using the contextual menu on bean file node.
Severity
  • 2 participants searched for this feature
Recommendation
  • Design and implement this feature

[EE-3] Missing container node for Enterprise Beans

Finding
  • Participant (NB5.0 user) searched for Enterprise Beans node where he wanted to select a session bean and add a business method.
Severity
  • 1 participant deleted the session bean and went to create it again
Recommendation
  • Consider providing the session bean file node with special "bean" icon. Put something like "Add Business" method on this node. Also hide the Web Services and Server Resource nodes if empty to show the difference between 5.0 and 5.5.

[EE-4] EJB module isn't on classpath of Web module

Finding
  • Participant wrote the EJB injection code manually in Servlet (to inject the bean interface). He didn't realize that the EJB module wasn't on classpath.
Severity
  • 1 participant used Fix Imports dialog which said that no imports were found for the bean interface. He decided to fix it by writing the import manually.
  • 3 participants wrote the FQN manually which didn't fix the problem.
  • 1 participant tried to write import manually with code completion which didn't work.
Recommendation
  • Consider providing a "Call EJB" kind of contextual action that generates the injection code and adds the EJB module to classpath.
  • Consider providing an editor hint for adding the EJB module to classpath. We have to scan all open projects (and required projects) anyway. So if we find the proper type in a project that isn't on classpath, we can offer to add it to classpath.
  • Consider adding the EJB module to classpath of web module if created from the enterprise application wizard.
  • Fix the wording in the information wording dialog. It says: "No imports found for the following types:". It should say something like "The following types are not present on project classpath:".

[EE-5] Deploy doesn't open up the web browser

Finding
  • Participant deployed enterprise application using the Deploy contextual action which didn't bring up the browser with application URL.
Severity
  • 2 participants had to write application URL manually.
Recommendation
  • Fix the naming of Deploy and Run actions. And make the current deploy action open the web browser.

[EE-6] Scanning in progress doesn't disappear

Finding
  • Participant wanted to finish the session bean wizard but the "Scanning in progress..." inline error didn't disappear even though the scanning finished. Thus the finish button was still disabled.
Severity
  • 1 participant had to cancel the wizard because of this problem.
Recommendation
  • This is a bug. The inline error should disappear when the scanning finishes.

[EE-7] Missing "Add Business Method" in contextual menu in editor

Finding
  • Participant (NB 4.1 user) looked for "Add Business Method" in contextual menu in editor.
Severity
  • 4 participants searched for this feature.
Recommendation
  • Consider adding some kind of "Add Business Method" item into the contextual menu in editor.

[EE-8] Using "Add Method" instead of missing "Add Business Method"

Finding
  • Participant didn't find "Add Business Method" in the editor so he used"Add Method" action on Method node underneath the beans file node in Project window. He thought this will create a business method.
Severity
  • 2 participants used this feature and expected it will add a regular business method
  • 1 participant didn't notice that the method wasn't added into the interface automatically which cause other problems later on.
Recommendation
  • Consider doing something with "Add Method", "Add Business Method", "Add Property" actions. They are very similar with similar results that can be confusing.

[EE-9] Using "Call Enterprise Bean" instead of writing injection manually

Finding
  • Participant (NB 4.1 user) wanted to use "Call Enterprise Bean" in servlet to look up the session bean. We had to stop him.
  • Participant (Eclipse 3.1 user) wanted to use contextual menu in editor to call EJB. He didn't find the right menu item.
Severity
  • 4 participants had to write the injection code manually instead of using a dialog.
Recommendation
  • Design and implement "Call EJB" dialog for Java EE 5.
  • Consider changing the "Enterprise Resources >" menu to something like "Add >".

[EE-10] Confusion from the light bulb tooltip

Finding
  • Participant hovered with the mouse above the light-bulb and it's tooltip said "Alt-Enter". So he hit Alt-Enter and read the message "Import ...." and then went to write the import manually. He didn't notice it was a menu. He thought it was a tooltip.
Severity
  • 1 participant didn't understand the function of the light bulb properly
Recommendation
  • Fix the tooltip and look of the list offering tips. If there is only one item it doesn't look like something to click on.

[EE-11] Cannot find Pull Up

Finding
  • Participant wanted to automatically add method to the classes interface. He was looking into the refactoring menu in editor for some time, but couldn't find what looked for.
Severity
  • 2 participants had to manually write the method into the interface
Recommendation
  • Consider changing the name Pull Up to something else.

[EE-12] Exceptions in web browser don't explain the problem

Finding
  • Participant ran web project from main toolbar (main project). This projects used a session bean from an EJB module that probably wasn't deployed even though "Package" checkbox was checked in Web module classpath. The browser showed up exceptions that didn't tell anything to the user about what the problem could be. The server log tab was covered by the build output tab. Participant didn't notice it.
  • Participant didn't understand the exception shown in web browser. Probably the root cause was that he injected session bean in servlet instead of its interface. In any case, the error message didn't help.
  • Participant didn't understand the exception shown in web browser. The root cause was missing persistence unit in the EJB module. Participant didn't understand the error message.
  • See screenshot 1, screenshot 2, screenshot 3, screenshot 4
Severity
  • 3 participants didn't know how to proceed
Recommendation
  • Show the user better description of errors.
  • Show the user better error description before (preferable) or during deployment if possible.

[EE-13] Missing "Add Method" on bean file node

Finding
  • Participant wanted to add business method into session bean using some kind of "Add Method" dialog.
Severity
  • 1 participant wanted to use a dialog for adding a business method but there wasn't any.
Recommendation
  • Consider adding an action for adding a business method into the bean file node's contextual menu.

[EE-14] Confusing how to add project to classpath

Finding
  • Participant wasn't sure how to fix the classpath problem. He looked into the project properties, then closed it. Facilitator had to tell him to use the libraries panel. Then he wanted to add a JAR file which didn't work either. Facilitator had to tell him again that he should add the project to classpath.
  • Participant wanted to use the Add Project dialog but closed it immediately after opening it. The he tried to Add JAR, but didn't know where the JAR was located.
  • Participant wanted to use the Add JAR/Folder dialog, but didn't know where the JAR was located. He didn't even click the Add Project button.
Severity
  • 3 participants didn't know how to proceed. Facilitator had to help him.
Recommendation
  • ???

[EE-15] Code completion works for unknown types

Finding
  • Participant invoked code completion with field that wasn't on classpath ("bean.|" - bean was a field of Test1Local type that wasn't on classpath). The code completion showed up offering basic items from Object, which was confusing because it looked like the completion didn't work properly.
Severity
  • 1 participant wrote the code manually without realizing that something was wrong.
  • A few other participants had the same problem.
Recommendation
  • The code completion should say something like "Test1Local isn't on project classpath." and ideally, it should offer to add the right project on classpath, if the project that contains Test1Local is in the reach of the IDE (the project is open or it is on classpath of open project?).

[EE-16] Hint is not well associated with error description

Finding
  • Participant had an error in the source code because he didn't import annotation "@EJB". He compiled the source code but didn't understand what was the problem (It said something about missing symbol). He looked at the tooltip on the error underline in the source code and didn't understand it either. He didn't notice or he didn't know how to use the light bulb.
Severity
  • 1 participant wasn't able to fix this simple error. Facilitator had to help him.
Recommendation
  • Even thought the error and fix was trivial and it might look like participants fault, the IDE could offer more help. The real problem is that we show errors in compiler output and in tooltips in editor and these errors are not presented together with editor hints. There's no real association of hints and errors.
  • Consider showing light bulbs next to the compiler error output.
  • Consider showing light bulb in an instant TODO window.
  • Consider showing some kind of pointer to light bulb in the tooltip.

[EE-17] No indication what's wrong if there's no index.jsp page

Finding
  • Participant added modules to empty enterprise application manually. Thus when he ran the application it didn't open up the web browser. Participant wasn't able to write the URL manually.
  • Participant deleted index.jsp page and when he deployed his application (that contained 1 servlet) he wasn't sure what was wrong. Participant wasn't able to write the URL manually.
Severity
  • 2 participants didn't know how to proceed. Facilitator had to help him.
Recommendation
  • The IDE should tell the user there's no index page when he runs the application.
  • The IDE should tell the user there's no URL set up.
  • Consider showing similar dialog to the dialog that shows up when user runs Java Application that doesn't have a main method set.
  • Consider showing a user friendly message in the browser saying something about missing index page or something like that.

[EE-18] Wizard layout jumps when inline error appears

Finding
  • The layout of Enterpriser Application wizard jumps when inline error appears and disappears.
Severity
  • 1 participant comments it is "weird jumping" .
Recommendation
  • The bottom part of the wizard panel (Server, ...) should be aligned to the top.

[EE-19] Error message not visible enough in Fix Imports

Finding
  • Participant used Fix Imports but it didn't do anything in the source code. He didn't notice the message is status line saying "Cannot perform Fix Imports on a source with syntax errors.".
Severity
  • 1 participant thought that the fix imports was successful.
Recommendation
  • Show the error message in a dialog.

[EE-20] Build fails with error message the user cannot understand

Finding
  • Participant deploys the enterprise application with EJB, Web and Client modules and gets the following error in build output: "...\build-impl.xml:229 java.net.MalformedURLException: no protocol: ${client.url}".
Severity
  • 1 participant didn't know how to proceed.
Recommendation
  • Show user friendly error messages.

[EE-21] Error when removing client module that doesn't exist from application

Finding
  • Participant deleted a client module that was part of an application. Then he wanted to remove the module in the J2EE Modules node, which threw exception and didn't remove it. He restarted the IDE and got the reference problem warning. The module wasn't shown under the J2EE modules node and the reference problems dialog didn't permit the user to remove it.
Severity
  • 1 participant spent 15 minutes on this problem with help from facilitator
Recommendation
  • It should be possible to remove missing module from J2EE Modules node and project properties and Broken References dialog.
  • Or the deleted module should be automatically removed from application.

[EE-22] Deployment error message not understood

Finding
  • Participant deployed an application and got the following error in build output. He didn't read the full error and didn't understand how to fix it.
    	<a lot of text here>
    	Trying to create reference for application in target server  failed;
          The context root [...] in application [...] <continues off the screen>
    	<hyperlink>...build-impl.xml:227: Deployment error:</hyperlink>
    	<red>The module has not been deployed.</red>
    	<red>See the server log for details.</red>
    	<red>BUILD FAILED</red>
      
    See screenshot
Severity
  • 1 participant didn't know how to proceed.
Recommendation
  • Improve formatting of the error output
  • Do not send the user to the server log.
  • Show only important messages in output window.
  • Improve the error message text.

[EE-23] Not clear how to undeploy modules

Finding
  • Participant didn't know how to undeploy existing modules. He went to the server manager. Facilitator had to point him to the Runtime tab.
  • Participant wanted to undeploy application from the contextual menu on project node in Project window.
Severity
  • 2 participants didn't know how to proceed.
Recommendation
  • ???

[EE-24] Not clear how to connect to database

Finding
  • Participant didn't know how to add a new database connection in Runtime window. Tried the contextual menu on Databases node and Drivers node.
Severity
  • 2 participants were significantly slowed down (finally found it).
  • 1 participant modified the existing sample connection instead of creating a new one (it also doesn't work)
  • 2 participants couldn't find it, facilitator had to help them
Recommendation
  • Add "Add Database Connection..." menu item or similar into contextual menu on Databases node.

[EE-25] Invisible light bulb

Finding
  • Participant doesn't see and doesn't use the light bulb associated with error icon saying "There is not specified a primary key property or field". He reads the error in tooltip, but doesn't click the light bulb which is right next to the error icon.
Severity
  • 1 participant fails to proceed
Recommendation
  • Error and hint should be the same icon. There should be an indication (a "hand" cursor) that the icon is click-able when the user hover mouse over the icon.

[EE-26] Newly created Persistence Unit is not selected in Project window

Finding
  • Participant created a new persistence unit. It was open in editor, but persistence.xml node wasn't selected in Project window.
Severity
  • This is a bug.
Recommendation
  • Fix it.

[EE-27] Fix Import dialog offers wrong import for Entity annotation

Finding
  • Participant used Fix Import action to fix import for @Entity annotation. The dialog offered "javax.swing.text.html.parser.Entity" as the first choice, which would be a wrong import for annotation type.
Severity
  • 1 participant had to manually select correct import type.
Recommendation
  • We should only offer annotation types when importing annotations.

[EE-28] No Refresh action on DB connection node

Finding
  • Participant wanted to refresh the view on database in Runtime window, but there wasn't a menu item in contextual menu on individual connection node. He found it on Tables node.
Severity
  • 1 participant expected the Refresh action on connection node
Recommendation
  • This action probably should be in the contextual menu on connection node.

[EE-29] Exception in browser not understood

Finding
  • Participant ran the enterprise application and got exception in browser window. Probably the reason was that he wanted to store an instance of entity class into DB that already contained an item with the same primary key. Participant didn't understand what was the problem.
  • Participant didn't understand the exception which pointed to his source code. He looked into the server log in the IDE, but he just read it and wasn't able to figure out what was the problem. The root cause probably was he forgot to add @EJB annotation on injected session bean.
  • Participant didn't understand the exception which pointed to his source code. He looked into the server log in the IDE, but he just read it and wasn't able to figure out what was the problem. The root cause of the exception probably was he forgot to add @Entity annotation on class he persisted with entity manager. He found the root cause exception in the server log.
  • Participant didn't understand exception in web browser. There wasn't his code in the stack trace.
  • See screenshot 1, screenshot 2, screenshot 3, screenshot 4
Severity
  • 4 participants didn't understand the error in web browser (1 participant figured out what was wrong using the server log)
Recommendation
  • Provide human readable errors. With better description. Better formatted.
  • Ideally the user should be able to click the error that would bring him back to the IDE on the location in the source code where the problem might be.
  • The user should be able to click on the exception in the server log and navigate to the source code.
  • Ideally, the server log would be better integrated with program output because in some cases it is covered by other tab. It also contains a lot of junk the user is not interested in.

[EE-30] Too many annotations in code completion

Finding
  • Participant wanted to use the column annotation but didn't know the name ("column"). There were too many annotations offered in the code completion so he didn't find the right one.
Severity
  • 1 participant wasn't able to find the right annotation just with the help from code completion
Recommendation
  • Consider providing a "recently" used items in code completion, that would initially contain the most commonly used annotations (Id, Entity, Session, Column, etc.).

[EE-31] Javadoc window doesn't explain how to use annotations

Finding
  • Participant wanted to use the column annotation, but didn't know how to write the attributes. He expected help in the javadoc popup window (when Column was selected in code completion), but it didn't help him.
Severity
  • 1 participant couldn't proceed just with the help from the IDE
Recommendation
  • Provide description how to use annotation in javadoc popup window.

[EE-32] Not clear error about database not running

Finding
  • Participant ran the enterprise application that used a database. It failed because the database server wasn't running. Participant didn't understand the error message that appeared in the output window.
  • Participant suspected that probably the database wasn't running, but didn't know how to start it up. He clicked in the runtime window on the application server nodes (JDBC resources) and databases nodes, but didn't see it.
  • See screenshot.
Severity
  • 5 participants didn't know how to proceed
Recommendation
  • Start the DB server automatically.
  • Provide user friendly error message.
  • Consider automatically adding connections for the existing data sources registered on servers. Or those that are used in applications.
  • Consider showing the database connections used in the application in the Project window.

[EE-33] Two apache derby drivers

Finding
  • Participant wanted to connect to database but selected wrong driver Apache Derby (Embedded) instead of Apache Derby (Net). He filled in the URL, which resulted in error message he didn't understand.
Severity
  • 1 Participant didn't know how to proceed.
Recommendation
  • Provide better name for drivers.
  • Tell the user he should use the other driver.
  • Something else???

[EE-34] Automatic code completion after @ char

Finding
  • Participant wanted automatic code completion after typing the @ character. He didn't hit Ctrl-Space and typed the annotation PersistenceContext manually.
Severity
  • 1 participant requested this behavior
Recommendation
  • Consider showing the code completion automatically after the '@' char, just like after typing the '.' char.

[EE-35] Not clear distinction between radio buttons in persistence xml editor

Finding
  • Participant wasn't sure which radio button to select in persistence xml editor (Container-managed or Application-managed).
Severity
  • 1 participant didn't know how to proceed
Recommendation
  • Improve wording in radio buttons
  • Provide combo box for selecting the data sources. This would help in this particular case when user has the data source set up on the server.

[EE-36] Not clear how to look into the database

Finding
  • Participant wanted to check the data in the database, but didn't find how to connect to the database. He didn't even look into the Runtime tab. He went to the Tools menu and other main menus.
Severity
  • 1 participant couldn't finish the task.
Recommendation
  • Consider renaming Runtime to Servers
  • Consider showing the database in the Projects window

[EE-37] Cannot find Encapsulate Fields

Finding
  • Participant (IDEA user) wanted to generate getters and setters for fields, but didn't use Encapsulate Fields item even though he was looking into the Refactoring menu. He probably expected "Getters and Setters". Note that this participant created the fields with "Add Field" dialog and expected that it would also create getters and setters.
Severity
  • 1 participant had to write getters and setters manually
Recommendation
  • Consider renaming the Encapsulate Fields menu item
  • Consider generating getters and setters in the Add Field dialog. And also consolidate all the Add dialogs on java, class and bean pattern nodes.

Participants

All participants were developers of enterprise and web applications with 1 or more years of experience. No participant had a real experience with Java EE 5. Some of them heard about Java EE 5 or read something about it.

Table: Experience of participants

Participant Technologies (years of exp.) Familiarity w/ Java EE 5 IDE App Server
P1 EJB(4), SQL(6), JSP(4), WS(2) Does not know anything about it. NB 5.0, NB 4.0 SJSAS 8.2, Sybase
P2 EJB(2), SQL(5), JSP(4), JSF(0.5) Knows that it exists. NB 4.1, JB JBoss 4 , SJSAS 8.1
P3 EJB(1), SQL(3), JSP(0.5), WS(1) Read about it. Knows Hibernate 2 and 3. NB 5.0, IDEA 4.5, Eclipse 3.0 WebLogic 8, Tomcat 5.5
P4 EJB(1.5), SQL(4), JSP(5), Struts(1.5), WS(0.5) Read about it. IDEA 5.1, FFJ, JB OC4J, Orion
P5 EJB(1), SQL(7), JSP(2), WS(1) Read about it. NB 5.0, NB 4.1, Eclipse 3.2 JBoss 4, SJSAS 8.1
P6 EJB(1.5), SQL(5), JSP(3), WS(1) Does not know anything about it. Eclipse 3.1 JBoss 3, WebLogic 8.1
P7 EJB(2), SQL(3), JSP(2), Struts(1.5), WS(1) Read about it. IDEA 5.1, JBuilder OC4J, Orion
P8 EJB(2), SQL(3), JSP(3) Read about it. Eclipse 3.1 Tomcat

Post Study Inquiry

Each participant was briefly interviewed in the end of the study. The interview had the following goals:

  • investigate, if the IDE "was helping" participants in getting familiar with the new technology Java EE 5 (identify elements that were helpful and elements that were confusing)
  • get the participants' overall impression of the NB support of Java EE 5
  • investigate the way participants usually get familiar with new technologies

Participant 1:

  • He would appreciate a tab containing all beans - when he searches for particular bean he has to search in all packages, one tab with beans would save his time.
  • He likes code completion.
  • He complains that it it is not possible to create business process "by the right click" as he was used to.
  • He criticizes descriptor editors, "it seems that they are not working."
  • He does not like JSP palette - "I do not use it".

Participant 2:

  • P2 liked code completion, he found it very helping in learning JAva EE 5.
  • When learning new technology he appreciates a sample piece of code (with comments) that works and represents a simple elementary function - "this is the best way how to understand and learn new technologies"
  • He complains that when he do changes in EJB or JAR, NB automatically synchronize, even the change is not fully done. Thus he has to do this changes in external editor.

Participant 3:

  • He liked generated clones - "It was very helpful."
  • He would appreciate wizards for configuration of database tabs.
  • He mentions that Java EE 5 is very similar to .NET.

Participant 4:

  • He criticizes slow responses, especially when fixing an error in the code - the error is indicated several seconds after it is fixed.
  • He misses the way of creating business methods on session beans as he he was used to in previous versions - "I am getting lost, I am getting into panic."
  • He mentions he must know all annotation for using this technology.

Participant 5:

  • He appetencies the code generated when creating New Session Bean.
  • He likes code completion.
  • He likes that JARs are generated automatically.
  • He does not like Project window - "too much tabs, it is confusing".
  • The way he gets usually get familiar with a new technology. He reads tutorials found on web.

Participant 6:

  • P6 appreciates support of the IDE in familiar with Java EE 5, especially code generation when opening new project.
  • He does not like that JARs are not automatically imported from the sessions. He likes code completion and the light-bulb for tips.
  • The way he gets usually get familiar with a new technology. He reads tutorials. (He searches for it at java.sun.com)

Participant 7:

  • P7 was not satisfied with level of support provided by the IDE in getting familiar with Java EE 5. He had to look for help outside of the IDE.
  • Automatic generation of annotation was helpful for him regarding to learning new technology. On the other hand he did not like that not all annotation were generated correctly and nor worked well.
  • He would appreciate automatic generation of getters and setters.
  • He considers EJB inject as the most difficult part of the tasks.
  • The way he gets usually get familiar with a new technology: He reads tutorial. (He searches for it at java.sun.com); He practices the new technology with the support of the IDE.

Participant 8:

  • P8 liked that many things are ready to use without his effort. He appreciates he did not need to take care of local interface.
  • Code completion was helpful for him. He liked the comments in html structure.
  • He would appreciate automatic import of public methods.
  • The way he gets usually get familiar with a new technology: He reads tutorial; He practices the new technology on several practice elementary tasks within the IDE.

Task Analysis

Tasks:

  • Task 1: Browse the IDE
  • Task 2: Create and launch a Hello World enterprise application
  • Task 3: Store an entity class (5 instances) into existing database
  • Task 4: Create entity classes from existing database (2 tables) and read data from the database
  • We had 3 other tasks the participants didn't get to because of the time constrain. See the task script (pdf) for all tasks.

The full wording of tasks is available in Task Script (pdf) .

Task 1: Browse the IDE

Objectives
  • Observe participants' behavior while browsing the IDE for the first time.
  • Observe whether participants notice and/or interact with the welcome screen or not.
Result
  • 2 participants clicked the link(s) in the welcome screen (P6, P7)
  • 2 participants looked at the welcome screen. They didn't click on it. (P4, P8)
  • 4 participants didn't use the welcome screen (P1, P2, P3, P5)
Notes
  • P1 closes the welcome screen without looking at it.
  • P1 says: "Looks the same (as 5.0)".
  • P2 doesn't closes the welcome screen even if he doesn't use it.
  • P3 says: "I don't see any difference comparing to 5.0".
  • P3 doesn't use the welcome screen and doesn't close it.
  • P4 looked at the welcome screen.
  • P4 observes the main menu and options.
  • P5 closed the welcome screen.
  • P5 observes the main menu and options. Even the old options dialog.
  • P6 looked at the welcome screen without clicking on it. Then he clicked the links (About and Tutorials).
  • P6 browses the main menu. Opens the New Project dialog. Creates a Web Application. Opened sun-web.xml file.
  • P7 clicks the welcome screen. He says the welcome screen looks like a modal dialog.
  • P7 opens the New Project dialog. Looks at the web project templates. Comments he would like to see subversion in IDE. Looks at options dialog.
  • P8 looks at the welcome screen.
  • P8 browses the main menu.

Task 2: Create and launch a Hello World enterprise application

Objectives
  • Test intuitiveness of creating and launching a simple application.
  • Test how the IDE helps the participants try EJB 3.0 - session beans, injection.
  • Test whether participants user features like Pull Up, Override Methods, Editor Hints, Code Completion, Fix Imports.
Expected Steps
  • Invokes File | New Project
  • Selects the Enterprise Application project and clicks Next
  • Uncheck the Create Application Client Module check box and finishes the wizard
  • Invokes File | New File on EJB Module project
  • Selects the Session Bean file template and clicks Next
  • Specifies the bean name, package and finishes the wizard
  • Writes the session bean methods manually (implementation and interface)
  • Optionally uses features like Pull Up, Override Methods, Editor Hints, Code Completion.
  • Invokes File | New File on Web Application project
  • Selects the Servlet file template and clicks Next
  • Specifies the servlet name, package and finishes the wizard
  • Invokes Project Properties on Web Application project
  • In the Libraries tab adds EJB Module Project to Compilation classpath and closes the properties
  • Writes the servlet code manually (EJB injection)
  • Optionally uses features like Code Completion, Fix Imports, Editor Hints.
  • Invokes Run | Run Main Project (or through the main toolbar)
Notes
  • P1 unchecks the application client module. Creates also remote interface. The session bean isn't selected in project window [EE-1]. Searches for "some way to add a business method to the bean" in contextual menu on bean file node [EE-2]. Cannot find the Enterprise Beans node to add the business method [EE-3]. Writes the code manually. Doesn't use editor features. Writes the injection code manually. Uses Fix Imports which doesn't import the bean class because it isn't on classpath. He writes the import manually [EE-4]. After a while he discovers, he needs to add it as a library. Adds the project to libraries using the Libraries node. Uses Deploy instead of Run. The opens up browser and manually writes in the address [EE-5]. Done.
  • P2 creates the enterprise application with the client module. Wants to create the session bean but the "Scanning in progress..." error doesn't disappear after the scanning finishes so he has to cancel the wizard [EE-6]. Looks for "Add Business Method" in the contextual menu in editor [EE-7]. Goes to the Project window and needs to expand the nodes [EE-1]. Uses the "Add Method" dialog invoked from the Method node underneath the session bean file node [EE-8]. He added the method manually into the interface. He didn't use Pull Up. Wanted to use "Call Enterprise Bean" in Servlet [EE-9]. Doesn't use code completion for injection. Notices it's not on classpath and adds the project manually. Hovers with the mouse above the light-bulb and it's tooltip says "Alt-Enter". So he types Alt-Enter and reads the message "Import ...." and goes to write the import manually [EE-10].
  • P3 creates an EJB module first. Writes the business method manually. Uses Encapsulate Fields. Looks for Pull Up, cannot find it, so writes it manually [EE-11]. Creates web application and new servlet. Used Editor Hint to import EJB. He is confused that the bean interface isn't recognized [EE-4]. Runs web project only (main project) from the main toolbar and gets exceptions in web browser. Doesn't seem to understand them [EE-12]. Fails to proceed. Facilitator has to help. Creates Enterprise Application and adds those modules. Gets a deployment error in output window. Based on that undeploys the existing modules and deploys the whole application. Done.
  • P4 creates the enterprise application and session bean. Wants to add a method using contextual menu on bean file node [EE-13], and in contextual menu in editor [EE-7]. Finds it on the class node [EE-8]. Uses the add method dialog, but he doesn't notice the method wasn't added into the interface. Wanted to use "Call Enterprise Bean" in Servlet [EE-9]. Writes import for the bean manually [EE-4]. Doesn't know how to fix the missing JAR on classpath [EE-14]. Fails to proceed. Facilitator has to help. Finally okay. Finds out that the method isn't in the interface. And asks again for some dialog to add a business method. Writes the method into the interface manually. Done.
  • P5 creates EJB module and session bean. Writes the business method manually. Creates a Web application and servlet. Copy-pastes the injection code from PDF to Servlet. It's confusing that code completion works for unknown symbols and it offers methods from Object [EE-15]. Builds the project and manually fixes the missing EJB module on classpath. He doesn't know that he needs to import the EJB annotation. He doesn't use Fix Import nor Editor Hint and apparently he doesn't understand the error message in tooltip and compiler output [EE-16]. Fails to proceed. Facilitator has to help. Writes import manually. Creates enterprise application and adds both modules into it. Deploys application and manually writes the application address into the web browser [EE-5]. Doesn't know to write it correctly so tries to Run the application. It doesn't work because he deleted index.jsp [EE-17]. Fails to proceed. Facilitator has to help. Writes the servlet manually in the web browser. Done.
  • P6 already created the web project in previous task. Creates entity class in web project and then realizes that it's an entity bean, not a session bean. Wants to create a session bean from a regular java class by adding annotations. Facilitator stops him. Creates enterprise app with client module. Writes the business method manually without adding it to the interface. Wanted to call EJB using the contextual menu in editor of servlet class [EE-9]. Compiles the web project but the EJB modules isn't on classpath [EE-4]. Fails to proceed. Facilitator has to help. Wanted to add project to classpath but closes the Add Project dialog [EE-14]. Writes the interface method manually. Done.
  • P7 creates enterprise application. Notices that the wizard layout jumps when inline error appears [EE-18]. Creates a session bean. Wants to add a business method using the contextual menu in editor [EE-7] and using the contextual menu on file node [EE-2] then he uses the Add Method dialog [EE-8]. Wants to use Call EJB dialog [EE-9]. Code completion doesn't work because the EJB module isn't on classpath [EE-4]. Finds out he needs to add the module to classpath. Doesn't know how to do it [EE-14]. Fails to proceed. Facilitator has to help. He use Fix Imports which didn't do anything just wrote message to status line that user didn't notice [EE-19]. Finds out that the method is not in the interface. Wants to use Pull Up, doesn't find it [EE-11]. Writes the method into the interface manually. Done.
  • P8 creates enterprise application with client module. Creates session bean. Wants to use "add business method" dialog in editor [EE-7]. Writes it manually but doesn't write it to the interface. Facilitator helps with the classpath problem to save some time. As the method is not in the interface, he injects the bean implementation. Deploys the application and gets the following error in build output: "...\build-impl.xml:229 java.net.MalformedURLException: no protocol: ${client.url}" [EE-20]. Fails to proceed. Facilitator has to help. As we're not sure what is the problem we suggest to remove the client module from EAR. Deletes the client module before removing it from the application. Removes the client module, which throws an exception and doesn't remove the module [EE-21]. Creates a new application and adds the EJB and Web modules into it. Then deploys the application which fails with "deployment error" [EE-22]. Fails to proceed. Facilitator has to help. Needs to undeploy the old modules from the server. Doesn't know how to do it [EE-23]. Runs the application but it doesn't open up the web browser [EE-17]. Fails to proceed. Facilitator has to help. Writes the URL manually, the web browser shows exception that participant doesn't understand [EE-12]. Fails to proceed. Facilitator has to help. Fixes the problem in the source code and wants to update the module on server. Tries to write the URL in web browser (without building and deploying the application). Doesn't show up the new app, so clicks the "restart server" icon in the server log. Doesn't work so runs the application. Done.

Task 3: Store an entity class (5 instances) into existing database

Objectives
  • Test how the IDE helps participants try EJB 3.0 - Java Persistence.
  • Test whether participants user features like Editor Hints, Code Completion - Generate Getters and Setters, Encapsulate Fields, Fix Imports.
  • Test intuitiveness of creating Persistence Units
  • Test intuitiveness of registering and using existing database.
Expected Steps
  • Invokes File | New Project
  • Selects the Enterprise Application project and clicks Next
  • Unchecks the Create Application Client Module check box and finishes the wizard
  • Invokes File | New File on EJB Module project
  • Selects the Entity Class file template and clicks Next
  • Specifies the class name, package and finishes the wizard
  • Writes the fields and getters and setters manually
  • Optionally uses features like Encapsulate Fields, Code Completion - Generate Getters and Setters, New Property Pattern dialog.
  • Invokes File | New File on EJB Module project
  • Selects the Session Bean file template and clicks Next
  • Specifies the bean name, package and finishes the wizard
  • Writes the session bean methods manually (implementation and interface)
  • Optionally uses features like Pull Up, Override Methods, Editor Hints, Code Completion, Fix Imports.
  • Invokes File | New File on Web Application project
  • Selects the Servlet file template and clicks Next
  • Specifies the servlet name, package and finishes the wizard
  • Invokes Project Properties on Web Application project
  • In the Libraries tab adds EJB Module Project to Compilation classpath and closes the properties
  • Writes the servlet code manually (EJB injection)
  • Optionally uses features like Code Completion, Fix Imports, Editor Hints.
  • Invokes Run | Run Main Project (or through the main toolbar)
Notes
  • P1 tries to connect to the sample database first. It doesn't work because the user ID isn't correct. Tries to add a new database connection, but doesn't know how [EE-24]. Creates a new enterprise application and new java class. Adds new property using New Property Pattern dialog. Adds @Entity annotation manually. Doesn't see the light bulb that would offer to create a new ID field [EE-25]. Fails to proceed. Facilitator has to help. Clicks the light bulb and generates the ID field. Creates a new session bean. Creates persistence unit from the wizard. It isn't selected in Project window after creating it [EE-26]. Uses Fix Import. Creates new servlet. Fix Import dialog offers wrong class for @Entity annotation [EE-27]. Writes the business method into interface manually. Uses Fix Imports again. Runs the application. Wants to use a Refresh item on the connection node [EE-28]. Finds out that only 1 player is in the database. Corrects the code and runs again. Gets exception (because the player with the same primary key is already in the DB). Doesn't know what is the problem [EE-29]. Asks for help. Fails to proceed. Facilitator has to help. Fixes the problem. Done.
  • P2 creates a new java class in the same project. Uses Encapsulate Fields. Creates persistence unit from the warning. Wants to map the field to a different column but doesn't know which annotation to use [EE-30]. Doesn't know how to use the Column annotation, the javadoc popup doesn't help [EE-31]. Uses Fix Imports. Creates session bean. Adds the business method into interface manually. Runs the application which fails because the DB server is not running [EE-32]. Fails to proceed. Facilitator has to help. Wants to add a database. Doesn't know how [EE-24]. Uses wrong DB driver, which results in error [EE-33]. Fails to proceed. Facilitator has to help. Runs the application and an exception appears in the browser [EE-29]. Fixes the missing @EJB annotation and runs again. New exception in browser [EE-29]. Fixes the missing @Entity annotation. Reads the tooltip error about missing constructor but ignores it and runs the application. Gets an error in output window. Looks into the server log and fixes the missing default constructor. Adds the default constructor. Runs the application. Done.
  • P3 creates a new enterprise application with the application client. Creates a new plain java class in the client module and manually adds the @Entity annotation. Facilitator has to stop him, because we're not sure that the client module would work. Now creates the entity class in the EJB module. Creates the persistence unit using the warning notification - unchecks the transaction APIs and keeps the default data source. Uses Encapsulate Fields. Creates new servlet. Creates new session bean. Writes the business method manually. Would like to see code completion after typing the @ char [EE-34]. Uses editor hints for imports. Tries to set up the database in persistence xml, but doesn't know whether to use container managed or application managed persistence [EE-35]. Fails to proceed. Facilitator has to help. Writes the injection code. Adds the EJB module to classpath manually. Finds out the business method is missing in the interface. Adds it to the interface. Runs the application and gets the error because the database isn't running [EE-32]. Wants to add a new connection. Doesn't know how [EE-24]. Modifies the existing sample connection but it doesn't work because of wrong password. Fails to proceed. Facilitator has to help. Done.
  • P4 creates a new enterprise application. Wants to create a new database connection. Doesn't know how [EE-24]. Starts the database server from Tools menu. Creates new entity class in EJB module, but doesn't click on the persistence unit warning button. Writes getters and setters manually. Creates new session bean. Uses Editor hints for imports. Creates a new class in EJB module. Creates new servlet. Adds the EJB module on classpath. Writes the business methods in the interface manually. Deploys and gets an error in web browser. He doesn't understand the error [EE-12]. Times out.
  • P5 creates new entity class in the existing EJB module. Creates new persistence unit in the warning dialog. Writes getters and setters manually. Deploys and gets error because the DB isn't running [EE-32]. Fails to proceed. Facilitator has to help. Deploys again and gets an error in the web browser. He doesn't understand the error [EE-29]. Fails to proceed. Facilitator has to help. Asks the participant to undeploy the application. He doesn't know how to do it [EE-23]. Fails to proceed. Facilitator has to help. Finally locates the problem in the server log. Times out.
  • P6 creates new enterprise application and entity class. Doesn't know how to proceed with persistence unit, so cancels the dialog. Finally creates persistence unit using the PU wizard with facilitator's help. Adds properties using the Add > Property dialogs (Bean Patterns node). Creates new session bean and adds business method using the Add Method dialog. Creates servlet and writes the persistence code. Deploys and gets the error about database not running [EE-32]. Finally fixes the problem. Cannot find how to look into the database [EE-36]. Times out.
  • P7 creates a new enterprise application with the client module. Wants to look into the database. Doesn't know how. Finally finds the Runtime tab, but doesn't know how to add the database connection [EE-24]. Facilitator helps. Creates new data source with wrong (lowercase) data source name "jdbc/gamesystem". Uses Add New Field for adding the field. Wanted to add a property (with getters and setters). Uses the dialog again, because he isn't sure what else is generated into xml files. Wants to use encapsulate field, but cannot find it [EE-37]. Creates new session bean. Uses Fix Imports. Creates new servlet. Adds the EJB module to classpath. Adds the business method to the interface manually. Runs and gets the error about "gamesystem not found". Wants to check the data source. Corrects the data source name. Times out.
  • P8 creates new enterprise application. Unchecks the client module. Creates new entity class and persistence unit using the warning dialog. Selected the drop and create strategy. Wanted to add the fields from the editor contextual menu, but writes them manually. Uses Encapsulate Fields. Creates new session bean. Uses hint for implementing abstract methods. Code completion shows up for the unknown type [EE-15]. Creates new servlet. Adds the EJB module to classpath. Uses Fix Imports. Runs the application. Database isn't running [EE-32]. Fails to proceed. Facilitator has to help. Done.

Task 4: Create entity classes from existing database (2 tables) and read data from the database

Objectives
  • Test how the IDE helps participants try EJB 3.0 - Java Persistence.
  • Test intuitiveness of using the entity class from database wizard.
Expected Steps
  • Invokes File | New File on EJB Module project
  • Selects the Entity Class from Database file template and clicks Next
  • Selects the GameSystem JDBC connection.
  • Specifies the package and click Next
  • Selects the Game and Game_Room tables and clicks Finish
  • Writes the query code in the existing session bean manually.
  • Invokes Run | Run Main Project (or through the main toolbar)
Notes
  • P1 creates entity classes without problem. Facilitator helps participant to write the query code (we don't have the instructions in the PDF file yet). Uses Fix Imports. Writes the whole code without problems. Runs the application. Done.
  • P2 wants to create empty java class. Facilitator points him to create them automatically. Finds the wizard and creates the classes. Writes the code correctly. Times out.

 

Project Features

About this Project

ui was started in November 2009, is owned by Jiří Kovalský, and has 44 members.
By use of this website, you agree to the NetBeans Policies and Terms of Use (revision 20160708.bf2ac18). © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo
 
 
Close
loading
Please Confirm
Close