JUnit Tests (release F)

Authors: Marián Petráš, Rudolf Bock

• Introduction
• Menus
  • Main Menu
  • Contextual menus
  • Compilation and Running
• JUnit Options
• The JUnit Wizard
• JUnit Result Window

Introduction

This UI Specification describes the UI implementation for JUnit in NetBeans Promo F. It is a continuation to the JUnit Functional Specification Document - Promotion F, which includes a detailed listing of all implemented function and features. In this UI Specification you can find an explanation to the used UI elements and their behavior in conjunction with the user interaction. This document doesn't deal with the processes behind these UI elements neither the implemented functions. These can be found in the mentioned JUnit FSD.

Overview:
JUnit tests can be managed in many ways. User should be able to create, open or run tests from inside the Editor, using the Wizard, from the Project Window using contextual menu or from the main menu. This follows a different situations and UI interactions.
Tests can be performed through the relevant items in main menu nad also using contextual menu in Projet Windows. Debuging of the prepared test can be eventually provided in the same way like debuging source. This UI Specs don't care about this funcionality.
JUnit results panel is a new feature, which will be implemented first in NetBeans Promo F. This panel should improve orientation in the generated test results.

Menus

Menus are the only way to manage JUnit Tests. They cover all JUnit tasks within the IDE.

Main Menu

Most items are located in the main menu. The main JUnit Tests menu can be found in the Tools menu (as a submenu). This menu contains three items:

Go To Test/Go To Tested Class (Ctrl-Alt-K on Windows and Cmd-Alt-K on MacOS)
This menu item represents an action which toggles between a source code and the corresponding test class source code. This action is enabled in the following cases:

1. the text editor is currently active and the cursor is located in a source code of a class for which a test class exists or could be created
2. exactly one node is selected in the active component and the node represents (a part of) a source code of a class for which a test class exists or could be created
3. the text editor is currently active and the cursor is located in a source code of a class which is a test class for another source class
4. exactly one node is selected in the active component and the node represents (a part of) a source code of a class which is a test class for another class

Go To Test
The action opens source code of the corresponding test class in the editor. If the selected node represents a method (or its part) or if text cursor in the active editor is located in a method, the text cursor is placed to the corresponding test method. If there are multiple corresponding test methods (in different test classes, each of them being from a different tests root), the first one (by the order of the source roots) is used.

If the test class was already opened in the editor (it just was not active and it might be obscured by other windows) and the text editor cursor was already located within the target boundaries (i.e. in the target class and, optionally, in the target method), the cursor is left at the current position, i.e. it is not moved to the beginning of the test class or method.

If a corresponding test class does not exist, a dialog appears, asking the user for confirmation about creation of the test class:

+----------------------------------------------------+
|                    Create Test                     |
+----------------------------------------------------+
| The corresponding test class and test method does  |
| not exist. To create ones, set the following       |
| options and press button Create.                   |
|                                                    |
| Class Name: foo.bar.BazTest                        |
| Location:   Test Packages                          |
|                                                    |
| [ ] Generate Method setUp()                        |
| [ ] Generate Method tearDown()                     |
| [x] Generate Default Method Body                   |
|                                                    |
|                                                    |
|                           [[ Create ]]  [ Cancel ] |
+----------------------------------------------------+

If the user selects the Cancel button, the action is cancelled, i.e. the dialog disappears and nothing else happens. Otherwise, a new test class (and, optionally, a test method) is created in the target location and the action continues as if the test class (and test method) already existed.

If one or more corresponding test classes were found but none of them contained the corresponding test method, the program offers the user creation of a new test method:

+-----------------------------------------------------+
|                     Create Test                     |
+-----------------------------------------------------+
| Test corresponding test method does not exist. To   |
| create one, set the following options and press     |
| button Create.                                      |
|                                                     |
| Class Name: foo.bar.BazTest                         |
| Location:   Test Packages                           |
|                                                     |
| [x] Generate Default Method Body                    |
|                                                     |
|                                                     |
|                             [[ Create ]] [ Cancel ] |
+-----------------------------------------------------+

If the user presses the Cancel button, the action is cancelled, i.e. the dialog disappears and nothing else happens. Otherwise, a new test method is created, according to the options specified in the dialog, and the action continues as if the test method already existed.

Go To Tested Class
This menu item opens the corresponding source class containing the proper source method. If no corresponding source class exists, a notification dialog appears and the action terminates. If one or more corresponding source classes exist (in different source roots) and the cursor is not located inside any test method, source class from the first source root is opened. If one or more corresponding source classes exist (in different source roots), the cursor is located inside a test method but none of the source classes contains a corresponding source method, source class from the first source root is opened. If one or more source classes (in different source roots) contain a corresponding source method, the first such source class (in the order of the source roots) is used and the cursor jumps to the beginning of the method. If the source class was already opened in the editor (it just was not active and it might be obscured by other windows) and the text editor cursor was already located within the target boundaries (i.e. in the source class and, optionally, in the source method), the cursor is left at the current position, i.e. it is not moved to the beginning of the source class or method.

Contextual menus

Menu items that manages JUnit Tests are present in two contextual menus – in the editor's context menu and in the Project window. Contextual menus should be straight focused on the content, this is why there are present only the necessary items.

Editor contextual menu

The picture shows the contextual menu in the editor. As a matter of fact, there is only one item added to the Go To submenu. It's the Test/Source, whose functionality is similar to the Go To Test/Source item from the main menu, it's use the same shortcut as well.

In the Project Window context menu, the JUnit Tests submenu is placed between the Refactor and Tools submenus. Simultaneously it was removed from the Tools submenu. All submenu items have the same keyboard shortcuts like in the main menu.

Compilation and Running

Test can be run from two places. From the main menu "Run" and from the contextual menu in the Editor. The main menu gives more options. User can test the whole project using menu item "Test '......' " (Alt-F6).

In the submenu "Run File" user can find items "Test '.......' " (Ctrl-F6 on Windows and Cmd-F6 on a Mac) and "Debug Test for '.......' " (Ctrl-Shift-F6 on Windows and Cmd-Shift-F6 on MacOS), which allows test a concrete class. The desired class can be chosen in the Project Window. User can also select a test class and then run it using menu item "Run File" in the contextual menu of the Project Window (shortcut Shift-F6).

Testing the Project - Run All Tests
The user can run tests for the whole project anywhere in the Editor when he press the shortcut Alt-F6. This way started tests will run in background. When the tests are run, the test result window is shown. Running tests in the background allow the user continue working.
[See also FSD chapter 3.3.1]

• Note: Debuging tests is possible using the build-in Debuger. This procedure is similar to the debuging of source classes as well. Test class for a selected source class can be debuged by choosing menu item "Debug Test for '.......' " or by pressing the shortcut Ctrl-Shift-F6 on Windows and Cmd-Shift-F6 on the Macintosh. Selected test class can be debuged with the classic Debug.... menu item.

JUnit Options

Options for the JUnit can be found in the IDE options dialog. The section Testing contains one item called "JUnit Module Settings".

Most of these options sets the default values for the UI elements used in Create Test Dialog and others. In the part "Expert" user can set, if there should be a main method created and how should look its body. The values can be changed in a dialog, which will be displayed after clicking on the "..." buttons on the right site.

The JUnit Wizard

The JUnit test classes can be created using a Wizard as well. Wizard is invoked by the main menu item "New File..." in the menu "File" or by the same contextual menu item in the Project Window. His functionality is mostly similar to the Create Tests dialog described above. The next picture shows one of this similarity. You can see the same sections A,B and D, which behaves in the same way as in the Create Tests dialog. Section D, which displays warnings and another informations, is drawn as multi-line text area with a vertical grow. Text shown in this area should be wrapped – not truncated.

In the Wizard on MacOS still occur a design flaw, which should be solved. The Wizard form for create test classes includes fields where user can type data and another for display purposes only. The second mentioned kind of elements represent a problem. While in Windows these elements look like disabled - not enterable, on the MacOS X they behave like read-only. This means, that these element are enterable, but you can't make any changes. Even you can't copy content from these enterable/read-only fields to the clipboard. This is a little bit confusing.

Suggested solution: On all platforms should be these elements disabled. These element should look dimmed. The property "enabled" should be set FALSE. This is not a good solution as it would cause an accessibility issue.

Creation of a new Test Class from an existing Source Class

The basic sections of the Wizard forms behaves in the same way as in the Create Test dialog. The result is the same as of the create test class for current source class. Wizard will use the window title "New Test for Existing Class".

The user can set the source class manually (by typing into the field "Class to test") or by the help of the "Select Class" dialog, which can be invoked by clicking on the button "Browse...". As the next picture shows, the list contains source packages and classes.


  This dialog must be vertically resizable by user.

If the user select more than one source class or if he select a package, the OK button must be disabled. The same rule must be applied in the Wizard forms as well – if the inserted class name is invalid or a empty string, the button "Finish" should be disabled.
If there are mapped more test-roots, the user can selected the proper one from the combobox "Location". [See also FSD chapters 5.1.1 and 3.1.2]

Creation of an Empty test Class

User can create an empty test class, which can contain only the basic skeleton with the methods setUp() and tearDown(). This function result in an empty test class as was specified by the Wizard form. The window title of the Wizard is "New Empty Test". [See also FSD chapters 5.1.2 and 7.1]

Create a new Test Suite

This Wizard form allows the user create a test suite for a test package. The user can select the proper package from the combobox. The newly created test suite will by located in the same test package.

JUnit Result Window

Tests results are no longer displayed in the default output window. Instead, the output is read by the JUnit module and the results are presented in a special JUnit output window. The JUnit output window is displayed and activated as soon as a test run finishes.

The results are displayed in a tree (see the image above). If there was just one test class run and at least one of the test methods failed or caused an error, the test class node is expanded so that the list of test methods is displayed. Otherwise only nodes representing failed tests classes with failed tests are expanded. Nodes representing failed tests may be further expanded to view the error message (if any) and a call stack of the exception that caused the failure or error.

If the output generated by Ant and the JUnit framework contains information about elapsed time of individual test methods, the elapsed times are displayed on the right of the tests' status labels, e.g. "passed  0.012 s".

The results may be filtered so that only failed test classes and failed tests within them are displayed. The filter is activated by pressing the toggle-button in the window's toolbar – see the image below:

If the report generated by Ant and JUnit framework contains only information about failed test classes and methods, the Filter toggle button is disabled.

If there are five or less failed test methods, all nodes are expanded by default, i.e. the user can see all available information (except elapsed times, see below). If there are more failed tests, only nodes representing test classes are expanded (no callstacks displayed).

The results window also allows to view the standard and error output produced by the executed tests. When the user presses the Output button, the tree of results is replaced with an output pane:

The user can switch between standard output and error output by selecting the corresponding labels (buttons) on the top of the window.

When output is displayed, the Filter toggle button is not visible.

The JUnit Results window is hidden by default (or in background) while test are running. As soon as the tests are finished, this window is brought to top. [See also FSD chapters 6.1, 6.2, 6.3, 6.4]