new GUI Builder (Matisse)
- Use Cases
- Design specification
- Basic Concept
- Detail specification
- Modes in Editor
- Layout mode
- Adding a component
- Guiding lines
- Grouping a component
- Inserting a component
- Copying a component
- Moving a component
- Resizing a component
- Removing a component
- Align mode
- Other modes
- Toolbar in Editor
- Contextual Menu
- Other Enhancements
- Ideas to the future
- Open issues
- Unimplemented features in Promo F
Building UI application in Java is quite difficult, because there are too many layout managers and none of them is ideal for designing GUI easily. The main reason is, that user needs to understand behavior and restrictions of those layouts first. Thus we decided to focus on user and create new GUI builder based on current Form Editor in the Netbeans. This tool will fix the problem and allows users to design good looking and behaving GUI application (supporting i18n techniques) without to much thinking about layout manager existing in the background.
The main goal of this document is the following:
- Specify UI and dynamic behavior for this GUI Builder
- Describe cooperation of GUI Builder with rest part of IDE
- Suggest other possible enhancements for the future
- Experiences with current Layout Managers
Important note: Not all of suggested features will be implemented in promo F, because we are focused on Editor area mainly in the first phase of the Matisse project.
List of requirements for GUI Builder was created based on:
- Previous research of competitive approaches
- Discussions about problems of current approaches
- Usability study of first Prototype (Java and Flash)
- Second usability study of Java Prototype
- Collected feedbacks from more WEB discussions (e.g. JavaLobby)
- Feedback from Java One presentation
This document focuses on the following requirements which should be accomplished:
- Intuitive and better GUI Builder then competitive approaches
- Users shouldn't think about layout manager, but only about actual design of his application
- Easy operations with components (adding, rearranging, resizing, removing)
- Clear visual and logical indication of different states of components and user's actions
- Take dynamical behavior of i18n support into account, because it's hard to arrange with current Layouts Managers
- Reuse typical behavior of users that was confirmed in other IDEs or applications as much as possible
- Visual consistency of components and operations in GUI Builder
- To be consistent with other product of SUN
- Support of L&F standards
This chapter mentions some typical use cases, which we choose for testing of our approach (natural layout). The main goal was to successfully solve situations, which could come into being. We went through the following dialogs step by step and created detail descriptions of users actions (it isn't part of this document). Then we found concepts, which should cover those use cases.
figure 1 - Use cases which we tested in detail
Another logical and more common use cases, which we would like to solve in design part:
- User would like to put components into designer quickly without to much thinking about right design of a layout
- User would like to put components into designer and care about the right design of all layout immediately
- User would like to rearrange components inside the layout easily
- User would like to change properties by components directly from the area of layout
- User would like to change properties using the Property Sheet window
The following UI cases were generated by testing prototypes: http://ui.netbeans.org/docs/ui/guibuilder/cases.html
We start from a fact, that user is primarily focused in the Editor area, in the Component Palette (at the start of designing mainly) and she needs to access other windows occasionally (e.g. Properties, Inspector). This behavior was observed on usability study. User should be able to realize most of actions in the layout area in causal cooperation with other part of IDE. To support this we suggest next concept:
User uses different modes during design, and we support those modes in the GUI Builder by a mode switcher situated in the Editor's Toolbar. There are the following modes and you can imagine them as a different layers in the editor:
- Layout mode
- Align mode
- Edit text of components
- Edit name of components
- Set TAB traversal
- Connection mode
User is able to do the following in the Layout mode, which is the most important one:
Some actions could be done automatically according to users action:
- Grouping components, when components are aligned by shared baseline or by guide lines
- Changing behavior of components (set fill property when component was aligned at the both sides)
- Resize component, when height of another one in the group was changed
- Move component, when length of some other in the group was changed
The intelligent help system is able to assist user during creation of the layout and is situated below the Editor's Toolbar. The typical help message could be:
- When the layout area is empty this message appears: Put component into the empty area by Drag&Drop or by Pick&Plop from a Palette
- When user is dragging component and guiding lines appears, the message says: You can place component in right distance from another one
- When user selects some component, the message says: You can use Ctrl+Mouse for copying components
Figure 2 - Visualization of different modes. More details will be mentioned in the text below
This chapter of document describes in detail what was mentioned above and contains visualization too. First part is focused on the modes in editor and user's typical operations in the Layout mode with the inclusion of grouping components together. Then there is a section about Align mode and description of other modes will be added later in the future.
I mentioned support of more possible modes (Layout mode, Align mode, Edit
text of components, Edit name of components, Set TAB traversal and Connection
User can change actual mode by different ways:
- From a ComboBox situated in the Editor's Toolbar.
- Drag&Drop or Pick&Plop by mouse from the Component Palette change it into the Layout mode.
- By keyboard combination (ALT key activate Align mode), which change mode temporarily, as long as user presses the key.
Layout mode is activated by default and help messenger will
navigate user to other ones in proper moment.
Case 3. from above causes switching a mode in the ComboBox temporarily, so it's some kind of navigation too.
When the user selects one component, the resize handles are
displayed around components (see figure 3).
The user can select more components (by Mouse multi selection or by Ctrl+mouse click) and then resize handles are visible only by first component and rest of them contains only orange border (see figure 4).
Color of selection border is #FF8200 and multi selection box during mouse dragging has the same color with appropriate mouse gesture.
Each component which is situated inside the multiselection box (even only by its small part) is included into multiselection.
figure 3 - Non selected and selected component
figure 4 - Multiselection box during mouse dragging and after mouse release
Components could be added into the layout area by more possible ways:
- Drag&Drop component from the Component Palette
- Pick&Plop component from the Component Palette (then popup menu with basic set of Properties can appear...open issue)
- Double clicking on component inside the Palette
- By pressing Enter, when some component is selected in the Palette
- Choosing component from a Contextual menu over the layout area
There is different visualization for first and second methods
of adding component. Component is displayed
under mouse cursor, when user selects component in the Palette
and she is moving it over the Editors area. When moved components
is close enough to some other component/border/group, Guiding
If user adds component from Contextual menu then it will appear at a mouse position.
If user adds component by double clicking or by pressing Enter, then it will appear in the middle of layout area (when there is an empty space) or at the nearest possible empty space below.
When component is placed into the layout, then it stays in selection mode (see figure 3).
These guide lines help the user to position components in right
distances or at the same row/column. Guiding lines can be
displayed during adding/inserting/resizing/moving of components.
There are horizontal, vertical or combined guiding lines
according to right distances among components/containers/groups
(based on L&F guidelines) and alignment of them.
We defined the following types of guiding lines:
- Sequential lines - Displayed when a component is attached to another one (see figure 5)
- Aligned lines - Displayed when a component is aligned with another one (see figure 5)
- Base lines - Displayed when a position of text inside component is aligned with a text inside another component (see figure 6)
- Central lines - Used for position components in the centre of dialog (figure 6b)
- Indented lines - Special case of sequential lines and are used in a case, when component could be indented from another above dragged one (e.g. sub group of checkboxies)
When user is pressing ALT key during dragging, the guiding
lines are displayed still but snap ability is out of function. This behavior
can happen in the case when component is forced to the position which user
doesn't want from some reason.
When user is pressing SHIFT key during dragging, then user can move only horizontally or vertically and last direction of moving will be kept. When some guide lines have been shown before pressing modifier, then the guide lines will be kept too, but only in one direction once again.
The combination of ALT and SHIFT keys can happened too.
And when ESC is pressed, operation is canceled.
If components are attached/aligned by those guiding lines, then they could be grouped together automatically. If they are grouped, then this group behaves as a one component according to showing next guiding lines by this group (see Flash prototype below). The exact rules are defined elsewhere.
figure 5 - Aligned and Sequential guiding lines
figure 6 - Baseline and Sequential guiding line
figure 6b - Central line for placing component in the centre of dialog (first two pictures).
Next picture is showing situation before Centre Components action and last one after invocation of that action.
Components can be grouped together horizontally and vertically. This grouping is realized automatically according to alignment of components situated at one row/column or manually by actions in the Editor's toolbar or from Contextual menu. There can be two possible groups:
- Open group - Aligning is created only by one line on one side.
- Close group - Components are aligned on both sides
Grouped components react on position/size changes of other
components in the group.
Visualization of groups is done by full line in the same color as guide line has.
When components are grouped at one baseline, then their positions are corrected automatically because of their different height (see flash prototype below).
Flash prototype - Automatic correction of position according to baseline group
User can add components into the area where isn't enough space or add component between another ones. Then components around/containers/all layout must react on this state. When user drops/plops component then the other component will move appropriately. In case of more possibilities, a Popup menu with possible actions will appear by placed component.
Flash prototype- Button is moved automatically because of new components is being added
Flash prototype- TextField is moved automatically after droping because was situated at the right side of button under it
Components can be copied by the following methods:
- User selects one or more components and uses Ctrl+C and Ctrl+V keyboard combinations. Then component(s) will appear under copied selection.
- User invokes Copy and Paste menu item actions from a contextual menu over layout area or from Inspector. Then component(s) will appear at the mouse position before invocation of the Contextual menu.
- User selects component(s), presses Ctrl and move by component(s) using mouse. Then it will create identical component (or set of them) in the direction of mouse moving. This functionality can be used effectivelly in a case of copying row or column with including more components.
Described behavior should work for multiselection too.
Users can move components by these ways:
- By Mouse - Component disappears from old position and is being dragged by user. When component is aligned to other component/border/group, guiding lines are displayed and are suggesting right position. The Help Messenger reacts on this state by suitable message too. When ESC is pressed, components returns at position before dragging.
- By Keyboard - User will move component by using arrow keys. Or she can use Shift+arrow keys combination too. It will cause moving a component in the layout according to guiding lines.
- By Mouse and Keyboard combination - When Shift key is pressed during dragging, then user can move only horizontally or vertically. It will preserve X or Y positions of component.
Beyond this user's moving can happen automatic moving, such that components are moved accordingly with another one was moved/resized.
User is able to resize components by mouse or by keyboard in the following style:
- When a component is selected, then handlers for resizing in appropriate directions are visible.
- User can D&D those handlers and the component will resize to those directions.
- Each component has defined default directions for resizing, but can be unlocked and then all handlers will appear.
- User can use Shift key together with mouse dragging and the component will resize to X and Y directions equally.
- Ctrl+Arrows will be used for resizing of component by keyboard or we will remove this keyboard resizing at all (open issue)
- Most components can be resized only horizontally and another ones (e.g.
textarea) in all directions by default. This limitation can be canceled
from Contextual menu above design area.
- Alignment and component's fill property are changed automatically when component is resized by user
- The properties in the Property Sheet should be synchronized with changes made by mouse or by keyboard in Design area. There is section Layout containing four properties (Width, Height, Horizontal fill and Vertical fill) for promo F. Width and Height properties can contain numbers or Default state value. Horizontal fill and Vertical fill properties are checkboxies.
Flash prototype - Alignment and component's fill property are changed automatically
Most of alignment and grouping will be done in Layout mode
automatically (by snapping components each together). It means,
that user doesn't need to know about it so much. But users should
be capable of change this alignment additionally in some cases.
And that's why we would like to offer this as a second mode.
Next part is about possible approaches and ideas for the Align mode:
A) Simple switching between Layout mode and Align mode
- User can switch into this mode by ComboBox situated in the Editor's Toolbar, by clicking on selected component (on the area out of text inside) or by ALT key press when component is selected (The A key is used instead of ALT in the Flash prototype below). This solution is favoured and recommended by HIE team.
figure 9 - Flash prototype of switching between Layout and Align mode. Try to select component and use A key
B) Aligning could be done by special floating palette (present solution is close to this and was presented on J1)
- When user selects a components, then small align palette with possibility to set align or fill of the component will appear (see figure 10).
- User can cancel it temporarily by ECS key press or hide it from Help assistant popup menu (It's visible by default).
- This palette contains four buttons with arrows for aligning and two buttons in the center for filling.
- Palette is displayed close to selected component (preferred place is above component) by default. The new position of the floating palette should be determined in a case of more selected components or lack of space.
- The palette can be dragged by mouse on different position inside designer. Then it's switch to the draggable mode and it can be switched back to the automatic mode by action from contextual menu invoked above palette.
figure 10 - Align palette will appear by selected component by default (1st picture) and can be draggable somewhere else (2nd picture)
C) Set the alignment from Editor's toolbar (was done like first temporary solution)
The same arrows (left, right, up, down)+handlers for horizontal and vertical
fill which were mentioned above will be situated in the toolbar as a toggle buttons
(see figure 11).
figure 11 - Arrow buttons and fill property buttons in the toolbar
Note: This solution isn't recommended by HIE team because handlers are far away from components in design area and buttons are in the sequence which doesn't have a sence.
D) Set the alignment from the small palette in contextual menu
This approach is good compromise because palette isn't so obtrusive at the first moment, but user can find it quite easy still.
figure 12 - Possibility to change alignement of component from the contextual menu
E) Set the alignment of component by menu items situated in contextual menu above component
The easiest implementation way is to put all six actions into the contextual menu like submenu with menu items, but then it's meaning isn't clear and illustrative enough.
Note: We discussed more variants for the Align mode and the best approach
is opened issue still (HIE recommend A or D approach).
Feedback of alignment have been changed many times according to discussions and usability studies too. The present solution witch anchors to the nearest container or another components is visible on pictures above.
Let's mention other modes that have been listed in intro of this document with short description.
- Edit text of components - All texts inside components will be displayed as editable textfields above components. So user can quickly change all of them.
- Edit name of components - The same like previouse mode but with code names of components.
- Set TAB traversal - Possibility to define TAB order by components quickly only by assigning number of order by clicking on them. It will be visualized like knots with numbers linked by lines.
- Connection mode - User can define connections between components visually by D&D instead of connecting components by old Connection Manager invoked from Editor Toolbar. Then rest steps of precise definition of appropriate action will be reused from that Connection manager wizard after D&D.
The Toolbar contains four areas with different functions. Let's describe it in the detail from left to right:
- There are two toggle buttons for switching between Source editor and Design editor at the left side
- Then there is a area with actions (Preview action and Check Design action, Guide lines switching action, action for switching on/off Anchors witch dotted lines, Locking/Unlocking resizeability of some components).
- Next actions are intended for aligning of component groups (different types), two actions for setting the same horizontal or vertical size and Group/Ungoup toggle button (open issue).
- Switcher among modes (Combobox) is situated at the right side of the Toolbar.
- Actions for Palette customizer and Padding customizer can be situated at left side of that Combo box in Layout mode, but it should be discussed more (open issue)
figure 14 - Editor Toobar with Group+Align actions, Auto resizing and other functionality
The contextual menu should contain the following additional actions:
| Edit Text
| Change Variable Name...
| Events >
| Preview Design...
| Anchor > [ ] Left
| [ ] Right
| [ ] Top
| [ ] Bottom
| Align > ( ) Left to Column
| ( ) Right to Column
| ( ) Top to Row
| ( ) Bottom to Row
| ( ) Left
| ( ) Right
| ( ) Top
| ( ) Bottom
| Auto Resizing > [ ] Horizontal
| [ ] Vertical
| Same Size > [ ] Same Width
| [ ] Same Height
| Set Default Size
| Space Around Component...
| Set Layout > Group Layout
| Other Layouts
| Customize Layout...
| Add From Palette >
| Change Order...
Anchor - User can change anchors by
D&D-ing of selected component(s) or by actions in the Anchor submenu.
Align - The same actions like in the Editors toolbar for alignment+grouping, when more components are selected.
Auto Resizing - User can define (for selected components) horizontal or vertical ability to be resized automaticaly due to dialog (container) resizing.
Same Size - Posibility to define same size by more selected components. These actions should be situated in the toolbar too. Visual feedback by those components will be made by small badge on right upper side (combinations: and see position on figure 14).
Unlock/Lock resizeability - Resizeability in some direction can be locked by components by default (e.g. height of textfields, buttons, comboboxies, labels). So this action will unlock it.
Set Default Size - Possibility to cancel the exact size which was defined by mouse manipulation or in the Properties
Space Around Component... - Will invoke dialog and user can define exact size of empty spaces around selected component (solution for promo F). But better approach is to incorporate the definition of spaces into alignment area on figure 12.
Customization of the Palette can be invoked from Tools menu too by the following action
Tools > Palette Manager > Swing/AWT Components MIDP Components HTML/JSP Code Clips
- Switching from other LayoutManager to Group Layout will cause some inteligent rearangement of components. It can be done automatically if it's possible or with warning dialog wit message describing, that the result of conversion could be strange and far away from perfect state.
- Highlighted panel during mouse over state or during visualization of anchors.
- Customizer for different padding among components and borders of containers should exist in the future.
- Close cooperation with other windows in the IDE, mainly with the Property Sheet and the Inspector. We haven't been focused so much on this area yet.
- Creating custom combination of components in the Component Palette. One possible way is to D&D actual multiselection from the Designer to the Palette. Another one is importing new set of custom components in the Palette Customizer.
- Showing exact size (+lock status) of components in the status line (figure 2)
- Visualization of real dialog with transparent effect in the background.
- There is the only way how to start new GUI Builder (through New... wizard) but new user can have problem to find this way. So some more straight way can exist.
- Special Layout menu in the main menu containing current actions from Contextual menu + typical ones in other tools
- Free Design is draft name and we should check if it's the right one.
- Actions for Palette customizer and Padding customizer can be situated in the Editor's toolbar, but it could be discussed more.
- Add Centre Horizontally and Centre Vertically actions to the contextual menu and Editor Toolbar.
- Should be there some difference in behavior when user uses Drag&Drop or Pick&Plop? E.g. in showing guide lines during adding components or not showing them. Or popup menu with basic set of Properties can appear immediatelly after Pick&Plop.
- We need to check (e.g. on usability) if the ability of adjusting of group position automatically (according to new high of the group) is right approach and user will understand/appreciate it. If it isn't, then we should offer only action for auto-correction invoked by user.
- The exact rules for alingning and grouping should be defined elsewhere.
- Still isn't clear whether dynamic adjustment of groups should be done automatically or manually.
- Anchoring can be changed from the contextual menu only
- Grouping into column or row hasn't any special visual indication and is presented like kind of Alignment actions rather
- Resizing of Multiple components at once
- Special mode for multicopying of selected components mentioned in section "Copying a component"
- Some differences in Keyboard manipulation
- There is no Combo in the Editor Toolbar including mentioned modes.
- Possibility to switch on/off Guide lines and Anchors (actions in the Toolbar)
- User can't lock/unlock resizeability of components (e.g. labels, buttons) from the Toolbar
- There is no global customization of padding
- Visualization of the real dialog in transparent mode (visible e.g. on figure 2)
- Central alignment