Building UI application in Java is quite difficult, because
there are too many layout managers and none of them is ideal for
designing GUI easily. The main reason is, that user needs to
understand behavior and restrictions of those layouts first. Thus
we decided to focus on user and create new GUI builder based on
current Form Editor in the Netbeans. This tool will fix the
problem and allows users to design good looking and behaving GUI
application (supporting i18n techniques) without to much thinking
about layout manager existing in the background.
The main goal of this document is the following:
Specify UI and dynamic behavior for this GUI Builder
Describe cooperation of GUI Builder with rest part of IDE
Suggest other possible enhancements for the future
Experiences with current Layout Managers
Important note: Not all of suggested features will be implemented
in promo F, because we are focused on Editor area mainly in the first phase
of the Matisse project.
2. Requirements
List of requirements for GUI Builder was created based on:
Previous research of competitive approaches
Discussions about problems of current approaches
Usability study of first Prototype (Java and Flash)
Second usability study of Java Prototype
Collected feedbacks from more WEB discussions (e.g. JavaLobby)
Feedback from Java One presentation
This document focuses on the following requirements which
should be accomplished:
Intuitive and better GUI Builder then competitive
approaches
Users shouldn't think about layout manager, but only
about actual design of his application
Easy operations with components (adding, rearranging,
resizing, removing)
Clear visual and logical indication of different states
of components and user's actions
Take dynamical behavior of i18n support into account,
because it's hard to arrange with current Layouts
Managers
Reuse typical behavior of users that was confirmed in
other IDEs or applications as much as possible
Visual consistency of components and operations in GUI
Builder
To be consistent with other product of SUN
Support of L&F standards
3. Use Cases
This chapter mentions some typical use cases, which we choose
for testing of our approach (natural layout). The main goal was
to successfully solve situations, which could come into being. We
went through the following dialogs step by step and created
detail descriptions of users actions (it isn't part of this
document). Then we found concepts, which should cover those use
cases.
figure 1 - Use cases which we tested in
detail
Another logical and more common use cases, which we would like
to solve in design part:
User would like to put components into designer quickly
without to much thinking about right design of a layout
User would like to put components into designer and care
about the right design of all layout immediately
User would like to rearrange components inside the layout
easily
User would like to change properties by components
directly from the area of layout
User would like to change properties using the Property
Sheet window
We start from a fact, that user is primarily focused in the
Editor area, in the Component Palette (at the start of designing
mainly) and she needs to access other windows occasionally (e.g.
Properties, Inspector). This behavior was observed on usability
study. User should be able to realize most of actions in the
layout area in causal cooperation with other part of IDE. To
support this we suggest next concept:
User uses different modes during design, and we support those
modes in the GUI Builder by a mode switcher situated in the
Editor's Toolbar. There are the following modes and you can
imagine them as a different layers in the editor:
Layout mode
Align mode
Edit text of components
Edit name of components
Set TAB traversal
Connection mode
User is able to do the following in the Layout mode, which is
the most important one:
Add component(s),
Copy component(s)
Select component(s)
Resize component(s)
Move components(s)
Remove component(s)
Insert component (s)
Set Aligning by moving
Set Filling by resizing
Set other Properties
Group components together
Exclude component(s) from group
Some actions could be done automatically according to users
action:
Grouping components, when components are aligned by
shared baseline or by guide lines
Changing behavior of components (set fill property when
component was aligned at the both sides)
Resize component, when height of another one in the group
was changed
Move component, when length of some other in the group
was changed
etc.
The intelligent help system is able to assist user during
creation of the layout and is situated below the Editor's
Toolbar. The typical help message could be:
When the layout area is empty this message appears: Put
component into the empty area by Drag&Drop or by
Pick&Plop from a Palette
When user is dragging component and guiding lines
appears, the message says: You can place component in
right distance from another one
When user selects some component, the message says: You
can use Ctrl+Mouse for copying components
etc.
Figure 2 - Visualization of different modes.
More details will be mentioned in the text below
Detail specification
This chapter of document describes in detail what was
mentioned above and contains visualization too. First part is
focused on the modes in editor and user's typical operations in
the Layout mode with the inclusion of grouping components
together. Then there is a section about Align mode and
description of other modes will be added later in the future.
Modes in Editor
I mentioned support of more possible modes (Layout mode, Align mode, Edit
text of components, Edit name of components, Set TAB traversal and Connection
mode).
User can change actual mode by different ways:
From a ComboBox situated in the Editor's
Toolbar.
Drag&Drop or Pick&Plop by mouse
from the Component Palette change it into the Layout
mode.
By keyboard combination (ALT key
activate Align mode), which change mode
temporarily, as long as user presses the key.
Layout mode is activated by default and help messenger will
navigate user to other ones in proper moment.
Case 3. from above causes switching a mode in the ComboBox temporarily, so
it's some kind of navigation too.
Layout mode
Selected component(s)
When the user selects one component, the resize handles are
displayed around components (see figure 3).
The user can select more components (by Mouse multi selection
or by Ctrl+mouse click) and then resize handles
are visible only by first component and rest of them contains
only orange border (see figure 4).
Color of selection border is #FF8200 and multi selection box during mouse
dragging has the same color with appropriate mouse gesture.
Each component which is situated inside the multiselection box (even only by
its small part) is included into multiselection.
figure 3 - Non selected and selected
component
figure 4 - Multiselection box during mouse dragging and after
mouse release
Adding a component
Components could be added into the layout area by more
possible ways:
Drag&Drop component from the
Component Palette
Pick&Plop component from the
Component Palette (then popup menu with basic set of
Properties can appear...open issue)
Double clicking on component inside the Palette
By pressing Enter, when some component is selected in the Palette
Choosing component from a Contextual menu
over the layout area
There is different visualization for first and second methods
of adding component. Component is displayed
under mouse cursor, when user selects component in the Palette
and she is moving it over the Editors area. When moved components
is close enough to some other component/border/group, Guiding
lines appear.
If user adds component from Contextual menu then it will
appear at a mouse position.
If user adds component by double clicking or by pressing Enter, then it
will appear in the middle of layout area (when there is an empty space) or
at the nearest possible empty space below.
When component is placed into the layout, then it stays in
selection mode (see figure 3).
Guide lines
These guide lines help the user to position components in right
distances or at the same row/column. Guiding lines can be
displayed during adding/inserting/resizing/moving of components.
There are horizontal, vertical or combined guiding lines
according to right distances among components/containers/groups
(based on L&F guidelines) and alignment of them.
We defined the following types of guiding lines:
Sequential lines - Displayed when a component is attached
to another one (see figure 5)
Aligned lines - Displayed when a component is aligned
with another one (see figure 5)
Base lines - Displayed when a position of text inside
component is aligned with a text inside another component
(see figure 6)
Central lines - Used for position components in the centre of dialog
(figure 6b)
Indented lines - Special case of sequential lines and are used in a
case, when component could be indented from another above dragged one
(e.g. sub group of checkboxies)
When user is pressing ALT key during dragging, the guiding
lines are displayed still but snap ability is out of function. This behavior
can happen in the case when component is forced to the position which user
doesn't want from some reason.
When user is pressing SHIFT key during dragging, then user
can move only horizontally or vertically and last direction of moving will be
kept. When some guide lines
have been shown before pressing modifier, then the guide lines will be kept too,
but only in one direction once again.
The combination of ALT and SHIFT keys can happened
too.
And when ESC
is pressed, operation is canceled.
If components are attached/aligned by those guiding lines,
then they could be grouped together automatically.
If they are grouped, then this group behaves as a one component according to
showing next guiding lines by this group (see Flash prototype below). The exact
rules are defined elsewhere.
figure 5 - Aligned and Sequential guiding
lines
figure 6 - Baseline and Sequential guiding
line
figure 6b - Central line for placing component
in the centre of dialog (first two pictures).
Next picture is showing situation
before Centre Components action and last one after invocation of that action.
Grouping a component
Components can be grouped together horizontally and
vertically. This grouping is realized automatically according to
alignment of components situated at one row/column or manually by actions in
the Editor's toolbar or from Contextual menu. There can
be two possible groups:
Open group - Aligning is created only by one line on one
side.
Close group - Components are aligned on both sides
Grouped components react on position/size changes of other
components in the group.
Visualization of groups is done by full line in the same color as guide
line has.
When components are grouped at one baseline, then their
positions are corrected automatically because of their different
height (see flash prototype below).
Flash prototype - Automatic correction of
position according to baseline group
Inserting a component
User can add components into the area where isn't enough space
or add component between another ones. Then components
around/containers/all layout must react on this state. When user
drops/plops component then the other component will move
appropriately. In case of more possibilities, a Popup menu with
possible actions will appear by placed component.
Flash prototype- Button is moved automatically because
of new components is being added
Flash prototype- TextField is moved
automatically after droping because was situated at the right side of button
under it
Copying a component
Components can be copied by the following methods:
User selects one or more components and uses Ctrl+C
and Ctrl+V keyboard combinations. Then
component(s) will appear under copied selection.
User invokes Copy and Pastemenu itemactions from
a contextual menu over layout area or from Inspector.
Then component(s) will appear at the mouse position
before invocation of the Contextual menu.
User selects component(s), presses Ctrl and move
by component(s) using mouse. Then it will create
identical component (or set of them) in the direction of
mouse moving. This functionality can be used effectivelly in a case of
copying row or column with including more components.
Described behavior should work for multiselection too.
Moving a component
Users can move components by these ways:
By Mouse - Component disappears from old position and is
being dragged by user. When component is aligned to other
component/border/group, guiding lines are displayed and
are suggesting right position. The Help Messenger reacts
on this state by suitable message too. When ESC
is pressed, components returns at position
before dragging.
By Keyboard - User will move component by using arrow
keys. Or she can use Shift+arrow keys
combination too. It will cause moving a component in the
layout according to guiding lines.
By Mouse and Keyboard combination - When Shift
key is pressed during dragging, then user can
move only horizontally or vertically. It will preserve X
or Y positions of component.
Beyond this user's moving can happen automatic moving, such
that components are moved accordingly with another one was
moved/resized.
Resizing a component
User is able to resize components by mouse or by keyboard in
the following style:
When a component is selected, then handlers for resizing
in appropriate directions are visible.
User can D&D those handlers and the component will
resize to those directions.
Each component has defined default directions for
resizing, but can be unlocked and then all handlers will
appear.
User can use Shift key together with
mouse dragging and the component will resize to X and Y
directions equally.
Ctrl+Arrows will be used for resizing of
component by keyboard or we will remove this keyboard
resizing at all (open issue)
Most components can be resized only horizontally and another ones (e.g.
textarea) in all directions by default. This limitation can be canceled
from Contextual menu above design area.
Alignment and component's fill property are changed automatically when
component is resized by user
The properties in the Property Sheet should be synchronized with changes
made by mouse or by keyboard in Design area. There is section Layout containing
four properties (Width, Height, Horizontal fill and Vertical fill) for
promo F. Width and Height properties can contain numbers or Default
state value. Horizontal fill and Vertical fill properties are checkboxies.
Flash prototype - Alignment and component's
fill property are changed automatically
Align Mode
Most of alignment and grouping will be done in Layout mode
automatically (by snapping components each together). It means,
that user doesn't need to know about it so much. But users should
be capable of change this alignment additionally in some cases.
And that's why we would like to offer this as a second mode.
Next part is about possible approaches and ideas for the Align mode:
A) Simple switching between Layout mode and Align mode
User can switch into this mode by ComboBox situated
in the Editor's Toolbar, by
clicking on selected component (on the area out of text inside) or by ALT
key press when
component is selected (The A key is used instead of ALT in
the Flash prototype below). This solution is favoured and recommended
by HIE team.
figure 9 - Flash prototype of switching
between Layout and Align mode. Try to select component and use A key
B) Aligning could be done by special floating palette (present solution is
close to this and was presented on J1)
When user selects a components, then small align palette
with possibility to set align or fill of the component
will appear (see figure 10).
User can cancel it temporarily by ECS key press or hide
it from Help assistant popup menu (It's visible by
default).
This palette contains four buttons with arrows for
aligning and two buttons in the center for filling.
Palette is displayed close to selected component (preferred place
is above component) by default. The new position of the floating palette
should be determined in a case of more selected components or lack of space.
The palette can be dragged by mouse on different position inside designer.
Then it's switch to the draggable mode and it can be switched back to the
automatic mode by action from contextual menu invoked above palette.
figure 10 - Align palette will appear by
selected component by default (1st picture) and can be draggable somewhere else
(2nd picture)
C) Set the alignment from Editor's toolbar (was done like first temporary
solution)
The same arrows (left, right, up, down)+handlers for horizontal and vertical
fill which were mentioned above will be situated in the toolbar as a toggle buttons
(see figure 11).
figure 11 - Arrow buttons and fill property buttons in the
toolbar
Note: This solution isn't recommended by HIE team because
handlers are far away from components in design area and buttons are in the sequence
which doesn't have a sence.
D) Set the alignment from the small palette in contextual menu
This approach is good compromise because palette isn't so obtrusive at the
first moment, but user can find it quite easy still.
figure 12 - Possibility to change alignement of component
from the contextual menu
E) Set the alignment of component by menu items situated in contextual
menu above component
The easiest implementation way is to put all six actions into the contextual
menu like submenu with menu items, but then it's meaning isn't clear
and illustrative enough.
Note: We discussed more variants for the Align mode and the best approach
is opened issue still (HIE recommend A or D approach).
Feedback of alignment have been changed many times according to discussions
and usability studies too. The present solution witch anchors to the nearest
container or another components is visible on pictures above.
Other Modes
Let's mention other modes that have been listed in intro of this document
with short description.
Edit text of components - All texts inside components will be displayed
as editable textfields above components. So user can quickly change all of
them.
Edit name of components - The same like previouse mode but with code names
of components.
Set TAB traversal - Possibility to define TAB order by components quickly
only by assigning number of order by clicking on them. It will be visualized
like knots with numbers linked by lines.
Connection mode - User can define connections between components visually
by D&D instead of connecting components by old Connection Manager invoked
from Editor Toolbar. Then rest steps of precise definition of appropriate
action will be reused from that Connection manager wizard after D&D.
Toolbar of Editor
The Toolbar contains four areas with different functions.
Let's describe it in the detail from left to right:
There are two toggle buttons for switching between Source
editor and Design editor at the left side
Then there is a area with actions (Preview action
and Check Design action, Guide lines switching action,
action for switching on/off Anchors witch dotted lines,
Locking/Unlocking resizeability of some components).
Next actions are intended for aligning of component
groups (different types), two actions for setting the
same horizontal or vertical size and Group/Ungoup toggle button (open issue).
Switcher among modes (Combobox) is situated at
the right side of the Toolbar.
Actions for Palette customizer and Padding customizer can be situated
at left side of that Combo box in Layout mode, but it should be
discussed more (open issue)
figure 14 - Editor Toobar with Group+Align actions, Auto resizing
and other functionality
Contextual menu above component(s)
The contextual menu should contain the following additional actions:
|-------------------------------- | Edit Text | Change Variable Name... | Events > | Preview Design... |-------------------------------- | Anchor > [ ] Left | [ ] Right | [ ] Top | [ ] Bottom | Align > ( ) Left to Column | ( ) Right to Column | ( ) Top to Row | ( ) Bottom to Row | -------------------------- | ( ) Left | ( ) Right | ( ) Top | ( ) Bottom | Auto Resizing > [ ] Horizontal | [ ] Vertical | Same Size > [ ] Same Width | [ ] Same Height | Set Default Size | Space Around Component... |-------------------------------- | Set Layout > Group Layout | Other Layouts | Customize Layout... | Add From Palette > | Change Order... |-------------------------------- | Cut | Copy | Paste | Delete |-------------------------------- | Properties... |--------------------------------
Anchor - User can change anchors by
D&D-ing of selected component(s) or by actions in the Anchor submenu. Align - The same actions like in the Editors toolbar for
alignment+grouping, when more components are selected. Auto Resizing - User can define (for
selected components) horizontal or vertical
ability to be resized automaticaly due to dialog (container) resizing. Same Size - Posibility to define same size by more selected
components. These actions should be situated in the toolbar too. Visual
feedback by those components will be made by small badge on right upper
side (combinations: and
see position on figure 14). Unlock/Lock resizeability - Resizeability in some direction
can be locked by components by default (e.g. height of textfields, buttons,
comboboxies, labels). So this action will unlock it. Set Default Size - Possibility to cancel the exact
size which was defined by mouse manipulation or in the Properties Space Around Component... - Will invoke dialog and user
can define exact size of empty spaces around selected component (solution
for promo F). But better approach is to incorporate the definition of spaces
into alignment area on figure 12.
Tools menu
Customization of the Palette can be invoked from Tools menu too by the following
action
Switching from other LayoutManager to Group Layout will cause some inteligent
rearangement of components. It can be done automatically if it's possible
or with warning dialog wit message describing, that the result of conversion
could be strange and far away from perfect state.
Highlighted panel during mouse over state or during visualization of
anchors.
Customizer for different padding among components and borders of containers
should exist in the future.
Ideas to the future
Close cooperation with other windows in the IDE, mainly with the Property
Sheet and the Inspector. We haven't been focused so much on this area yet.
Creating custom combination of components in the Component Palette. One
possible way is to D&D actual multiselection from the Designer to
the Palette. Another one is importing new set of custom components in
the Palette Customizer.
Showing exact size (+lock status) of components in the status line (figure
2)
Visualization of real dialog with transparent effect in the background.
There is the only way how to start new GUI Builder (through New... wizard)
but new user can have problem to find this way. So some more straight way
can exist.
Special Layout menu in the main menu containing current actions from Contextual
menu + typical ones in other tools
5. Open issues
Free Design is draft name and we should check if it's the right one.
Actions for Palette customizer and Padding customizer can be situated
in the Editor's toolbar, but it could be discussed more.
Add Centre Horizontally and Centre Vertically actions to the contextual
menu and Editor Toolbar.
Should be there some difference in behavior when user uses Drag&Drop
or Pick&Plop? E.g. in showing guide lines during adding components
or not showing them. Or popup menu
with basic set of Properties can appear immediatelly after Pick&Plop.
We need to check (e.g. on usability) if the ability of
adjusting of group position automatically (according to
new high of the group) is right approach and user will
understand/appreciate it. If it isn't, then we should
offer only action for auto-correction invoked by user.
The exact rules for alingning and grouping should be
defined elsewhere.
Still isn't clear whether dynamic adjustment of groups
should be done automatically or manually.
6. Unimplemented features in Promo F
Anchoring can be changed from the contextual menu only
Grouping into column or row hasn't any special visual indication and is
presented like kind of Alignment actions rather
Resizing of Multiple components at once
Special mode for multicopying of selected components mentioned in section
"Copying a component"
Some differences in Keyboard manipulation
There is no Combo in the Editor Toolbar including mentioned modes.
Possibility to switch on/off Guide lines and Anchors (actions in the Toolbar)
User can't lock/unlock resizeability of components (e.g. labels, buttons)
from the Toolbar
There is no global customization of padding
Visualization of the real dialog in transparent mode (visible e.g. on
figure 2)
and
see position on figure 14). 
