Large Interactive Display Surfaces (LIDS)
The large interactive display surfaces (LIDS) concept started with the
creation of the "Whiteboard Paradigm". There were many
available technologies that could be used as LIDS, however, most
were prohibitively expensive, and many still did not support
appropriate interaction styles. The goal of the LIDS research project
has been to develop inexpensive technologies to use as
displays, and investigate the interaction issues generated by their
use. Furthermore work has gone into investigating potential uses for
such technologies, and creating the software to support these uses.
In personal computing, "direct manipulation" interfaces
such as the desktop metaphor are used extensively. These interfaces
are helpful to the new user; by providing familiar cues and a direct
mapping from interface to action they allow a user to learn an
interface very quickly. Furthermore as experience with computers has
become widespread, developers can rely more on users knowing concepts
like the "double-click" that are not immediately obvious from
When a group of users need to use a computer together though, this
seemingly direct manipulation interface is shown not to be as direct as
it initially seemed. An object on the screen is not moved directly
"on-screen" but manpulated by moving the mouse which is next to
the computer. This is fine for a single user — because they are
moving the mouse, the manipulation feels direct. However, when someone
not in control of the mouse has an interest in what is happening in
the interface, mouse movements are too small and obscure, resulting in
the need for a more direct manipulation.
The whiteboard paradigm of interaction is one possible answer to the
lack of directness experienced by users when more than one person is
using a traditional machine. With a whiteboard What You Do Is What You
See (WYDIWYS), and therefore the entire interaction is transparent to
everyone involved. When interacting with a computer within the
whiteboard paradigm, interactions will be performed with a pen-like
device in a manner paralleling a pen on an ordinary whiteboard, and the
software will support whiteboard-style interactions. Advantages over
using an ordinary whiteboard include being able to capture information
to a hard disk for later use before clearing the screen, the ability to
use traditional computer applications such as Microsoft
PowerPoint™, and interoperability with other devices (such as
The main characteristics of the whitebaord paradigm of interaction are:
a shared visual space;
total transparency of interaction;
use of an existing medium for meeting support;
encouragement of spontaneous interaction.
These characteristics rely on the informality and ubiquity of ordinary
One of the goals of the LIDS project is to develop an affordable
technology from readily available materials. Presently a LIDS package
All of these components are (comparatively) cheap and readily
available. The Mimio® pen is an integral part of the LIDS
research, at least here at Waikato, and some work has gone into
researching how ordinary PC software can be operated with the pen.
The development of this hardware has allowed research into how the
LIDS screen can be used for lecture capture and meeting support, these
being two of the area where ordinary whiteboards are commonly used and
where the advantages of the LIDS technology could be very beneficial.
To this end, two major pieces of software have been developed:
- "gesture recognition" software that allows
operation of Microsoft PowerPoint™ via pen input (including
slide navigation and drawing on slides);
- the "shadow software" for remote meetings, which
given two LIDS screens with attached video camera will cast a
"shadow" of the person standing in front of one
screen onto the other. Combined with the actions from one
screen (e.g. writing or sketching), appearing on the other,
this promotes awareness amongst remote users.
This software has already undergone some usability testing in University of Waikato
There is a great potential for research with the LIDS technology,
both in hardware and software directions. At present work is going on
in four different institutions to a variety of ends. Major directions
LIDS has great potential for informal interaction by means
of handwriting, rather than keyboard use. However, handwriting
recognition software is still not accurate enough to allow interaction
which is not error-prone and frustrating. Investigation into how best
to avoid this frustration when using a LIDS screen is ongoing. One
possible solution is to use the "Graffiti" input method,
developed for the Palm®.
Another area where the LIDS screen has great potential
is in mathematical equations; at present these are tiresome to
digitise. Investigation into how LIDS could allow these to be written
naturally, and then convert them to some recognised format (such as
Microsoft Word™) is ongoing.
LIDS as a software development tool
Beryl Plimmer is
investigating how interfaces drawn on the LIDS screen may be
translated directly into Visual Basic. This would enable very rapid
prototyping of interface ideas, even by interface designers without
strong technical skills.
John Grundy at the University of Auckland is
investigating how the LIDS technology might be used as a software
engineering tool, for example in capturing UML diagrams or use case
scenarios. This works in with the use of LIDS for meeting support.
This research may lead to LIDS technology being integrated with
industry standard software, such as Rational Rose.
of Waikato Usability Laboratory utilises the LIDS technology in
collaborative design activities.
Hardware and technology development
The LIDS screen is a technology which is still under
development. There is great potential for investigation into both
hardware and the underlying software technology employed by LIDS.
Here at Waikato, an attempt is being made to write a Mimio®
driver for Linux, and investigation into how to detect the use of
more than one pen on the screen at one time is ongoing.
At University of Auckland, Rick
Mugridge is investigating the use of a "double-ended"
pen, where both ends of the pen are detected by the software.
This allows the angle of the pen to be used as an input; potential
uses of this are LIDS-based calligraphy, and mode changes based on
The Auckland University of
Technology team are developing the LIDS hardware, including
investigating packages of hardware for schools, and varying
configurations of hardware for different environments. This team
is examining new screen materials, different pen technologies, and
projector requirements as part of their work.
Use and usability
There is huge scope for investigating possible uses and
usability of the LIDS technology. These investigations might
include traditional usability, sociological studies of how a LIDS
affects group interactions, studies of the LIDS as a mechanism for
e-learning... there are many possibilities.
There is one study currently planned by the Auckland University of Technology
in conjunction with experts in education: LIDS is schools. Schools
will be provided with the technology and a list of possible ways to
use it. Once the technology is in place, its use and reception
will be observed.
Speech recognition and lecture capture
Some work has gone into capturing the speech of a lecture, and
using it to annotate Microsoft PowerPoint™ slides shown during a
lecture. This is coupled with the PowerPoint™ control software
described in The LIDS technology at present to
allow lecturers to give lectures with the LIDS technology, and
attach the speech they give to appropriate parts of the slideshow
automatically. Research into how best to allow lecturers to edit
the resulting annotations and speech is ongoing.
There are, of course, many other potential avenues of research with the
LIDS screen. What is mentioned here is just some of the research
currently being done.
- Apperley, M., Jansen, S., Jeffries, A., Masoodian, M.,
Paine, L., Rogers, B., and Voyle, T. (2002): "LLC Lecture
Capture and Editing Tool for Online Course
Delivery". Proceedings of E-Learn, World Conference on
E-Learning in Corporate, Government, Healthcare and Higher
Education, Montreal, Canada, October 15-19, p 1866-69.
- Plimmer, B. and Apperley, M. (2002): "Computer Aided
Sketching to capture preliminary design", in Porceedings of
the Third Australasian Interface Conference, Melbourne, Australia.
- Plimmer, B, and Apperley, M. (2002): "Freeform: An
informal environment for interface prototyping", in
Proceedings of the SIGCHI-NZ Symposium on Human Computer
Interaction Hamilton, New Zealand, July 11-12 p 11-12.
- Apperley, M. Dahlberg, B., Jeffries, A., Paine, L., Phillips,
M. and Rogers, B. (2001): "Development and application of
large interactive display surfaces", in Symposium on Human
Computer Interaction Palmerston North, New Zealand, p 3-7.
- Apperley, M. Dahlberg, B., Jeffries, A., Paine, L., Phillips,
M. and Rogers, B. (2001): "Lightweight capture of
presentations for review", in Volume 12 IHM-HCI Conference
on Human Computer Interaction Lille, France, p 41-42.
- Plimmer, B. and Apperley, M. (2001): "FreeForm: Informal
Design on a large interactive display surface", in
Symposium on Human Computer Interaction Palmerston North,
New Zealand, p 81-83.
- Plimmer, B. and Apperley, M. (2001): "From sketch to form
on a large interactive display surface" in 14th Annual
Conference of the National Advisory Committee on Computing
Qualifications, Napier, New Zealand, p 371-374.
- Apperley, M. and Masoodian, M. (2000): "Supporting
collaboration and engagement using a whiteboard-like display", in
Shared Environments to Support Face to Face Collaboration: A
CSCW Workshop, Philadelphia, Pennsylvania, p 22-26.
For more publications from this group, see our publications page.