Pen-based Musical Interaction

The ease-of-learning is an important factor deciding upon the success of novel musical interfaces. However, reducing the complexity of the musical interaction runs the risk of a too restricted expressive power of the instrument. Traditional instruments draw their expressivity to a large extent from the fine-grained control possibilities, which they offer to the performer. Traditional instruments require that performes practice the instrument for long periods until they have reached this level of fine-grained control. In contrast, pen-based musical interfaces enable the performer to use the motor skills already learnt for writing. This empowers the user to reach a high level of control more quickly.

We are currently developing a novel pen-based musical interface for live performance which uses digital pens on a tabletop display. We use the Anoto digital pen technology. The Anoto pen is a ballpen with a built-in camera near the pen tip. By detecting and decoding a nearly invisible dot pattern, which is printed on the surface the pen is used on, the pen decodes its current position. In contrast to other approaches for capturing pen input, the Anoto technology provides for input on large surfaces with a very high resolution (about 850 dpi). In addition to the sound which is generated in real-time by a synthesizer, the tabletop display provides visual feedback to the performer.

Our main interaction concepts are as follows:

  • The performer generates sounds by spanning virtual chords between two pens on the table surface.
  •  The instrument provides for collaborative use, letting multiple performers span chords between their personal pens.
  • In addition to the tabletop display, the performer can use interactive paper surfaces for creating sounds. While using paper as an input medium does not provide for co-located feedback, paper is mobile and can be flexibly arranged in the physical space depending on the needs of the performer. As a consequence, interaction with the instrument is not restricted to the flat table surface, but extends to the 3D physical space.


Aristotelis Hadjakos, Jürgen Steimle: "Stiftbasierte musikalische Interaktion", Workshop on Innovative Computer-based Musical Interfaces (ICMI'09), 2009.


Jürgen Steimle

Aristotelis Hadjakos

Tim Klein


If you are interested in our work, don't hesitate to contact us.




Two Full Papers Accepted at CHI 2018

We will present two full papers at CHI 2018: Off-Line Sensing: Memorizing Interactions in Passive 3D-Printed ObjectsMartin Schmitz, Martin Herbers, Niloofar Dezfuli, Sebastian Günther, Max... [more]

Category: Startseite, Allgemeine News


Full Paper Accepted at CHI 2017

We will present a full paper at CHI 2017: Flexibles: Deformation-Aware 3D-Printed Tangibles for Capacitive Touchscreens(Martin Schmitz, Jürgen Steimle, Jochen Huber, Niloofar Dezfuli & Max Mühlhäuser)  [more]

Category: Allgemeine News, Startseite


Hessian Minister for Science tested new version of TK's Smart Table, Permulin

During a recent event at the Future Innovation Lab of TU Darmstadt's spin-off company Leap-In-Time by Prof. Ruth Stock-Homburg, the Hessian Minister for Science and Arts, Boris Rhein, was one of the first to have a closer look on... [more]

Category: Allgemeine News


TK’s Smart Table featured in Commerzbank video about the future of work

Commerzbank, Germany’s second largest bank, produced a vision film about the future of work and workplaces. It features three areas of TU Darmstadt‘s spin-off company Leap-in-Time headed by Prof. Ruth Stock-Homburg from the... [more]

Category: Allgemeine News


Three Posters Accepted at CHI 2016

TK is going to present three Late-Breaking-Work posters at this year's CHI conference in San Jose. The accepted posters are: FreeTop: Finding Free Spots for Projective AugmentationJan Riemann, Mohammadreza Khalilbeigi, Martin... [more]

Category: Allgemeine News


Paper Accepted at UIST 2015

We will present the following paper at UIST 2015: Capricate: A Fabrication Pipeline to Design and 3D Print Capacitive Touch Sensors for Interactive Objects(Martin Schmitz, Mohammadreza Khalilbeigi, Matthias Balwierz, Roman... [more]

Category: Allgemeine News


TK Receives Surface Hub Award from Microsoft Research

Microsoft Research recently developed a number of prototypes of a novel electronic whiteboard. A worldwide competition was launched where pertinent research groups could propose innovative research projects that would leverage... [more]

Category: Allgemeine News

Displaying 1 to 7 of 18
<< First < Previous 1-7 8-14 15-18 Next > Last >>
A A A | Drucken Print | Impressum Impressum | Sitemap Sitemap | Suche Search | Kontakt Contact | Webseitenanalyse: Mehr Informationen
zum Seitenanfangzum Seitenanfang