Background Research

Motivation

Current ubiquitous computing research is exploring the ways wireless artefacts can acquire and analyse real-time sensor data. Sheridan's PhD Thesis explored how the invisible and under-explored phenomena acceleration can be used to drive ad-hoc artistic expression through naturalistic interaction through the use of accelerometers [1]. During her research, she collaborated with external partners to develop a prototype ubiquitous computing system for wireless, peer-to-peer interaction and used this system for highly mobile and wireless ensemble performance [2] for indoor and outdoor environments. (At the time of the grant application), two technology demonstrations have taken place [3, 4] and an international journal paper was produced discussing an early prototype [5].

Sheridan's results highlighted several problems which make the collection of acceleration data through wireless, peer-to-peer networking an interesting data set. These include:

• The high data rate at which acceleration is streamed,
• Variability between patterns,
• Signal noise,
• Lossiness.

Sheridan identified the need to:

• Identify the algorithms that are used to interpret off-line analysis of acceleration data,
• Determine whether these analysis tools can be redeveloped for real-time analysis,
• Develop a prototype algorithm,
• Collection and analysis of large sets of acceleration data,
• Test algorithms for real-time analysis of acceleration data.

To collect the large data sets, the applicant will collaborate with external partners.

 

Research Phases

Small Grant - completed

• Phase 1 (Weeks 1-3): Interviews and summaries
• Phase 2 (Month 4-7): Testing of existing algorithms, data collection
• Phase 3 (Month 8-13): Prototype development, data collection
• Phase 4 (Month 14-15): Summary of key findings, additional publications ([6], [7], [8], others in progress), website

Ongoing Research - in progress

• Grants: EPSRC, AHRC grant
• Visuals: 3-D animated graphics
• Audio: 3-D sound

 

Investigation: Interviews

Sheridan identified several key research areas which are currently investigation the use of acceleration data. She interviewed several people and includes a summary of these interviews below. Detailed interviews available by request.

 

Games Development for Nokia

Dr. Paul Coulton, Deparment of Communications at Lancaster University
Accelerometer: 3D, 6G accelerometer with approximately 11bit accuracy.
Objective: Identify ways to use accelerometers embedded in mobile phones as input devices for games. For example, Snakes, car racing or Marble Game (as tilt sensor) or Graffiti virtual spray can (as gesture recognition). The Nokia 5500 phone contains a 3-D accelerometer. Using this accelerometer coupled with BlueTooth technology, gamers can use their mobile phones like a wireless mouse to control motion on a screen. Accessing the 3-D motion sensors requires the use of the Symbian Sensor API which is similar in function to J2ME’s Mobile Sensor API (JSR-256).

 

Wearable Computing

Christos Efstratiou, Computing Department Lancaster University.
Accelerometer: Single axis ADXL (710?), 10G. Initially used 50G but not necessary.
Objective: Identify time spent drilling by detecting when drill is on or off The collaborative project NEMO (Computing, Management Science, Psychology) is exploring the areas of Health and Safety, workplace monitoring and self-monitoring. This particular project looks to indentify the amount of time that workers spend drilling (to avoid "white knuckle syndrome"). An accelerometer is placed on a drill. Information is streamed from the accelerometer to a wearable badge which the worker wears on their uniform. Workers can then see how much time they spend drilling and thus know when to stop.

 

Security (for mobile phones)

Rene Mayrhofer
Accelerometer: 3D, 6G accelerometer with approximately 11bit accuracy.
Objective:
Summary:

 

Tangible Interfaces

Nic Villar, Computing Department, Lancaster University
Accelerometer: 3D, 6G accelerometer with approximately 11bit accuracy.
Objective: Creating a reconfigurable interface. The Pin&Play is a new approach of ad-hoc networking among objects that people can attach to large surfaces, such as notes that people pin to notice boards or artifacts that people hang on the walls in their home. Initially, it incorporated augmentation of common vertical surfaces such as walls and notice boards with low-cost conductive material to create smart surfaces as a communication medium. The objects are attached to such surfaces by means of simple pin connectors, to provide users with a familiar mechanism for adding objects to the network. More recent applications include creating a reconfigurable surface for DJs and gaming. Various "pins" are fitted with accelerometers and when the pins make contact with the wired surface the acceleration data is then sent to the controlling PC. The PC interface can then be controlled by manipulating the pins (ie. rotating the pins left and right).

 

Summary of Interviews

Of the four projects discussed above, three are using the accelerometers as binary tilt sensors (left/right, up/down or on/off) - effectively as tilt switches. Therefore threshold detection is sufficient for real-time interaction. Generally, analysis was done post-hoc - data may have been collected in real-time but analysis of the data was done only after data collection. Despite the similarities, there was much variation in the language used to describe the type of activity being detected - some called it "motion detection" whilst others called it "vibration sensing". Three projects (Games, Wearables, Tangibles) used the same accelerometers - 3-D, 6G accelerometer with approximately 11bit accuracy. Power is an issue - can accelerometers be self-powered? In all cases, specialised software/hardware is needed for each application.

 

Direction for prototyping

The following questions arose from our interviews:

• What other ways can we analyse acceleration data in real-time?
• What kinds of algorithms can we use?
• What comparisons can we make to other types of accelerometers/products (such as Wii)?
• Can we combine accelerometers with other sensors to make sense of the data?
• What other applications make sense?
• Can we build simple tools for visualising and sonifying the accleration data?

 

Prototyping Equipment, Algorithms and Testing (currently being updated)

Equipment

Sheridan purchased several types of accelerometer boards and products and conducted experimental evaluations and tests, each of which is described below.

Sparksfun Electronics

• Wireless Accelerometer/Tilt Controller version 2.5
• Wireless Accelerometer/Tilt Controller version 2.5 - RP-SMA

Nintendo Wii

• Games Console
• Additional Nunchuck controller
• Additional Wii controller

TMote Invent

• 8 individual modules

Summary of Findings

Available by request.

 

Acceleration Publications

1. Sheridan, J.G. (2006) Digital Live Art: Mediating Wittingness in Playful Arenas, PhD Thesis, Computing Department, Lancaster University, UK.
2. Sheridan, J. G., Bayliss, A., Bryan-Kinns, N. “iPoi.” Public performance. F.city. Ludus Dance Studios, Lancaster UK, 29th September, 2006.
3. Sheridan, J. G., Bayliss, A., Bryan-Kinns, N. “iPoi.” Demonstration at the CCID Symposium, at Engage: HCI ‘06, London UK, 12th September, 2006.
4. Sheridan, J. G., Bayliss, A., Bryan-Kinns, N. “iPoi: acceleration as a medium for digital live art.” Demonstration at the 8th International Conference on Ubiquitous Computing (Ubicomp’06). Orange County, California, USA, September 2006.
5. Bayliss, A., Sheridan, J.G., & Villar, N. “New Shapes on the Dancefloor: Influencing ambient sound and vision  with computationally-augmented poi.” International Journal of Performance Arts and Digital Media. Intellect: Bristol. March 2005.
6. Bryan-Kinns, N. and Sheridan, J. (2007). Supporting Mutual Engagement in Creative Collaboration. Workshop on Tools in Support of Creative Collaboration, 6th Creativity and Cognition Conference, 13 June, Washington, USA.
7. Sheridan, J.G., Bryan-Kinns, N. and Baylss, A. (2007). Encouraging Witting Participation and Performance in Digital Live Art. 21st British HCI Group Annual Conference, 3-7 September, Lancaster, UK.
8. Sheridan, J.G., Bayliss, A. and Bryan-Kinns, N. (2007). The interior life of iPoi: objects that entice witting transitions in performative behaviour. International Journal of Performance Arts and Digital Media 3(1). Intellect: Bristol.