Hearing, speech and touch

Sound grabs attention but carries little detail

Intuition

You can look away from a screen. You cannot easily look away from a fire alarm. That is why sound is the channel for alerts and a poor channel for numbers.

Lecture 2 describes sound as pressure waves picked up by the ear. The range of human hearing is considered to be 20 Hz to 20 kHz, but we are far more sensitive between 1 kHz and 4 kHz. Intensity is perceived as loudness and measured in decibels: whisper 15 dB, conversation 60 dB, car horn 110 dB, rock concert 120+ dB. Frequency is perceived as pitch and measured in Hz. Timbre is why different instruments have different sound.

Sound alerts warn the user of abnormal or undesirable situations. The lecture’s identification exercise used a fire alarm, an ambulance, a car horn and a car parking sensor. Sound alerts also appear as earcons (audible alerts that convey events) in Lecture 3.

Compare

Advantages of sound alerts

• Reduce demands on visual attention
• Reduce the amount of information needed on the screen
• Attention grabbing: users can choose not to look, but it is harder to avoid hearing

Disadvantages of sound alerts

• Low resolution: cannot convey high detail
• Presenting absolute data is difficult
• Need two sounds or vibrations to tell if a value is going up or down
• Annoyance to the user and to others who overhear

Exam detail

Sample exam question 13 marks these as the advantages: reduces the need for visual attention, allows users to multitask more easily, reduces the amount of information needed on the screen. Wrong options were “increase screen clutter” and “always preferred over visual alerts”. The word “always” is the giveaway.

Speech carries more but costs more processing

Speech conveys more information than plain sounds but needs more processing. A phoneme is the smallest unit of sound that distinguishes one word from another. Accent and speed affect understanding. In phonemic restoration, shown by Richard Warren (1970), people do not notice parts of speech that have been removed because perception fills them in. Sine-wave speech (Remez et al., 1981) shows speech can be understood without the usual speech cues.

Design implications: sounds should be audible and distinguishable, comfortable to the listener, and speech output should be intelligible. Accentuate intonation of artificial voices, since they are harder to understand than human voices.

Touch is skin, muscles and joints

The lecture uses ISO 9241-910:2011, the framework for tactile and haptic interaction. Haptics is sensory and/or motor activity based in the skin, muscles, joints and tendons, and covers two parts. Tactile is connected to the sense of touch. Touch is the sense based on receptors in the skin. Kinaesthesis is sense and motor activity based in the muscles, joints and tendons: the sense of self-movement and body position.

There are three main modalities. Discriminative touch uses mechanoreceptors (receptors for tactile sensation, in the skin, so cutaneous). Temperature and pain use free nerve endings. The kinesthetic senses use muscle and joint receptors and include proprioception. Fingertip spatial acuity is about 1 to 2 mm, found with the two-point threshold test.

Pitfall

Do not file proprioception under touch. It belongs to the kinesthetic senses (muscles and joints), while touch is skin. Likewise, tactile and haptic are not interchangeable: haptic is the umbrella and tactile is one part.

Optional aside

The mechanoreceptor table on the slide gives four receptor types (Merkel’s disks, Ruffini endings, Meissner’s corpuscles, Pacinian corpuscles) with adaptation type and frequency bands, for example slow-adapting types in the 0 to 30 Hz and 0 to 15 Hz bands and rapid types up to 80 to 400 Hz. The extracted text scrambled the function column, so the receptor-to-function pairings cannot be read reliably here. Check the PDF before memorising them.

Try it: worked example

Worksheet W3 asks for sounds that tell road users an electric vehicle is approaching, at high or low speed. How would you apply the lecture?

Show answer and steps

AnswerUse audible, distinguishable, comfortable sounds, with pitch or a second sound distinguishing speed.

1. Need: alert people who are not looking at the car, so sound is the right channel (it is attention grabbing).
2. The sound must be audible and distinguishable from background traffic noise.
3. It must be comfortable and not annoying, since people nearby overhear it.
4. Sound has low resolution for absolute data, so do not try to encode exact speed. Use a change such as a higher pitch for faster travel, which signals up or down.
5. Real examples in the worksheet resources include the Jaguar I-PACE vehicle sound.

Recall

Name two advantages and two disadvantages of sound alerts.

Advantages: reduces visual attention demand, reduces on-screen information, hard to ignore. Disadvantages: low resolution, absolute data is hard, needs two sounds for up or down, annoyance.