Lecture Week 1


lecture 1lecture 2lecture 3lecture 4lecture 5lecture 6lecture 7lecture 8lecture 9lecture 10lecture 11lecture 12workshop 1workshop 2workshop 3workshop 4workshop 5workshop 6workshop 7workshop 8workshop 9workshop 10workshop 11workshop 12Default HomeReal HomeFlex PageReadingsTut 1Tut 2Tut 3Tut 4Tut 5Tut 6Tut 7 lecture 1lecture 2lecture 3lecture 4lecture 5lecture 6lecture 7lecture 8lecture 9lecture 10lecture 11lecture 12workshop 1workshop 2workshop 3workshop 4workshop 5workshop 6workshop 7workshop 8workshop 9workshop 10workshop 11workshop 12Default HomeReal HomeFlex PageReadingsTut 1Tut 2Tut 3Tut 4Tut 5Tut 6Tut 7
 
<course profile: main assessment> <default page home> <real page home> <flex page> <readings> <updates> < forum>
<lectures: 1 2 3 4 5 6 7 8 9 10 11 12> <workshops: 1 2 3 4 5 6 7 8 9 10 11 12> <tutorials: 1 2 3 4 5 6 7 >
 
 
 
 

LECTURE WEEK 1 MMST 12016

SOUND, AUDIO, LISTENING AND MEANING

By Brendan Murphy


Objectives

After this week's lecture you should:

 

  • Understand the fundamental technical qualities of sound, and understand the basic parameters of digital audio.
  • Be aware that the relationship between audio and meaning is different in quality to the relationship between the written word and meaning, or the image and meaning.
  • Be able to discuss the relationship between audio and meaning with reference to some of these audio-specific qualities.
  • Be able to discuss audio using the analytical schema developed by Michel Chion.
  • Be aware of the limitations of Chion's schema for multimedia and be aware of some of the ways his approach can be extended to be more useful for multimedia.
  • Be able to discuss uses of digital audio with reference to Chion's model.

 

Readings

Rumsey, Francis, Sound Recording: An Introduction, Oxford: Focal press 1992, pp176-179. [READING 1]

Chion, Michel, Audio-vision: Sound on Screen, Columbia University Press, pp 25-34. [READING 2]

filmsound.org, “Michel Chion's terminology”, http://www.filmsound.org, accessed 27/05/02.

The Soundry http://library.thinkquest.org/19537/ accessed 27/05/02

 

Introduction

Rock music blares, doors slam

people yell, children scream

sirens whine, trucks rumble and roar

and rock music blares. Blares, blares, blares.

Is there any escape? From noise?

“Escape from Noise”, Negativland,  SST Records 1987

 

The US based sound artists and alternative music outfit Negativland once based an entire CD around the different types of noise that surround us in our daily lives. Over the course of an LP they were repeatedly asking the same question: Is there any escape from noise? This seems like an interesting point from which to begin this lecture, as it reminds us of the difference between audio and visual sources of information – and noise.  The senses of sight and hearing are fundamentally different. The  written word, for example, is something we can turn our atention to, and cut out of our awareness if we wish. Sound on the other hand, is something that surrounds us, and while we can deal with it in many different ways, we cannot control our access to it via our gaze.

In this lecture we will be considering some of the ways in which audio differs from other types of communication. In doing so we will be considering the nature of sound and of hearing. We will also be covering the basic physics of sound.

The Physics of Sound

As common for multimedia courses at CQU, this course covers practical skills and theoretical approaches. While the practical component of this course focuses on the skills needed to work effectively with digital audio, it is important to introduce some basic technical terms and concepts. In order to introduce this terminology I want to begin by taking a brief detour into basic acoustics, or the physics of sound.

One good introductory resource can be found at The Soundry. Visit this site now. In the "Physics" section, read “What is sound?” and “properties of a sound wave”. Also page your way through the "Ear" section, paying particular attention to the section entitled "Auditory Scene Analysis"

[aside: The Soundry is one of the many web sites hosted by thinkquest.org. Thinkquest is a non-profit organisation hosting educational material created by and for school teachers and students. While you are patently not school students, be aware that the physics and  biology of sounds are not an assessable part of this course. For many of you, this will be a refresher. Others will no doubt be relieved that we only require a school level coverage of these scientific topis. While visiting the Soundry, please take the time to read the ThinkQuest disclaimer [<http://www.thinkquest.org/disclaimer.html>]

You should take careful note of the following terms:

  • wavelength
  • amplitude
  • period
  • pitch

Also, take note of the physical process that underlies sound. Sound is a form of compression wave, propagated through any medium. In most that we experience, that medium is the air. In that sense, there is no escape from noise, as we are immersed in the very medium through which sound propagates. As we shall see later, that has some very important implications for our understanding of audio.

Take note also of the discussion of the sound field analysis that we carry out  as we interpret the audio sources around us, as this will have bearing on later discussion regarding the qualities of sound that differentiate it from visual modes of communication.

Principles of analogue and digital audio.

While the focus of  this course is digital audio, it is necessary to understand analogue audio to be able to understand what is significant about digital audio. You should also be aware that not all audio processes that go on inside your computer are digital. Some processes, for example capturing sound from your minidisc players or other recording devices involve analogue stages, and many audio devices in a PC, such as a  CD player, can produce both analogue and digital output.

The above readings illustrate the basic technical parameters of sound. You can see that a sound  is a continuous wave. At any point in time, the wave has a particular amplitude, which changes with time according to the frequency of the sound. The way sound is stored by analogue media is very similar to the way in which sound is transmitted through a physical medium. For example, consider the way in which an anlogue device such as a tape recorder converts sounds in a physical medium to a stored representation of a sound.

In the above illustration, ‘A’ represents the sound wave. In other words, the variations in air pressure over time that constitute a sound. ‘B’ represents this as a wave, where the amplitude (in physical terms, the compression or rarefaction of the air) is represented on the vertical (y) axis, and time is represented on the horizontal (x) axis.

The box represents the device used to continually measure the compression of the air and to produce a representation of the sound wave in a different medium. ‘C’ is a transducer, an electronic device that generates a voltage (‘D’) that continuously varies according to the amplitude of the sound wave. ‘E’ represents the output. In the case of magnetic tape, this would be a record head, and its output would be a magnetic field, the strength of which at any time is determined by the voltage. Passing a magnetic tape over this record head at a constant speed would effectively write a continual trace on the storage medium (tape). This is represented at ‘F’. Whereas the dense areas of ‘A’ represent the peaks of compression in the air, the dense areas of ‘F’ would represent areas of high magnetic field.

Playback of the sound is essentially the same process in reverse. This time, we would start with the tape moving over a transducer (in this case a read head). This transducer would generate a voltage that would vary continually following the variation of magnetic field strength. This voltage would be fed to an output device, in this case a speaker, which would produce compression and rarefaction of the air following the voltage produced by the transducer.

The "advantage" of analogue technology is that the copy which is made is continuous, and represents the sound wave at every point in time. However analogue technology has inherent disadvantages, due to basic laws of physics governing the nature of energy and work.

It might not be immediately obvious that sound is a form of energy. Think of what it feels like to stand close to a large, loud speaker playing music with a heavy beat. Very low frequencies can be perceived not only by our ears, but by our sense of touch registering the force applied by the compression wave to the skin. The very process of hearing is a process of energy transfer, as sound is converted to mechanical movement by the eardrum, and then to vibration in the cochlea. The ear is in effect a set of biological transducers converting the sound waves to different forms, the last one in the chain generating nerve signals to the brain.

Analogue technology allows us to produce a continuous copy, or representation, of a signal. Regardless of the quality of the transducer that makes this conversion, the copy can never be perfect. Two of the basic laws of physics prevent this. While the physics of sound is not the main focus of this lecture, or of this course, it is worth examining why this is so, as it helps us understand the advantages of digital audio technology.

The First Law of Thermodynamics states that, in any process where two systems interact, energy is always conserved, The total amount of energy put into a system by a particular process must always equal the total energy output by a system. Consider once more the example above, whereby compression waves in the air are converted to differing magnetic fields on a magnetic tape. The energy being input must pass through the transducer, and in the process of conversion, energy is lost. This energy loss might affect the analogue recording in a number of ways. For example, high frequencies might be lost, amplitude over certain frequencies may be reduced, or other forms of distortion might creep in.

The Second Law of Thermodynamics states that in any system entropy, the mathematical measure of disorder, must increase with time. Consider how this might apply to magnetic tape. Under a microscope, the ferous coating on a magnetic tape is revealed as a multitudes of small magnetic domains, each acting as a small bar magnet. If the magnetic fields of the domains are arranged randomly on the microscopic level, the result on the macroscopic level is a negligible measurable magnetic field. If in a particular region the magnetic fields of the domains are aligned, on the macroscopic level there will be measurable magnetic field. The process of recording introduces order into the randomly aligned magnetic domains. The Second Law of Thermodynamics suggests that this order will diminish over time.

The British scientist and writer C.P. Snow once famously summed up these laws as follows:

1.You can't win

2.You can't break even.

In practice, this means that non-digital, storage media, such as magnetic tape and film stock, are prone to degredation. In addition, each time information is transferred via an analogue process, the information being tranferred degrades. A copy of a copy of a copy of a cassette tape will contain a very weak signal, and a lot of noise.

Digital media relies on a different process. Rather than recording a continuous trace, digital recording takes a regular series of snapshots, and records as data the level of the signal being measured at regular intervals. This is explained in the reading taken from Chapter 10 of Rumsey and McCormick's Sound and Recording: an introduction.
[READING 1]

The digital representation of an audio (or other) signal is in essence a series of numbers which require interpretation by hardware or software. Rather than being a copy or representation of  the sound signal (albeit with loss and distortion)  the digital representation is effectively an approximation of the sound.

Since sound information is perceived by our sense of hearing, the final step in any digital process involves the conversion of the digital signal to an analogue form – the sound wave. However with digital technology, all the processing steps between sound capture and playback of final product are computational, not physical, so in general loss and distortion is far less. A voicemail system with a digitally stored greeting will not degrade, no matter how long it has been stored, whereas an old fashioned tape driven answering machine message will sound a little worse every time a message plays back, and this degradation becomes very noticeable over time.

The physics and biology of sound and theoretical approaches to audio, meaning and culture.

In this lecture I have spent a lot of time discussing scientific topics, albeit at a very basic level. The aim in discussing this theoretical material is not to produce budding physicists or physiologists. Indeed this course aims to provide you with a range of perspectives on the way audio works within a culture to produce meaning, in order that the digital audio skills you develop may be deployed intelligently and creatively. Part of the diversion into scientific territory has been to give you the basic knowledge you will need to understand the technology with which you will be dealing in this course. The other important reason for spending time on sound and hearing is to help you realise some of the specific qualities of audio that proceed and delimit any symbolic engagement with audio texts.

In other words, recognition of the physical and biological processes involved in sound and hearing is not the same thing as understanding the different ways in which we make use of sound for cultural and communicative purposes. There are differences between the auditory and visual senses, and therefore we engage with visual and auditory information in different ways. We are less interested in a detailed analysis of the ways in which we apprehend the audio and the visual than we are in the difference between them, which is profound. One branch of philosophy that would amenable an analysis of this qualitative difference is Phenomenology. 

Husserl and the Phenomenologists were less interested in inquiring into the physical world, preferring to focus on the immediacy of experience. Husserl argued that we did not have access to the world, but to the phenomena that our senses had access to. Such an approach would be of only partial use to us here, however it is important to identify that the difference we are talking about here is a phenomenological difference.

Recall the material from the Soundry site that briefly described the way in which our sense of hearing is used to interpret and analyse an auditory field. Analysis of visually-based texts, such as the written word or cinema, take generally take for granted an active participation  on behalf of the reader (or viewer, or listener). However the auditory sense is not directional as vision is. The auditory world we experience phenomenologically is an engulfing field, and we have different strategies for engaging with it.

One theorist who has spent some time considering the different modes of listening we make use of, and their significance, is Michel Chion. Chion is primarily interested in film, but unlike many film theorists he is not constrained by the primacy of vision and the gaze in describing and analysing film. In fact, his seminal work Audio-Vision places the auditory before the visual, at least in its title.

Now read an excerpt from Chion's Audio-Vision, where he outlines three different modes of listening. As you read, consider to what degree these filmic listening modes can be extended to describe our interaction with sound phenomena more broadly.

Michel Chion, Audio-Vision [READING 2]

Chion's three modes of listening are quite interesting, and you should keep them in mind as you work through this course. The assessment pieces in particular should allow for you to design audio texts that work across all these modes. Conceptualising the sounds you produce in these terms should help you produce effective and well-rounded material.

Chion's nomenclature of sound involves many more categories. Rather than go into them in detail, I ask that you visit the filmsound.org site and read the definition of each of Chion's terms.

 

NOTE: THERE ARE DISCUSSION QUESTIONS REGARDING THIS LECTURE AND, IN PARTICULAR, CHION'S 'MODES OF LISTENING' IN TUTORIAL #1.