Reaching criterion in phonetic transcription:

validity and reliability of non-native speakers

Lisa-Ann Lane, Jeannette Denton & Daniel Suslak

Sun. 9-10:40 C

A critical indeterminancy of sociolinguistic transcription is the relative weight of two factors: (1) what the transcriber believes a sound should be (based on native speaker knowledge and/or adherence to a phonological theory of the language) and (2) the language independent perceptual acuity representing what is actually produced by the speaker. Since much of current sociolinguistic and dialectological research utilizes phonetic transcriptions as the basis for determining (socio-)linguistic shifts and the like, it is crucial that we explore and understand the limitations of the tools we employ for deriving our data sets, especially at the base level of data collection. In order to address these important issues (cf. Nettelbladt 1993; Kerswell & Wright 1990; Vieregge 1987 & 1989; Wright 1983), the validity and reliability of phonetic transcriptions by three graduate linguistic students is being studied. An unfamiliar language was chosen in order to control for native speaker content based filtering of the signal as well as to control for the transcribers being "...influenced by their (phonological) knowledge of language variety of being transcribed." (Kerswill & Wright 1990:258)

The methodology for this experiment involved briefing transcribers on minor adjustments made to the IPA listening to a set of standardized pronunciation tapes for approximately 10 hours. They were then given two tapes, 10 hours apart. Each of these (style 1) tapes contained a word list read by an elderly female informant recorded in her home in Thyborøn, Denmark. After approximately 20 hours of training with the Style 1 tapes, a third tape (Style 2) was given to the transcribers, containing phrases taken sequentially from a longer segment of spontaneous speech from a sociolinguistic interview of an elderly male, recorded at his home in Thyborøn, Denmark. The transcribers were instructed to produce at least two separate transcriptions of each of the three tapes. The first pass was to be done without prior listening or consultation with the master transcription which was produced by a linguist who is a native speaker of the dialect. The second pass was done after the allotted training time was nearly past. This was phase 1 of the study. The transcribers were then each given seven excerpted sociolinguistic interviews to transcribe. Upon completion of the 7th interview (equaling roughly 40 hours of transcribing interviews) the transcribers listened to and produced one transcription each of the second of the Style 1 tapes and the Style 2 tape without consultation with the master transcriptions. This was Phase 2 of the study. Phase 3 involved the same methodology as Phase 2 in that after another six interviews (equaling roughly 40 hours more transcription) the transcribers again produced a transcription of the two aforementioned tapes. By having the transcribers listen to and produce transcriptions for the same tapes (isolated words and phrases) at three controlled stages during the academic year, we have been able to measure their transcriptions across two speech styles for inter and intratranscriber reliability and for agreement with the transcriptions produced by the native speaker linguist (i.e., validity testing).

Our data reveal that even after the relatively short training time the scores for both validity and reliability for valid transcription tests increased. The data from Phases 2 and 3 strengthen our support of non-native speaker transcribers in the production of phonetic transcriptions, hence supporting a more "analytic" (sound perception driven) as opposed to "semantic" (context driven) approach to transcription (cf. Vieregge 1987). Notwithstanding that some authors (e.g., Labov 1989) suggest transcription by machine authors as well as "auditory comparison", it is still not possible to associate instrumentally produced data with orthographic data and does not obviate the problem here. Our data underscores that more caution must be taken in the production and especially the use of transcribed data, regardless of the language abilities of the transcriber. We will address some of the many factors which come into play and affect the production of a phonetic transcription. The most obvious question to be answered is, of course, whether or not it is possible for linguistics to produce reliable and valid transcriptions.