Contexts and constructs: Implications for the testing of listening in pilots’ communication with air traffic controllers
DOI:
https://doi.org/10.23925/2318-7115.2020v41i4a4Keywords:
ESP Assessment, Aeronautical English, Listening Comprehension, Testing Listening, ICAOAbstract
The International Civil Aviation Organization (ICAO) published the Language Proficiency Requirements for pilots and air traffic controllers (ATCOs) in 2003. Research has shown that there is still a lack of clarity regarding what needs to be assessed in terms of the English used by pilots and ATCOs in radiotelephony (DOUGLAS, 2014; EMERY, 2014; KIM; BILLINGTON, 2016; KIM; ELDER, 2015; MONTEIRO, 2019; READ; KNOCH, 2009). The purpose of the present study was to investigate the nature of the listening tasks performed by pilots as an essential step in test development and validation. This explanatory sequential mixed-methods study (CRESWELL, 2015), elicited questionnaire responses from 156 pilots (Phase 1) and subsequently interviewed six aeronautical English experts (Phase 2) to better understand the characteristics of the listening tasks that pilots undertake during radiotelephony communications. Quantitative and qualitative data were analysed, and findings were merged. They provide information that may usefully inform the development of the listening test construct and the test specifications.
Metrics
References
ADANK, P.; B. G. EVANS; J. STUART-SMITH; S. K. SCOTT. 2009. Comprehension of familiar and unfamiliar native accents under adverse listening conditions. Journal of Experimental Psychology: Human Perception and Performance, 35.2: 520-529. doi: 10.1037/a0013552
ALDERSON, J. C.; C. CLAPHAM.; D. WALL. 1995. Language Test Construction and Evaluation. Cambridge: Cambridge University Press.
BACHMAN, L.F. 2007. What is the construct? The dialectic of abilities and contexts in defining constructs in language assessment. In FOX, J. et al. (Eds.), Language Testing Reconsidered, pp. 41-71. Ottawa, ON: University of Ottawa Press.
BACHMAN, L.F.; A. S. PALMER. 1996. Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
________________________________. 2010. Language assessment in practice. Oxford: Oxford University Press.
BARSHI, I. 1997. Effects of linguistic properties and message length on misunderstandings in aviation communication. Doctoral thesis. University of Colorado, Boulder, CO, USA.
BRINDLEY, G. 1998. Assessing listening abilities. Annual Review of Applied Linguistics, 18.1: 171–191. doi: 10.1017/S0267190500003536
BUCK, G. 2001. Assessing Listening. Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511732959
CRESWELL, J. W. 2015. A Concise Introduction to Mixed Methods Research. Los Angeles, CA: Sage.
_______________. 2014. Research design: Qualitative, quantitative and mixed method approaches. Thousand Oaks, CA. Sage Publications.
CUSHING, S. 1994. Fatal words: Communication clashes and aircraft crashes. University of Chicago Press.
DOUGLAS, D. 2001. Language for specific purposes assessment criteria: Where do they come from? Language Testing, 18.2: 171-185. doi:10.1177/026553220101800204
____________. 2004. Assessing the language of international civil aviation: Issues of validity and impact. Proceedings from the International Professional Communication Conference, IEEE Professional Communication Society, p. 248-252. Minneapolis: IEEE.
____________. 2010. Understanding language testing. London: Hodder Education.
____________. 2014. Nobody seems to speak English here today: Enhancing assessment and training in aviation English. Iranian Journal of Language Teaching Research, 2.2: 1- 12.
EMERY, H. J. 2014. Developments in LSP testing 30 years on? The case of aviation English. Language Assessment Quarterly, 11.2: 198-215. doi: 10.1080/15434303.2014.894516
ESTIVAL, D.; B. R. C. MOLESWORTH. 2009. A Study of EL2 Pilots Radio Communication in the General Aviation Environment. Australian Review of Applied Linguistics, 32: 24.1-24.16.
_____________________________________. 2016. Native English speakers and EL2 pilots: an experimental study. In ESTIVAL, D.; C. FARRIS.; B. MOLESWORTH. Aviation English: A lingua franca for pilots and air traffic controllers. New York: Routledge.
GARCIA, A. C. M. 2015. What do ICAO language proficiency test developers and raters have to say about the ICAO language proficiency requirements 12 years after their publication? A qualitative study exploring experienced professionals’ opinions. Master`s dissertation. Lancaster University, Lancaster, UK.
FIELD, J. 2019. Rethinking the Second Language Listening Test – From Theory to Practice. Equinox.
FOX, J.; ARTEMEVA, N. 2017. From diagnosis toward academic support: Developing a disciplinary, esp-based writing task and rubric to identify the needs of entering undergraduate engineering students. ESP Today, 5.2: 148–171. https://doi.org/10.18485/esptoday.2017.5.2.2
HUGHES, A. 2003. Testing for language teachers (2nd ed.). Cambridge: Cambridge University Press.
INTERNATIONAL CIVIL AVIATION ORGANIZATION. 2010. Manual on the Implementation of ICAO Language Proficiency Requirements, 2nd Edition (Doc. 9835). Montreal, Canada: Author.
JACOBY, S.; T. MCNAMARA. 1999. Locating competence. English for Specific Purposes, 18.3: 213–241.
KIM, H.; R. BILLINGTON. 2016. Pronunciation and comprehension in English as a Lingua Franca Communication: Effect of L1 influence in international aviation communication. Applied Linguistics, first published online January 14, 2016. doi:10.1093/applin/amv075
KIM, H.; C. ELDER. 2009. Understanding Aviation English as a lingua franca: Perceptions of Korean aviation personnel. Australian Review of Applied Linguistics, 32.3: 23.1-23.17.
_________________. 2015. Interrogating the construct of aviation English: Feedback from test takers in Korea. Language Testing, 32.2: 129−149.
KNOCH, U. 2009. Collaborating with ESP stakeholders in rating scale validation: the case of the ICAO rating scale. Spaan Fellow Working Papers in Second or Foreign Language Assessment 7: 21-46.
LYNCH, T.; D. MENDELSHON. 2010. Listening. In N. Schmitt (Ed.), An introduction to applied linguistics (2nd ed.), p. 180-196. London: Hodder Arnold.
MESSICK, S. 1996. Validity and washback in language testing. ETS Research Report Series, i–18. doi:10.1002/j.2333-8504.1996.tb01695.x
MONTEIRO, A. L. T. 2019. Reconsidering the measurement of proficiency in pilot and air traffic controller radiotelephony communication: from construct definition to task design. Doctoral thesis. Carleton University, Ottawa, Canada.
O’MALLEY, J. M.; A. CHAMOT; L. KUPPER. 1989. Listening comprehension strategies in second language acquisition. Applied Linguistics, 10: 418–37.
PFEIFFER, A. 2009. Inter-rater reliability in an aviation speaking test. Master’s dissertation. Lancaster University, Lancaster, UK.
PRINZO, O.V. 2009. The ICAO English language proficiency rating scale applied to enroute voice communications of U.S. and foreign pilots. (DOT/FAA/AM‐09/10). Washington D.C., Federal Aviation Administration.
PRINZO, O.V.; T. W. BRITTON. 1993. ATC/Pilot Voice Communications: A Survey of the Literature. (DOT/FAA/AM-93/20). Washington, DC: Federal Aviation Administration.
PRINZO, O.V.; A. M. HENDRIX; R. HENDRIX. 2009. The outcome of ATC message length and complexity on En Route pilot readback performance. DOT/FAA/AM-09/2. Washington, DC: Federal Administration.
READ, J.; U. KNOCH. 2009. Clearing the air: Applied linguistic perspectives on aviation communication. Australian Review of Applied Linguistics, 32.3: 21.1–21.11. doi: 10.2104/aral0921.
RÉVÉSZ, A.; T. BRUNFAUT. 2013. Text Characteristics of Task Input and Difficulty in Second Language Listening Comprehension. Studies in Second Language Acquisition, 35: 31-65. doi:10.1017/S0272263112000678.
SALDAÑA, J. 2013. The coding manual for qualitative researchers qualitative data analysis: a methods sourcebook (2nd ed.). London: Sage Publications.
SCHMIDT-RINEHART, B. 1994. The effects of topic familiarity on second language listening comprehension. Modern Language Journal, 18: 179–89.
SHIMITZU, T.; K. MAKISHIMA; M. YOSHIDA; H. YAMAGISHI. 2002. Effect of background noise on perception of English speech for Japanese listeners. Auris Nasus Larynx, 29: 121–125.
SPOLSKY, B.; SIGURD, B.; SATO, M.; WALKER, E.; ARTERBURN, C. 1968. Preliminary studies in the development of techniques for testing overall second language proficiency. Language Learning, 18: 79–101. https://doi.org/10.1111/j.1467-1770.1968.tb00224.x
TAYLOR, L.; A. GERANPAYEH. 2011. Assessing listening for academic purposes: Defining and operationalising the test construct. Journal of English for Academic Purposes, 10.2: 89–101.
TIEWTRAKUL, T.; S. R. FLETCHER. 2010. The challenge of regional accents for aviation English language proficiency standards: A study of difficulties in understanding in air traffic control–pilot communications, Ergonomics, 53.2: 229-239. DOI: 10.1080/00140130903470033?
WAGNER, E. 2014. Assessing listening. In A. KUNNAN (Ed.), Companion to language assessment, 1: 47-63. Oxford, UK: Wiley-Blackwell.
WU, Q.; B. MOLESWORTH; D, ESTIVAL. 2019. An investigation into the factors that affect miscommunication between pilots and air traffic controllers in commercial aviation. The International Journal of Aerospace Psychology, p. 53-63.
WU, Y. 1998. What do tests of listening comprehension test? – A retrospection study of EFL test-takers performing a multiple-choice task. Language Testing, 15: 21-44.
Downloads
Published
Versions
- 2023-06-12 (2)
- 2020-10-26 (1)
How to Cite
Issue
Section
License
The authors grant the journal all copyrights relating to the published works. The concepts issued in signed articles are the absolute and exclusive responsibility of their authors.
Esta obra está licenciada com uma Licença 