Focus types in Brazilian Portuguese:
Multimodal production and perception
DOI:
https://doi.org/10.1590/1678-460x202258944Palabras clave:
focalization; audiovisual prosody; Brazilian Portuguese.Resumen
This paper aims at describing prosodic focalization as a multimodal phenomenon in Brazilian Portuguese, evaluating the role of two modalities in focus production and perception: audio (A), visual (V), and their combined audiovisual presentation (AV). Five focus types are considered, according to their semantic-pragmatic values: (a) in declarative sentences: (i) IF - informational focus (answer to a previous question, conveying new information), (ii) CF- contrastive (strong) focus (correction of information considered wrong); (iii) ATF - attenuated (weak) focus (proposition of an alternative solution to previous information); (b) in interrogative sentences: (i) INTF - interrogative focus (a new information is requested in the question); (ii) SF - surprise focus (one casts doubt on a previous information). Also, structural factors were evaluated, as focus extension and position in the sentence. After running a multimodal perceptual experiment and developing an acoustic and visual analysis on focus production, results show that multimodality plays a relevant role in focus production and perception. Different acoustic and visual parameters, or configuration of parameters, contribute to conveying distinct meanings, according to each focus type.
Citas
Alexanderson, S., House, D., & Beskow, J. (2013). Aspects of cooccurring syllables and head nods in spontaneous dialogue. Proceedings of 12th International Conference on Auditory-Visual Speech Processing (AVSP2013). https://www.isca-speech.org/archive_v0/avsp13/papers/ av13_169.pdf.
Baltrušaitis, T., Mahmoud, M., & Robinson, P. (2015). Cross-dataset learning and person-specific normalisation for automatic action unit detection. Proceedings of 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG 2015). https://doi.org/10.1109/FG.2015.7284869.
Baltrušaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018). Openface 2.0: Facial behavior analysis toolkit. Proceedings of 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (pp. 59-66). https://doi.org/10.1109/FG.2018.00019.
Bergmann, K., Kahl, S., & Kopp, S. (2014). How is information distributed across speech and gesture? A cognitive modeling approach. Cognitive Processing, 15(1: Special Issue: Proceedings of KogWis 2014), S84-S87. https://pub.uni-bielefeld.de/record/2700040.
Boersma, P., & Weenink, D. (2020). Praat: Doing phonetics by computer (Version 6.1.16) [Computer software]. http://www.praat.org/.
Bolinger, D. L. (1954). English prosodic stress and Spanish sentence order. Hispania, 37(2), 152-156. https://doi.org/10.2307/335628.
Borràs-Comes, J., & Prieto, P. (2011). ‘Seeing tunes.’ The role of visual gestures in tune interpretation. Laboratory Phonology, 2(2), 355-380. https://doi.org/10.1515/labphon.2011.013.
Carnaval, M., Moraes, J. A., & Rilliard, A. (2019). Marcação de foco estreito e o acento secundário em interrogativas totais no português do Brasil. Working Papers em Linguística, 19(2), 136-167. https:// doi.org/10.5007/1984-8420.2018v19n2p136.
Chomsky, N. (1971). Deep structure, surface structure and semantics interpretation. In D. D. Steinberg, & L. A. Jakobovits (Eds.), Semantics: An interdisciplinary reader in philosophy, linguistics and psychology (pp. 183–216). Cambridge University Press.
Cochran, W. G., & Cox, G. M. (1992). Experimental designs. 2nd ed., Wiley classics library ed. Wiley.
Crawley, M. J. (2013). The R book. 2nd ed. Wiley.
Crespo-Sendra, V. C., Kaland, C., Swerts, M., & Prieto, P. (2013). Perceiving
incredulity: The role of intonation and facial gestures. Journal of Pragmatics, 47(1). https://doi.org/10.1016/j.pragma.2012.08.008.
Cruz, M., Swerts, M., & Frota, S. (2015). Variation in tone and gesture within language. In The Scottish Consortium for ICPhS 2015 (Ed.), Proceedings of the 18th International Congress of Phonetic Sciences. http://repositorio.ul.pt/bitstream/10451/25020/1/ICPHS0452.pdf.
De Ruiter, J. P. (1998). Gesture and speech production [Doctoral dissertation]. Katholieke Universiteit.
Dik, S. (1980). On the typology of focus phenomena. In T. Hoekstra, H. van der Hulst, & M. Moortgat (Eds.), Perspectives on Functional Grammar (pp. 41–74). De Gruyter. https://doi.org/10.1515/9783112329603-005.
Dohen, M., & Lœvenbruck, H. (2009). Interaction of audition and vision for the perception of prosodic contrastive focus. Language and Speech, 52(2–3), 177–206. https://doi.org/10.1177/0023830909103166.
Ekman, P., Friesen, W. V., & Hager, J. C. (2002). Facial action coding system: The manual. Research Nexus.
Elordieta, G., & Irurtzun, A. (2009). The prosody and interpretation of non-exhaustive narrow focus in Basque. Anuario Del Seminario de Filología Vasca Julio de Urquijo, 43(1–2), 205-230. https://ojs.ehu. eus/index.php/ASJU/article/view/1692.
Esteve-Gibert, N., Borras-Comes, J., Asor, E., Swerts, M., & Prieto, P. (2017). The timing of head movements: The role of prosodic heads and edges. The Journal of the Acoustical Society of America 141, 4727-4739. https://doi.org/10.1121/1.4986649.
Esteve-Gibert, N., & Prieto, P. (2013). Prosodic structure shapes the temporal realization of intonation and manual gesture movements. J Speech Lang Hear Res, 56(3), 850-864. https://doi.org/10.1044/1092- 4388(2012/12-0049).
Fernandes, F. R. (2007) Ordem, focalização e preenchimento em português: Sintaxe e prosódia [Doctoral dissertation]. State University of Campinas. https://doi.org/10.47749/T/UNICAMP.2007.398459.
Frota, S., & Vigário, M. (2000). Aspectos de prosódia comparada: Ritmo e entoação no PE e no PB. In R. V. Castro, & P. Barbosa (Eds.), Actas do XV Encontro Nacional da Associação Portuguesa de Lingüística (pp. 533-555). Associação Portuguesa de Linguística (APL). https:// apl.pt/wp-content/uploads/2017/12/1999-35.pdf.
Gries, S. T. (2013). Statistics for linguistics with R: A practical introduction. 2nd revised ed. De Gruyter Mouton.
Gussenhoven, C. (2006). Types of focus in English. In C. Lee, M. K. Gordon, & D. Büring (Eds.), Topic and focus: Cross-linguistic perspectives on meaning and intonation (pp. 83-100). Springer.
Halliday, M. A. K. (1967). Notes on transitivity and theme in English: Part 2. Journal of Linguistics, 3(2), 199-244. https://doi.org/10.1017/ S0022226700016613.
Jackendoff, R. (1972). Semantic interpretation in generative grammar. 6th print. The MIT Press Classics.
Kelly, S. D., Özyürek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science 21(2), 260-267. https://doi. org/10.1177/0956797609357327.
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Kay (Ed.), The Role of Nonverbal Communication (pp. 207-227). De Gruyter Mouton.
Krahmer, E., & Swerts, M. G. J. (2004). More about brows: A cross- linguistic analysis-by-synthesis study. In Z. Ruttkay, & C. Pelachaud (Eds.), From brows to trust: Evaluating Embodied Conversational Agents, Human-Computer Interaction Series, (pp. 191-216). Kluwer Academic Publishers.
Krahmer, E., & Swerts, M. G. J. (2009). Audiovisual prosody: Introduction to the special issue. Language and Speech, 52(2-3), 129-133. https://doi.org/10.1177/0023830909103164.
Krifka, M. (2008). Basic notions of information structure. Acta Linguistica Hungarica, 55(3-4), 243-276. https://doi.org/10.1556/ ALing.55.2008.3-4.2.
Lambrecht, K. (1994). Information structure and sentence form: Topic, focus, and the mental representations of discourse referents. Cambridge University Press. https://doi.org/10.1017/CBO9780511620607.
Loehr, D. (2012). Temporal, structural, and pragmatic synchrony between intonation and gesture. Journal Laboratory Phonology, 3, 71-89. https://doi.org/10.1515/lp-2012-0006.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press. https://press.uchicago.edu/ucp/books/ book/chicago/H/bo3641188.html.
Moraes, J. A. (2006). Variações em torno de tema e rema. Cadernos do IX Congresso Nacional de Lingüística e Filologia, Universidade do Estado do Rio de Janeiro, 279-289.
Moraes, J. A., Carnaval, M., & Coelho, A. B. B. (2015). A manifestação prosódica do foco em interrogativas totais no Português do Brasil e sua percepção. ReVEL, 10(spe), 170-194. http://revel.inf.br/files/25 628f323ed484f9952532a1604fbb93.pdf.
Miranda, L., Swerts, M., Moraes, J., & Rilliard, A. (2021). The role of the auditory and visual modalities in the perceptual identification of Brazilian Portuguese statements and echo questions. Language and Speech, 64(1), 3-23. https://doi.org/10.1177/0023830919898886.
Nespor, M., & Vogel, I. (1986). Prosodic phonology. Foris Publications. Özyürek, A., Willems, R. M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience,
(4), 605-616. https://doi.org/10.1162/jocn.2007.19.4.605.
Pouw, W., & J. A. Dixon. (2019). Quantifying gesture–speech synchrony. Proceedings of the 6th Meeting of Gesture and Speech in Interaction
(pp. 68-74). Germany.
Pouw, W., De Jonge-Hoekstra, L., Harrison, S. J., Paxton, A., & Dixon,
J. A. (2021). Gesture– speech physics in fluent speech and rhythmic upper limb movements. Annals of the New York Academy of Sciences, 1491(1), 89-105. https://doi.org/10.1111/nyas.14532.
Prieto, P., Pugliesi, C., Borràs-Comes, J., Arroyo, E., & Blat, J. (2011). Crossmodal prosodic and gestural contribution to the perception of contrastive focus. Interspeech 2011, 977-980. https://doi. org/10.21437/Interspeech.2011-397.
Prieto, P., Puglesi, C., Borras-Comes, J., Arroyo, E., & Blat, J. (2015). Exploring the contribution of prosody and gesture to the perception of focus using an animated agent. Journal of Phonetics, 49, 41-54. https://doi.org/10.1016/j.wocn.2014.10.005.
R Core Team. (2021). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.R- project.org/.
Selkirk, E. (1986). On derived domains in sentence phonology. Phonology Yearbook, 3, 371-405. https://doi.org/10.1017/S0952675700000695. Shattuck-Hufnagel, S., Ren, A., & Tauscher, E. (2010). Are torso movements during speech timed with intonational phrases? Proceedings of the Speech Prosody 2010 (paper 974). https://www. isca-speech.org/archive/speechprosody_2010/shattuckhufnagel10_
speechprosody.html.
Shattuck-Hufnagel, S., & Ren, A. (2018). The prosodic characteristics of
non-referential co-speech gestures in a sample of academic-lecture- style speech. Frontiers in Psychology, 9. https://doi.org/10.3389/ fpsyg.2018.01514.
Tenani, L. (2002). Domínios prosódicos do português do Brasil: Implicações para a prosódia e para a aplicação de processos fonológicos [Doctoral dissertation]. State University of Campinas. https://doi.org/10.47749/T/UNICAMP.2002.253138.
Truckenbrodt, H., Sandalo, F., & Abaurre, B. (2009). Elements of Brazilian Portuguese intonation. Journal of Portuguese Linguistics, 8(1), 75- 114. https://doi.org/10.5334/jpl.122.
Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S. 4th ed. Springer.
Wagner, P., Malisz, Z., & Kopp, S. (2014). Gesture and speech in interaction: An overview. Speech Communication, 57, 209-232. https://doi.org/10.1016/j.specom.2013.09.008.
