A multimodal approach to polysemy: the senses of touch

  1. Bolumar Martínez, Irene 1
  2. Alcaraz Carrión, Daniel
  3. Valenzuela Manzanares, Javier
  1. 1 Universidad de Murcia
    info
    Universidad de Murcia

    Murcia, España

    ROR https://ror.org/03p3aeb86

    Geographic location of the organization Universidad de Murcia
Journal:
Language and Cognition

ISSN: 1866-9808,  1866-9859

Year of publication: 2024

Volume: 16

Issue: 4

Pages: 1697-1717

Type: Article

DOI: 10.1017/LANGCOG.2024.23 GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Language and Cognition

Sustainable development goals

Abstract

This study investigated whether speakers use multimodal information (speech and gesture) to differentiate the physical and emotional meanings of the polysemous verb touch. We analyzed 302 hand gestures that co-occurred with this perception verb. For each case, we annotated(1)themeaningoftouch(physicalvs.emotional),(2)thegesturereferentspeakers physically touched (other-touch vs. self-touch), (3) the personal pronoun following the verb and (4) if they used intensifiers and negation. There were three main findings. First, we have seen that when speakers express the physical meaning, they are likely to reach an external referent (other-touch), but when they imply the emotional meaning, they tend to touch their ownbody(self-touch). Second, the most frequent co-speech gesture (chest-touching gesture) was associated with the emotional meaning, uncovering the metaphor . Third, this study showed that the physical meaning of touch usually coexists with a wide variety of personal pronouns and negation words; in contrast, the emotional meaning of touch occurs primarily with the pronoun me and it is usually modified by intensifiers. Thus, speakers use both speech and gesture to differentiate the meanings of the polysemous verb touch.

Bibliographic References

  • 10.1515/9783110197761.1.51
  • 10.1075/celcr.20.c18
  • 10.1515/9783110199109.2.27
  • 10.2752/174589311X12893982233551
  • 10.1017/CBO9780511554346
  • 10.1007/s10936-009-9127-1
  • 10.1515/sem-2019-0120
  • 10.1075/hcp.15.16iba
  • 10.1075/celcr.19.16kov
  • McNeill, (2013), Body - Language - Communication: An international handbook on multimodality in human interaction, 1, pp. 28
  • 10.1162/jocn.2007.19.7.1175
  • Monachini, (1994), Euralex 1994 proceedings, pp. 117
  • 10.1017/CBO9780511620904
  • 10.1177/0956797614534695
  • 10.1037/0000269-002
  • 10.1093/acprof:oso/9780199592746.001.0001
  • McNeill, (1992), Hand and mind: What gestures reveal about thought
  • Ibarretxe-Antuñano, (2002), Miscelánea: A Journal of English and American Studies, 25, pp. 93
  • 10.1515/semi.1969.1.1.49
  • Goldin-Meadow, (2003), Hearing gesture: How our hands help us think
  • Klepousniotou, (2007), The cognitive basis of polysemy: New sources of evidence for theories of word meaning, pp. 17
  • Maricchiolo, (2014), Body - Language - Communication: An international handbook on multimodality in human interaction, 2, pp. 1461
  • Fernández Jaén, J. (2012). Semántica cognitiva diacrónica de los verbos de percepción física del español.
  • 10.1075/celcr.19.03iba
  • 10.1515/9783110476972
  • 10.1515/9783110895698.3
  • 10.1016/j.lingua.2015.02.002
  • 10.1207/s15516709cog0000_62
  • 10.1075/gest.3.2.02hol
  • Lakoff, (1980), Metaphors we live by
  • 10.1007/s10936-021-09827-7
  • Uhrig, (2018), Anglistentag 2017 in Regensburg: Proceedings, pp. 99
  • Ibarretxe-Antuñano, I. (1999). Polysemy and metaphors in perception verbs: A cross-linguistic study.
  • 10.7208/chicago/9780226471013.001.0001
  • 10.1002/asi.22859
  • 10.1016/j.bandl.2011.12.016
  • 10.2752/174589311X12893982233597
  • 10.1075/cilt.20
  • 10.15388/RESPECTUS.2020.38.43.54
  • 10.1515/9783110199109.1.3
  • Lakoff, (1999), Philosophy in the flesh: The embodied mind and its challenge to Western thought
  • 10.3389/fpsyg.2020.587129
  • 10.2307/417135
  • Ibarretxe-Antuñano, (2021), Lenguaje y cognición
  • 10.1515/ling-2020-0039
  • 10.1146/annurev-psych-113011-143802
  • Quirk, (1972), ELT Journal, 29, pp. 83
  • Fernández Jaén, (2016), Revista de Investigación Lingüística, 19, pp. 199
  • 10.1075/pb.vii.8
  • 10.1515/9783110213348
  • 10.1075/ijcl.17.3.04har
  • 10.1177/001316446002000104
  • 10.1515/cog-2014-0055
  • Alcaraz Carrión, D., & Valenzuela, J. (2021). Distant time, distant gesture: Speech and gesture correlate to express temporal distance. Semiotica, 2021(241), 159–183. https://doi.org/10.1515/sem-2019-0120
  • Argaman, O. (2009). Linguistic markers and emotional intensity. Journal of Psycholinguistic Research, 39(2), 89–99. https://doi.org/10.1007/s10936-009-9127-1
  • Burenhult, N., & Majid, A. (2011). Olfaction in Aslian ideology and language. The Senses and Society, 6(1), 19–29. https://doi.org/10.2752/174589311X12893982233597
  • Carrillo-de-Albornoz, J., & Plaza, L. (2013). An emotion-based model of negation, intensifiers, and modality for polarity and intensity classification. Journal of the American Society for Information Science and Technology, 64(8), 1618–1633. https://doi.org/10.1002/asi.22859
  • Cohen,J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104
  • Cooperrider, K., & Mesh, K. (2022). Pointing in gesture and sign. In A. Morgenstern, & S. Goldin-Meadow (Eds.), Gesture in language: Development across the lifespan (pp. 21–46). De Gruyter Mouton. https://doi. org/10.1515/9783110567526-002
  • de la Fuente, J., Santiago, J., Román, A., Dumitrache, C., & Casasanto, D. (2014). When you think about it, your past is in front of you: How culture shapes spatial conceptions of time. Psychological Science, 25(9), 1682–1690. https://doi.org/10.1177/0956797614534695
  • Ekman, P., & Friesen, W. V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1(1), 49–98. https://doi.org/10.1515/semi.1969.1.1.49
  • Evans, N., & Wilkins, D.(2000). Inthe mind’sear: Thesemantic extensions of perception verbs in Australian languages. Language, 76(3), 546–592. https://doi.org/10.2307/417135
  • Falkum, I. L., & Vicente, A. (2015). Polysemy: Current perspectives and approaches. Lingua, 157, 1–16. https://doi.org/10.1016/j.lingua.2015.02.002
  • Fernández Jaén, J. (2012). Semántica cognitiva diacrónica de los verbos de percepción física del español.
  • Fernández Jaén, J. (2016). Unos modales y epistémicos del verbo sentir. Revista de Investigación Lingüística, 19, 199–226.
  • Fontaine, J. J. R., Scherer, K. R., & Soriano, C. (2013). Components of emotional meaning: A sourcebook. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199592746.001.0001
  • Gaby, A. (2008). Gut feelings: Locating intellect, emotion and life force in the thaayorre body. In F. Sharifian, R. Dirven, N. Yu, & S.Niemeier (Eds.), Culture, body, and language: Conceptualizations of internal body organs across cultures and languages (pp. 27–44). De Gruyter Mouton. https://doi.org/10.1515/9783110199109.2.27
  • Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. The Belknap Press of Harvard University Press.
  • Goldin-Meadow, S., & Alibali, M. W. (2013). Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 64(1), 257–283. https://doi.org/10.1146/annurev-psych-113011-143802
  • Hardie, A. (2012). CQPweb– Combining power, flexibility and usability in a corpus analysis tool. International Journal of Corpus Linguistics, 17(3), 380–409. https://doi.org/10.1075/ijcl.17.3.04har
  • Hinnell, J., & Parrill, F. (2020). Gesture influences resolution of ambiguous statements of neutral and moral preferences. Frontiers in Psychology, 11, 1–10. https://doi.org/10.3389/fpsyg.2020.587129
  • Hirst, G. (1987). Semantic interpretation and the resolution of ambiguity. Cambridge University Press. https://doi.org/10.1017/CBO9780511554346
  • Holle, H., &Gunter,T.C.(2007).Theroleoficonicgesturesinspeechdisambiguation:ERPevidence.Journal of Cognitive Neuroscience, 19(7), 1175–1192. https://doi.org/10.1162/jocn.2007.19.7.1175
  • Holler, J., & Beattie, G. (2003). Pragmatic aspects of representational gestures: Dospeakers use them toclarify verbal ambiguity for the listener? Gesture, 3(2), 127–154. https://doi.org/10.1075/gest.3.2.02hol
  • Ibarretxe-Antuñano, I. (1999). Polysemy and metaphors in perception verbs: A cross-linguistic study.
  • Ibarretxe-Antuñano, I. (2002). -- as a cross-linguistic conceptual metaphor. Miscelánea: A Journal of English and American Studies, 25, 93–119.
  • Ibarretxe-Antuñano, I. (2006). Cross-linguistic polysemy in tactile verbs. In J. Luchjenbroers (Ed.), Cognitive linguistics investigations: Across languages, fields and philosophical boundaries (pp. 235–253). John Benjamins Publishing Company. https://doi.org/10.1075/hcp.15.16iba -
  • Ibarretxe-Antuñano, I. (2019). Perception metaphors in cognitive linguistics: Scope, motivation, and lexicalisation. In L. J. Speed, C. O’Meara, L. S. Roque, & A. Majid (Eds.), Perception metaphors (pp. 43–64). John Benjamins Publishing Company. https://doi.org/10.1075/celcr.19.03iba
  • Ibarretxe-Antuñano, I., & Valenzuela, J. (2021). Lenguaje y cognición. Editorial Síntesis.
  • Jansegers, M. (2017). Hacia un enfoque múltiple de la polisemia: Un estudio empírico del verbo multimodal «sentir» desde una perspectiva sincrónica y diacrónica. De Gruyter. https://doi.org/10.1515/9783110476972
  • Jansegers, M., Vanderschueren, C., & Enghels, R. (2015). The polysemy of the Spanish verb sentir:A behavioral profile analysis. Cognitive Linguistics, 26(3), 381–421. https://doi.org/10.1515/cog-2014-0055
  • Klepousniotou, E. (2007). Reconciling linguistics and psycholinguistics: On the psychological reality of linguistic polysemy. In M. Rakova, G. Pethö, & C. Rákosi (Eds.), The cognitive basis of polysemy: New sources of evidence for theories of word meaning (pp. 17–46). Peter Lang.
  • Kövecses, Z. (2019). Perception and metaphor: Thecase ofsmell. In L.J. Speed, C.O’Meara, L.S. Roque, &A. Majid (Eds.), Perception metaphors (pp. 327–346). John Benjamins Publishing Company. https://doi. org/10.1075/celcr.19.16kov
  • Kövecses, Z. (1986). Metaphors of Anger, Pride and Love: A lexical approach to the structure of concepts. John Benjamins Publishing Company. https://doi.org/10.1075/pb.vii.8
  • Krishna, P. P., Arulmozi, S., &Mishra,R.K.(2022).Doyouseeandhearmore?AstudyonTeluguperception verbs. Journal of Psycholinguistic Research, 51(3), 473–484. https://doi.org/10.1007/s10936-021-09827-7
  • Lacey,S., Stilla, R., & Sathian, K. (2012).Metaphorically feeling: Comprehendingtexturalmetaphorsactivates somatosensory cortex. Brain and Language, 120(3), 416–421. https://doi.org/10.1016/j.bandl.2011.12.016
  • Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the mind. The University of Chicago Press.
  • Lakoff, G., & Johnson, M. (1980). Metaphors we live by. University of Chicago Press.
  • Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. Basic Books.
  • Majid, A., & Levinson, S. C. (2011). The senses in language and culture. The Senses and Society, 6(1), 5–18. https://doi.org/10.2752/174589311X12893982233551
  • Maricchiolo, F., de Dominicis, S., Ganucci Cancellieri, U., di Conza, A., Gnisci, A.,& Bonaiuto, M.(2014). Cospeech gestures: Structures and functions. In C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill, & J. Bressem(Eds.),Body-Language-Communication:Aninternationalhandbookonmultimodalityinhuman interaction (Vol. 2, pp. 1461–1473). De Gruyter Mouton. https://doi.org/10.1515/9783110302028.1461
  • McNeill, D. (1992). Hand and mind: What gestures reveal about thought. The University of Chicago Press.
  • McNeill, D.(2013). Gesture asawindowontomindandbrain,andtherelationshiptolinguisticrelativity and ontogenesis. In C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill, & S. Tessendorf (Eds.), BodyLanguage- Communication: An international handbook on multimodality in human interaction (Vol. 1, pp. 28–54). De Gruyter Mouton. https://doi.org/10.1515/9783110261318.28
  • Monachini, M., & Roventini, A. (1994). Italian audition verbs: A corpus- and frame-based analysis. In W. Martin, W. Meijs, M. Moerland, E. ten Pas, P. van Sterkenburg, & P. Vossen (Eds.), Euralex 1994 proceedings (pp. 117–127). Vrije Universiteit.
  • Nerlich, B., & Clarke, D. (2003). Polysemy and flexibility: Introduction and overview. In B. Nerlich, Z. Todd, V. Herman, & D. Clarke (Eds.), Polysemy: Flexible patterns of meaning in mind and language (pp. 3–30). De Gruyter Mouton.
  • Norrick, N. R. (1981). Semiotic principles in semantic theory (Vol. 20). John Benjamins Publishing Company. https://doi.org/10.1075/cilt.20
  • Núñez, R., & Sweetser, E. (2006). With the future behind them: Convergent evidence from aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30(3), 401–450. https://doi.org/10.1207/s15516709cog0000_62
  • Quirk, R., Greenbaum, S., Leech, G., &Svartvik, J. (1972). A grammar ofcontemporary English. ELTJournal, 29(1), 83–88. https://doi.org/10.1093/elt/29.1.83
  • Selmistraitis, L., & Boikova, R. (2020). Source domains of smell related metaphorical collocations: Study based on corpus of contemporary American English. Respectus Philologicus, 38(43), 11–24. https://doi. org/10.15388/RESPECTUS.2020.38.43.54
  • Sharifian, F., Dirven, R., Yu, N., & Niemeier, S. (2008). Culture and language: Looking for the “mind” inside the body. In F. Sharifian, R. Dirven, N. Yu, & S. Niemeier (Eds.), Culture, body, and language: Conceptualizations of internal body organs across cultures and languages (pp. 3–26). De Gruyter Mouton. https:// doi.org/10.1515/9783110199109.1.3
  • Sweetser, E. E. (1990). From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge University Press.
  • Taylor, J. R. (2006). Polysemy and the lexicon. In G. Kristiansen, M. Achard, R. Dirven, & F. J. Ruiz de Mendoza Ibáñez (Eds.), Cognitive linguistics: Current applications and future perspectives (pp. 51–80). Mouton de Gruyter. https://doi.org/10.1515/9783110197761.1.51
  • Uhrig,P.(2018).NewsScapeandthedistributedlittleredHenLab–Adigitalinfrastructureforthelarge-scale analysis of TV broadcasts. In A. Zwierlein, J. Petzold, K. Böhm, & M. Decker (Eds.), Anglistentag 2017 in Regensburg: Proceedings (pp. 99–114). Wissenschaftlicher Verlag Trier.
  • van Putten, S. (2020). Perception verbs and the conceptualization of the senses: The case of Avatime. Linguistics, 58(2), 1–38. https://doi.org/10.1515/ling-2020-0039
  • Winter, B. (2019). Sensory linguistics: Language, perception and metaphor (Vol. 20). John Benjamins Publishing Company. https://doi.org/10.1075/celcr.20
  • Yu, N. (2009). The Chinese HEART in a cognitive perspective: Culture, body, and language. De Gruyter Mouton. https://doi.org/10.1515/9783110213348