Overview of HOPE at IberLEF 2024Approaching Hope Speech Detection in Social Media from Two Perspectives, for Equality, Diversity and Inclusion and as Expectations

  1. García-Baena, Daniel
  2. Balouchzahi, Fazlourrahman
  3. Butt, Sabur
  4. García-Cumbreras, Miguel Ángel
  5. Tonja, Atnafu Lambebo
  6. García-Díaz, José Antonio
  7. Bozkurt, Selen
  8. Chakravarthi, Bharathi Raja
  9. Ceballos, Hector G.
  10. Valencia-García, Rafael
  11. Sidorov, Grigori
  12. Ureña-López, L. Alfonso
  13. Gelbukh, Alexander
  14. Jiménez-Zafra, Salud María
Revue:
Procesamiento del lenguaje natural

ISSN: 1135-5948

Année de publication: 2024

Número: 73

Pages: 407-419

Type: Article

DIALNET GOOGLE SCHOLAR lock_openAccès ouvert editor

D'autres publications dans: Procesamiento del lenguaje natural

Résumé

Este articulo presenta la segunda edición de la tarea compartida internacional sobre detección multilingüe del discurso de la esperanza, HOPE 2024, desarrollada como parte del taller IberLEF durante el congreso SEPLN 2024. Esta tarea estuvo compuesta por dos subtareas distintas: la primera de ellas se centró en la detección del discurso esperanzador en el ámbito de la Igualdad, Diversidad e Inclusión, mientras que la segunda se enfocó en las expectativas del discurso esperanzador atendiendo a dos niveles, binario y multiclase. Diecinueve equipos participaron en la competición y dieciséis enviaron sus notas de trabajo. En la primera subtarea, el equipo mejor clasificado logró una puntuación Macro F1 media de 71,61. En la segunda subtarea, los equipos líderes demostraron un rendimiento sólido con puntuaciones F1 superiores a 80,00 para la clasificación binaria y 78,00 para los entornos de clasificación multiclase.

Références bibliographiques

  • Ahani, Z., M. S. Tash, M. Tash, A. Gelbukh, and I. Gelbukh. 2024. Multiclass hope speech detection through transformer methods. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Armenta-Segura, J. and G. Sidorov. 2024. Ometeotl at hope2024@iberlef: Custom bert models for hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • ArunaDevi, S. and B. Bharathi. 2024. Machine learning based approach for hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Bade, G. Y., O. Koleniskova, J. L. Oropeza, G. Sidorov, and K. F. Bergene. 2024. Hope speech in social media texts using transformer. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEURWS.org.
  • Balouchzahi, F., G. Sidorov, and A. Gelbukh. 2023. Polyhope: Two-level hope speech detection from tweets. Expert Systems with Applications, 225:120078.
  • Bruininks, P. and B. F. Malle. 2005. Distinguishing hope from optimism and related affective states. Motivation and Emotion, 29:324–352.
  • Bui Hong, S., Q. Le Minh, and T. Dang Van. 2024. ABCD Team at HOPE 2024: Hope Detection with BERTology Models and Data Augmentation. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Burnap, P., G. Colombo, R. Amery, A. Hodorog, and J. Scourfield. 2017. Multi-class machine classification of suicide-related communication on twitter. Online Social Networks and Media, 2:32–44, 08.
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez. 2023. Spanish pre-trained bert model and evaluation data.
  • Chakravarthi, B. R. 2020. HopeEDI: A multilingual hope speech detection dataset for equality, diversity, and inclusion. In Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, pages 41–53, Barcelona, Spain (Online), December. Association for Computational Linguistics.
  • Chakravarthi, B. R., V. Muralidaran, R. Priyadharshini, S. Cn, J. Mc-Crae, M. A. García, S. M. Jiménez-Zafra, R. Valencia-García, P. Kumaresan, R. Ponnusamy, D. García-Baena, and J. García-Díaz. 2022. Overview of the shared task on hope speech detection for equality, diversity, and inclusion. In B. R. Chakravarthi, B. Bharathi, J. P. McCrae, M. Zarrouk, K. Bali, and P. Buitelaar, editors, Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 378–388, Dublin, Ireland, May. Association for Computational Linguistics.
  • Chau Pham Quoc, H. and T. Dang Van. 2024. Choosing the Right Language Model for the Right Task. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Chiruzzo, L., S. M. Jiménez-Zafra, and F. Rangel. 2024. Overview of IberLEF 2024: Natural Language Processing Challenges for Spanish and other Iberian Languages. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEURWS.org.
  • Divakaran, S., K. Girish, and H. L. Shashirekha. 2024. Hope on the horizon: Experiments with learning models for hope speech detection in spanish and english. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEURWS.org.
  • Eid, M. and R. Larsen. 2008. The Science of Subjective Well-Being. Guilford Publications.
  • Ibrahim, M. 2024. Parameter-free Spanish hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Jiménez-Zafra, S. M., M. ´A. García, D. García-Baena, J. García-Díaz, B. R. Chakravarthi, R. Valencia-García, and L. A. Ureña-López. 2023. Overview of HOPE at IberLEF 2023: Multilingual Hope Speech Detection. Procesamiento del Lenguaje Natural, 71(0):371–381.
  • Kitzie, V. 2018. ”i pretended to be a boy on the internet”: Navigating affordances and constraints of social networking sites and search engines for lgbtq+ identity work. First Monday, 23, 07.
  • Krasitskii, M., O. Kolesnikova, L. C. Hernandez, G. Sidorov, and A. Gelbukh. 2023. Hope2023@iberlef: A cross-linguistic exploration of hope speech detection in social media. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2023), co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), CEURWS.org.
  • Kumaresan, P. K., B. R. Chakravarthi, S. Cn, M. ´A. García-Cumbreras, S. M. Jiménez Zafra, J. A. García-Díaz, R. Valencia-García, M. Hardalov, I. Koychev, P. Nakov, D. García-Baena, and K. K. Ponnusamy. 2023. Overview of the shared task on hope speech detection for equality, diversity, and inclusion. In B. R. Chakravarthi, B. Bharathi, J. Griffith, K. Bali, and P. Buitelaar, editors, Proceedings of the Third Workshop on Language Technology for Equality, Diversity and Inclusion, pages 47–53, Varna, Bulgaria, September. INCOMA Ltd., Shoumen, Bulgaria.
  • Lemlem, E., Y. Tsadkan, N. Amna, S. Grigori, and B. Ildar. 2024. Enhancing hope speech detection on twitter using machine learning and transformer models. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Milne, D., G. Pink, B. Hachey, and R. Calvo. 2016. Clpsych 2016 shared task: Triaging content in online peer-support forums. pages 118–127, 01.
  • Naseeb, A., K. L. Eyob, G. Sidorov, and O. Kolesnikova. 2024. Hope@iberlef 2024: Beyond binary bounds—classifying hope in online discourse. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Ngo, A. and H. T. H. Tran. 2023. Zootopi at hope2023iberlef: Is zero-shot chatgpt the future of hope speech detection? In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2023), co-located with the 39th Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), CEUR-WS.org.
  • Nguyen Thi, T. and T. Dang Van. 2024. An Empirical Study of Prompt Engineering with Large Language Models for Hope Detection in English and Spanish. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Palakodety, S., A. R. KhudaBukhsh, and J. G. Carbonell. 2019. Hope speech detection: A computational analysis of the voice of peace. arXiv preprint arXiv:1909.12940.
  • Pan, R., Ángela Almela, and G. Alcaraz-Mármol. 2024. UMUTeam at hope@iberlef 2024: Fine-tuning approach with sentiment and emotion features for hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Pérez, J. M., D. A. Furman, L. Alonso Alemany, and F. M. Luque. 2022. RoBERTuito: a pre-trained language model for social media text in Spanish. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7235–7243, Marseille, France, June. European Language Resources Association.
  • Pérez, J. M., M. Rajngewerc, J. C. Giudici, D. A. Furman, F. Luque, L. A. Alemany, and M. Vanina Mart´ınez. 2023. pysentimiento: A python toolkit for opinion mining and social nlp tasks.
  • Ponnusamy, K. K., M. Vegupatti, P. K. Kumaresan, R. Priyadharshini, P. Buitelaar, and B. R. Chakravarthi. 2024. VEL@iberlef 2024: Hope speech detection in spanish social media comments using bert pre-trained model. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Snyder, C., B. Hoza, W. Pelham, M. Rapoff, L. Ware, M. Danovsky, L. Highberger, H. Ribinstein, and K. Stahl. 1997. The development and validation of the children’s hope scale. Journal of Pediatric Psychology - J PEDIAT PSYCHOL, 22:399–421, 06.
  • Snyder, C. 2002. Hope theory: Rainbows in the mind. Psychological Inquiry, 13:249–275, 10.
  • Snyder, C. 1994. The Psychology of Hope: You Can Get There from Here. Free Press.
  • Subburaj, A., A. Kathiresan, R. Ponnusamy, P. Butilaar, and B. R. Chakravarthi. 2024. UniOfGalway@iberlef 2024: Hope speech recognition in spanish: A comparative analysis of transformer-based models. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
  • Ullah, F., M. T. Zamir, M. Ahmad, G. Sidorov, and A. Gelbukh. 2024. Hope: A multilingual approach to identifying positive communication in social media. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located with the 40th Conference of the Spanish Society for Natural Language Processing (SEPLN 2024), CEUR-WS.org.
La source de données: Dialnet