Artificial intelligence versus journalistsThe quality of automated news and bias by authorship using a Turing test

  1. La-Rosa Barrolleta, Leonardo Alberto 1
  2. Sandoval-Martín, Teresa 2
  1. 1 Universidad Nebrija
    info

    Universidad Nebrija

    Madrid, España

    ROR https://ror.org/03tzyrt94

  2. 2 Universidad Carlos III de Madrid
    info

    Universidad Carlos III de Madrid

    Madrid, España

    ROR https://ror.org/03ths8210

Revista:
Anàlisi: Quaderns de comunicació i cultura

ISSN: 0211-2175,  2340-5236

Año de publicación: 2024

Título del ejemplar: Miscel·lani

Número: 70

Páginas: 15-36

Tipo: Artículo

DOI: 10.5565/REV/ANALISI.3681 DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Anàlisi: Quaderns de comunicació i cultura

Resumen

Con la integración de la inteligencia artificial (IA) en los medios cada día se publican miles de noticias automatizadas en español. Este estudio compara la calidad de las noticias escritas por periodistas profesionales (de Efe) con las realizadas con programas informáticos de generación de lenguaje natural (NLG) (de narrativa) mediante un test de Turing. Basado en dimensiones cruciales para la percepción de noticias (Sundar,1999) — credibilidad, legibilidad y pericia periodística—, se usa una metodología experimental comprobada a escala internacional para explorar un tema inédito en español: información de salud. El experimento varió intencionadamente las autorías reales y declaradas — periodistas humanos e IA— para detectar posibles sesgos en torno a la credibilidad de la autoría. Se usó un cuestionario autoadministrado adaptado para encuestas en línea (N = 222) y se minimizaron los desequilibrios de género para asegurar la paridad de la muestra (N = 128). El estudio revela que no hay diferencias significativas entre las noticias generadas por IA y las elaboradas por periodistas profesionales. Ambos tipos de noticias son consideradas igualmente creíbles, aunque se detectaron algunos sesgos en la evaluación de la autoría declarada: la IA como autor es percibida como más creíble que el humano, mientras se considera que el periodista humano escribe textos más vívidos. El estudio concluye que es factible producir noticias automatizadas en español sin comprometer su calidad. En el panorama mediático global, los sistemas automatizados que emplean GLN, aprendizaje automático y sofisticadas bases de datos avanzan con éxito hacia nuevos ámbitos, como la información de salud.

Referencias bibliográficas

  • ARGUEDAS, A.R. and SIMON, F.M. (2023). Automating Democracy: Generative AI, journalism, and the future of democracy. Oxford: Oxford Internet Institute. https://doi.org/10.5287/ora-e262xv7no
  • BECKETT C. and YASEEN, M. (2023). Generating Change. A global survey of what news organisations are doing with AI. London: LSE. Retrieved from https://www.journalismai.info/research/2023-generating-change
  • CALVO-RUBIO, L.-M. and UFARTE-RUIZ, M.-J. (2020). “Percepción de docentes universitarios, estudiantes, responsables de innovación y periodistas sobre el uso de inteligencia artificial en periodismo”. Profesional de la información, 29 (1). https://doi.org/10.3145/epi.2020.ene.09
  • CARLSON, M. (2015). “The Robotic Reporter”. Digital Journalism, 3 (3), 416-431. https://doi.org/10.1080/21670811.2014.976412
  • CHANG, Y., WANG, X., WANG, J., WU, Y., YANG, L., ZHU, K., CHEN, H., YI, X., WANG, C., WANG, Y., YE, W., ZHANG, Y., CHANG, Y., YU, P.S., YANG, Q. and XIE, X. (2023). “A Survey on Evaluation of Large Language Models”. ArXiv preprint arXiv:2307.03109v7, 1-26. https://doi.org/10.48550/arXiv.2307.03109
  • CLERWALL, C. (2014). “Enter the Robot Journalist. Users’ perceptions of automated content”. Journalism Practice, 8 (5), 519-531. https://doi.org/10.1080/17512786.2014.883116
  • EUROPEAN COMMISSION (2018). A definition of Artificial Intelligence. Brussels: European Commission. Retrieved from https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_definition_of_ai_18_december_1.pdf
  • DANZON-CHAMBAUD, S. (2021). “Tow Report. Covering COVID-19 with automated news”. Columbia Journalism Review. Retrieved from https://www.cjr.org/tow_center_reports/covering-covid-automated-news.php
  • DANZON-CHAMBAUD, S. (2023). Automated news in practice: Changing the journalistic doxa during COVID-19, at the BBC and across media organizations [Doctoral thesis, School of Communications Dublin City University]. Doras. Retrieved from https://doras.dcu.ie/28004/1/DEPOSIT_JANUARY_2023_Danzon-Chambaud_Automated_News_Practice_PhD_Thesis.pdf
  • DIAKOPOULOS, N. (2019). Automating the News. MA: Harvard University Press.
  • DOMINGOS, P. (2015). The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. St Ives: Allen Lane.
  • DÖRR, K.N. (2016). “Mapping the Field of Algorithmic Journalism”. Digital Journalism, 4 (6), 700-722. https://doi.org/10.1080/21670811.2015.1096748
  • EFRON, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. California: Standford University. https://doi.org/10.1137/1.9781611970319
  • FANTA, A. (2017). Putting Europe’s Robots on the Map: Automated journalism in news agencies. Oxford: Reuters Institute for the Study of Journalism.
  • FLANAGIN, A.J. and METZGER, M.J. (2000). “Perceptions of Internet Information Credibility”. Journalism & Mass Communication Quarterly, 77 (3), 515-540. https://doi.org/10.1177/107769900007700304
  • GRAEFE, A. and BOHLKEN, N. (2020). “Automated journalism: A meta-analysis of readers’ perceptions of human-written in comparison to automated news”. Media and Communication, 8 (3), 50-59. http://dx.doi.org/10.17645/mac.v8i3.3019
  • GRAEFE, A., HAIM, M., HAARMANN, B. and BROSIUS, H.-B. (2018). “Reader’s perception of computer-generated news: credibility, expertise, and readability”. Journalism, 19 (5), 595-610. https://doi.org/10.1177/1464884916641269
  • GUISADO-CLAVERO M., ASTIER-PEÑA, M.P., GÓMEZ-BRAVO, R. and ARES-BLANCO, S. (2022). “Open data for monitoring COVID-19 in Spain: descriptive study”. Enfermedades Infecciosas y Microbiología Clínica, 42 (2), 80-87. https://doi.org/10.1016/j.eimc.2022.10.002
  • HAIM, M. and GRAEFE, A. (2017). “Automated News: Better than expected?”. Digital Journalism, 5 (8), 1044-1059. https://doi.org/10.1080/21670811.2017.1345643
  • INSTITUTO CERVANTES (2022). El español: una lengua viva. Informe 2022. Retrieved from https://cvc.cervantes.es/lengua/espanol_lengua_viva/pdf/espanol_lengua_viva_2022.pdf
  • JIA, C. (2020). “Chinese automated journalism: A comparison between expectations and perceived quality”. International Journal of Communication, 14, 2611-2632. Retrieved from https://ijoc.org/index.php/ijoc/article/download/13334/3083
  • JIA, C. and JOHNSON, T. (2021). “Source Credibility Matters: Does Automated Journalism Inspire Selective Exposure?”. International Journal of Communication, 15, 3760-3781.
  • JUNG, J., SONG, H., KIM, Y., IM, H. and OH, S. (2017). “Intrusion of software robots into journalism: the public’s and journalists’ perceptions of news written by algorithms and human journalists”. Computers in Human Behavior, 71, 291-298. https://doi.org/10.1016/j.chb.2017.02.022
  • LATAR, N. (2014). “The robot journalist in the age of social physics: the end of human journalism?”. In: G. EINAV (Ed.). The New World of Transitioned Media. New York: Springer International Publishing, 65-80. https://doi.org/10.1007/978-3-319-09009-2_6
  • LATAR, N. and NORDFORS, D. (2009). “Digital Identities and Journalism Content: How Artificial Intelligence and Journalism May Co-Develop and Why Society Should Care”. Innovation Journalism, 6 (7).
  • LECOMPTE, C. (2015). “Automation in the Newsroom”. Nieman Reports, 69 (3), 32-45. Retrieved from http://niemanreports.org/wp-content/uploads/2015/08/NRsummer2015.pdf
  • LEE, S.M. and KIM, T.Y. (1998). “A news on demand service system based on robot agent”. In: International Conference on Parallel and Distributed Systems Proceedings, 528-532. https://doi.org/10.1109/ICPADS.1998.741128
  • LERMANN-HENESTROSA, L., GREVING, H., KIMMERLE, J. (2023). “Automated journalism: The effects of AI authorship and evaluative information on the perception of a science journalism article”. Computers in Human Behavior, 138. https://doi.org/10.1016/j.chb.2022.107445
  • LINDEN, C. and TUULONEN, H. (2019). News Automation. The rewards, risks and realities of ‘machine journalism’. WAN-IFRA. Retrieved from https://wan-ifra.org/insight/report-news-automation-the-rewards-risks-and-realities-of-machine-journalism/
  • MATSUMOTO, R., NAKAYAMA, H., HARADA, T. and KUNIYOSHI, Y. (2007). “Journalist Robot: Robot System Making News Articles from Real World”. In: 2007 IEEE Intl conf on robotics and automation, 1234-1241. https://doi.org/10.1109/IROS.2007.4399598
  • MEEHAN, J.R. (1977). “Tale-Spin, an Interactive Programme That Writes Stories”. In: Proceedings of the Fifth International Joint Conference on Artificial Intelligence, 91-98. Retrieved from https://www.ijcai.org/Proceedings/77-1/Papers/013.pdf
  • MEYER, P. (1973). Precision journalism: A reporter’s introduction to social science methods. Bloomington: Indiana University Press.
  • MEYER, P. (1988). “Defining and measuring credibility of newspapers: Developing an index”. Journalism & Mass Communication Quarterly, 65 (3), 567-574. https://doi.org/10.1177/107769908806500301
  • MORAN, R.E. and SHAIKH, S.J. (2022). “Robots in the news and newsrooms: Unpacking meta-journalistic discourse on the use of artificial intelligence in journalism”. Digital Journalism, 10 (10), 1756-1774. https://doi.org/10.1080/21670811.2022.2085129
  • MORAVEC, V., MACKOVÁ, V., SIDO, J. and EKŠTEIN, K. (2020). “The Robotic Reporter in The Czech News Agency: Automated Journalism and Augmentation in the Newsroom”. Communication Today, 11 (1), 37-52. Retrieved from https://communicationtoday.sk/wp-content/uploads/03_MORAVEC-et-al_CT-1-2020.pdf
  • MURCIA VERDÚ, F. J., RAMOS ANTÓN, R. and CALVO RUBIO, L. M. (2022). “Comparative analysis of the sports chronicle quality written by artificial intelligence and journalists”. Revista Latina de Comunicación Social, 80, 91-111. https://doi.org/10.4185/RLCS-2022-1553
  • NAPOLI, P.M. (2014). “Automated Media: An Institutional Theory Perspective on Algorithmic Media Production and Consumption”. Communication Theory, 24 (3), 340-360. https://doi.org/10.1111/comt.12039
  • PARRA, S. and TORRENS, M. (2018). La inteligencia artificial: el camino hacia la ultrainteligencia. Barcelona: National Geographic.
  • RUTKIN, A. (2014). “Rise of Robot Reporters: When Software Writes the News”. New Scientist, 221. Retrieved from https://www.newscientist.com/article/dn25273-rise-of-robot-reporters-when-software-writes-the-news/
  • SANDOVAL-MARTÍN, T. and LA-ROSA, L. (2018). “Big Data as a differentiating sociocultural element of data journalism: the perception of data journalists and experts”. Communication & Society, 31 (4), 193-209. https://doi.org/10.15581/003.31.4.193-209
  • SANDOVAL-MARTÍN, T. and LA-ROSA, L. (2023). “Investigación sobre la calidad de las noticias automatizadas en la producción científica internacional: metodologías y resultados”. Cuadernos.info, 55, 114-136. https://doi.org/10.7764/cdi.55.54705
  • SCHWAB, K. (2016). La Cuarta Revolución Industrial (The Fourth Industrial Revolution). Barcelona: Debate.
  • SEPI (2020). Informe Anual 2020. Sociedad Estatal de Participaciones Industriales. Retrieved from https://www.sepi.es/sites/default/files/2022-03/informe_anual_2020_grupo_sepi_0.pdf
  • SEPI (2021). Informe Anual 2021. Sociedad Estatal de Participaciones Industriales. Retrieved from https://www.sepi.es/sites/default/files/2022-12/informe_anual_2021_grupo_sepi.pdf
  • SUNDAR, S. (1999). “Exploring receivers’ criteria for perception of print and online news”. Journalism & Mass Communication Quarterly, 76 (2), 373-386. http://dx.doi.org/10.1177/107769909907600213
  • SUNDAR, S. (2008). “The MAIN model: A heuristic approach to understanding technology effects on credibility”. In: M.J. METZGER and A.J. FLANAGIN (Eds.). Digital media, youth, and credibility. Cambridge, MA: The MIT Press, 73-100. https://doi.org/10.1162/dmal.9780262562324.073
  • TANDOC Jr, E.C., YAO, L.J. and WU, S. (2020). “Man vs. Machine? The Impact of Algorithm Authorship on News Credibility”. Digital Journalism, 8 (4), 548-562. https://doi.org/10.1080/21670811.2020.1762102
  • TEJEDOR-CALVO, S., CERVI, L., PULIDO, C.M. and PÉREZ-TORNERO, J.M. (2021a). “Análisis de la integración de sistemas inteligentes de alertas y automatización de contenidos en cuatro cibermedios”. Estudios sobre el mensaje periodístico, 27 (3), 973-983. https://doi.org/10.5209/esmp.77003
  • TEJEDOR, S., CERVI, L., TUSA JUMBO, F. and PORTALÉS, M. (2021b). “La información de la pandemia de la Covid-19 en las portadas de los diarios. Estudio comparativo de Italia, Reino Unido, España, Francia, Portugal, Estados Unidos, Rusia y Alemania”. Revista Mexicana de Ciencias Políticas y Sociales, 66 (242), 251-291. https://doi.org/10.22201/fcpys.2448492xe.2021.242.77439
  • TÚÑEZ-LÓPEZ, M., TOURAL-BRAN, C. and VALDIVIEZO-ABAD, C. (2019). “Automatización, bots y algoritmos en la redacción de noticias. Impacto y calidad del periodismo artificial”. Revista Latina de Comunicación Social, 74, 1411-1433. https://doi.org/10.4185/RLCS-2019-1391
  • TÚÑEZ-LÓPEZ, J. M., FIEIRAS CEIDE, C. and VAZ-ÁLVAREZ, M. (2021). “Impact of Artificial Intelligence on Journalism: transformations in the company, products, contents and professional profile”. Communication & Society, 34 (1), 177-193. https://doi.org/10.15581/003.34.1.177-193
  • TURING, A. (1950). “Computing Machinery and Intelligence”. Mind, LIX (236), 433-460. https://doi.org/10.1093/mind/LIX.236.433
  • UFARTE-RUIZ, M.J. and MANFREDI SÁNCHEZ, J.L. (2019). “Algoritmos y bots aplicados al periodismo. El caso de Narrativa Inteligencia Artificial: estructura, producción y calidad informative”. Doxa Comunicación, 29, 213-233. https://doi.org/10.31921/doxacom.n29a11
  • VAN-DER-KAA, H. and KRAHMER, E. (2014). “Journalist versus news consumer: The perceived credibility of machine written news”. In: Proceedings of the Computation + Journalism conference, Columbia University, New York, 24–25, October. Retrieved from https://research.tilburguniversity.edu/en/publications/journalist-versus-news-consumer-the-perceived-credibility-of-mach
  • WADDELL, T.F. (2018). “A Robot Wrote This? How perceived machine authorship affects news credibility”. Digital Journalism, 6 (2), 236-255. https://doi.org/10.1080/21670811.2017.1384319
  • WADDELL, T.F. (2019). “Can an Algorithm Reduce the Perceived Bias of News? Testing the Effect of Machine Attribution on News Readers’ Evaluations of Bias, Anthropomorphism, and Credibility”. Journalism & Mass Communication Quarterly, 96 (1), 82-100. https://doi.org/10.1177/1077699018815891
  • WÖLKER, A. and POWELL, T.E. (2021). “Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism”. Journalism, 22 (2), 86-103. https://doi.org/10.1177/1464884918757072
  • ZHENG, Y., ZHONG, B. and YANG, F. (2018). “When algorithms meet journalism: The user perception to automated news in a cross-cultural context”. Computers in Human Behavior, 86, 266-275. https://doi.org/10.1016/j.chb.2018.04.046
  • WU, Y. (2019). “Is automated journalistic writing less biased? An experimental test of auto-written and human-written news stories”. Journalism Practice, 14 (8), 1008-1028. https://doi.org/10.1080/17512786.2019.1682940
Fuente de los datos: Dialnet