{"id":11960,"date":"2026-10-28T15:51:12","date_gmt":"2026-10-28T14:51:12","guid":{"rendered":"https:\/\/leibniz-hbi.de\/?post_type=hbi-publications&p=11960"},"modified":"2026-10-29T16:20:18","modified_gmt":"2026-10-29T15:20:18","slug":"deliberation","status":"publish","type":"hbi-publications","link":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/","title":{"rendered":"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved?"},"content":{"rendered":"

In th\"Covereir publication, Patrick Frey, Prof. Dr. Cornelius Puschmann (ZeMKI, University of Bremen), Smita Bhattacharya<\/a>, and Dr. Gregor Wiedemann<\/a> (HBI) examine whether artificial intelligence can capture the deliberative quality of discussions and contribute to more constructive communication on social media, and if so, to what extent. The researchers from the ComAI<\/a> collaborative project evaluate deliberation on social media using large language models. Deliberation is a form of decision-making based on the exchange of arguments in which all participants are treated as equals.<\/p>\n

The publication “Can AI Gauge Deliberative Quality? Evaluating Deliberation on Social Media with Large Language Models\u201d is available as open access here<\/a>. It was published in the journal Media and Communication.<\/p>\n

Abstract<\/h2>\n

Although social media has become a central arena for deliberative discourse, debates are frequently characterized by incivility. Due to the volume of online discourse, it is challenging to assess when and where interventions, such as community-based counterspeech, are required. Accordingly, automated measurement of deliberative quality is a powerful tool for facilitating constructive online communication. Since deliberation is multidimensional, attempts to monitor online discourse must consider alternative forms of communication to foster inclusive deliberation. Whereas Type I deliberative dimensions, such as rationality and civility, are grounded in Habermasian ideals, Type II deliberation extends this perspective by incorporating alternative communicative forms, including emotionality and storytelling, into a more inclusive assessment of deliberative quality.<\/p>\n

Addressing the limitations of prior studies that primarily focused on classic Type I deliberative dimensions and tested only a small number of large language models, this study evaluates multiple open-weight models to conduct a nuanced assessment of deliberative quality. We applied the large language model that achieved the best overall balance between predictive performance, model size, and computational efficiency to a dataset of 64,223 comments from German-language climate change discourse on X.<\/p>\n

Our results demonstrate that large language models can reliably capture complex Type II deliberative dimensions despite class imbalances and subtle linguistic cues that generally hinder automated text classification. Class imbalance often impairs classification performance due to highly skewed label distributions, and Type II dimensions are particularly difficult to detect as they are often signaled by implicit, context-dependent markers.<\/p>\n

The empirical analysis also reveals distinct communicative patterns: While public speakers predominantly employ civil, rational, constructive, and common-good-oriented communication, other debating users exhibit higher levels of reciprocal engagement, humor, and negative emotions, alongside more frequent incivility.<\/p>\n

Our study demonstrates that AI-driven measurement can serve as a foundational tool for identifying norm violations, directing counterspeech toward specific debates, promoting inclusive digital public spheres, and observing broader long-term effects of moderation and interventions on the quality of debate.<\/p>\n

Frey, P., Bhattacharya, S., Wiedemann, G., & Puschmann, C. (2026). Can AI Gauge Deliberative Quality? Evaluating Deliberation on Social Media With Large Language Models. Media and Communication<\/em>, 14<\/em>, 12494. https:\/\/doi.org\/10.17645\/mac.12494<\/a><\/p>\n","protected":false},"featured_media":2579,"template":"","tags":[398],"class_list":["post-11960","hbi-publications","type-hbi-publications","status-publish","has-post-thumbnail","hentry","tag-miscellaneous"],"acf":[],"yoast_head":"\nCan the Quality of Online Discussions Be Automatically Assessed\u2014and Improved? - Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved?\" \/>\n<meta property=\"og:description\" content=\"In their publication, Patrick Frey, Prof. Dr. Cornelius Puschmann (ZeMKI, University of Bremen), Smita Bhattacharya, and Dr. Gregor Wiedemann (HBI) examine whether artificial intelligence can capture the deliberative quality of discussions and contribute to more constructive communication on social media, and if so, to what extent. The researchers from the ComAI collaborative project evaluate deliberation […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/\" \/>\n<meta property=\"og:site_name\" content=\"Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-29T15:20:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/07\/Header-Publikationen.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1380\" \/>\n\t<meta property=\"og:image:height\" content=\"400\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/\",\"url\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/\",\"name\":\"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved? - Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/leibniz-hbi.de\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/Header-Publikationen.jpg\",\"datePublished\":\"2026-10-28T14:51:12+00:00\",\"dateModified\":\"2026-10-29T15:20:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/#primaryimage\",\"url\":\"https:\\\/\\\/leibniz-hbi.de\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/Header-Publikationen.jpg\",\"contentUrl\":\"https:\\\/\\\/leibniz-hbi.de\\\/wp-content\\\/uploads\\\/2024\\\/07\\\/Header-Publikationen.jpg\",\"width\":1380,\"height\":400,\"caption\":\"flying book with open pages in front of a blue sky\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/hbi-publications\\\/deliberation\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Homepage\",\"item\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/\",\"name\":\"Leibniz HBI\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#organization\",\"name\":\"Leibniz HBI\",\"url\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/leibniz-hbi.de\\\/wp-content\\\/uploads\\\/2024\\\/01\\\/HBI_Logo-Kopie-2.png\",\"contentUrl\":\"https:\\\/\\\/leibniz-hbi.de\\\/wp-content\\\/uploads\\\/2024\\\/01\\\/HBI_Logo-Kopie-2.png\",\"width\":462,\"height\":123,\"caption\":\"Leibniz HBI\"},\"image\":{\"@id\":\"https:\\\/\\\/leibniz-hbi.de\\\/en\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved? - Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/","og_locale":"en_US","og_type":"article","og_title":"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved?","og_description":"In their publication, Patrick Frey, Prof. Dr. Cornelius Puschmann (ZeMKI, University of Bremen), Smita Bhattacharya, and Dr. Gregor Wiedemann (HBI) examine whether artificial intelligence can capture the deliberative quality of discussions and contribute to more constructive communication on social media, and if so, to what extent. The researchers from the ComAI collaborative project evaluate deliberation […]","og_url":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/","og_site_name":"Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut","article_modified_time":"2026-10-29T15:20:18+00:00","og_image":[{"width":1380,"height":400,"url":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/07\/Header-Publikationen.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/","url":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/","name":"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved? - Leibniz-Institut f\u00fcr Medienforschung Hans-Bredow-Institut","isPartOf":{"@id":"https:\/\/leibniz-hbi.de\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/#primaryimage"},"image":{"@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/#primaryimage"},"thumbnailUrl":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/07\/Header-Publikationen.jpg","datePublished":"2026-10-28T14:51:12+00:00","dateModified":"2026-10-29T15:20:18+00:00","breadcrumb":{"@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/#primaryimage","url":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/07\/Header-Publikationen.jpg","contentUrl":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/07\/Header-Publikationen.jpg","width":1380,"height":400,"caption":"flying book with open pages in front of a blue sky"},{"@type":"BreadcrumbList","@id":"https:\/\/leibniz-hbi.de\/en\/hbi-publications\/deliberation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Homepage","item":"https:\/\/leibniz-hbi.de\/en\/"},{"@type":"ListItem","position":2,"name":"Can the Quality of Online Discussions Be Automatically Assessed\u2014and Improved?"}]},{"@type":"WebSite","@id":"https:\/\/leibniz-hbi.de\/en\/#website","url":"https:\/\/leibniz-hbi.de\/en\/","name":"Leibniz HBI","description":"","publisher":{"@id":"https:\/\/leibniz-hbi.de\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/leibniz-hbi.de\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/leibniz-hbi.de\/en\/#organization","name":"Leibniz HBI","url":"https:\/\/leibniz-hbi.de\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/leibniz-hbi.de\/en\/#\/schema\/logo\/image\/","url":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/01\/HBI_Logo-Kopie-2.png","contentUrl":"https:\/\/leibniz-hbi.de\/wp-content\/uploads\/2024\/01\/HBI_Logo-Kopie-2.png","width":462,"height":123,"caption":"Leibniz HBI"},"image":{"@id":"https:\/\/leibniz-hbi.de\/en\/#\/schema\/logo\/image\/"}}]}},"permalink_manager":null,"_links":{"self":[{"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/hbi-publications\/11960","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/hbi-publications"}],"about":[{"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/types\/hbi-publications"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/media\/2579"}],"wp:attachment":[{"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/media?parent=11960"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/leibniz-hbi.de\/en\/wp-json\/wp\/v2\/tags?post=11960"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}