readme.md

A Dataset for Sentence-Level Factuality and Media Bias Prediction in Portuguese

Automated fact-checking and news credibility verification at scale require accurate prediction of news factuality and media bias. Here, we introduce a large sentence-level dataset, FactNews, composed of 6,191 sentences expertly annotated according to the factuality and media bias definitions proposed by AllSides. We used FactNews to assess the overall reliability of news sources by formulating two text classification tasks: predicting the sentence-level factuality of news reporting and the bias of media outlets. Our experiments demonstrate that biased sentences tend to contain more words than factual sentences and exhibit a predominance of emotional content. This fine-grained analysis of subjectivity and impartiality in news articles showed promising results for predicting the reliability of entire media outlets. Finally, due to the severity of fake news and political polarization in Brazil and the lack of research in Portuguese, both the dataset and baselines were developed specifically for Portuguese.

The following image illustrates the annotation schema used to label FactNews::

The following table describes in detail the FactNews labels, documents, and stories:

CITING / BIBTEX

Please cite our paper if you use our dataset:

@inproceedings{vargas-etal-2023-predicting,
    title = "Predicting Sentence-Level Factuality of News and Bias of Media Outlets",
    author = "Vargas, Francielle  and
      Jaidka, Kokil  and
      Pardo, Thiago  and
      Benevenuto, Fabr{\'\i}cio",
    editor = "Mitkov, Ruslan  and
      Angelova, Galia",
    booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.ranlp-1.127",
    pages = "1197--1206",
    }

FUNDING