MBBMD: Media Bias Bias-Mitigated Dataset

Abstract

MBBMD: Media Bias Bias-Mitigated Dataset MBBMD ( Media Bias Bias-Mitigated Dataset ) is a dataset designed for bias detection in Spanish-language news articles. The dataset is structured hierarchically into two annotation levels, enabling research in binary bias classification as well as fine-grained bias categorization. This dataset has been created to support Natural Language Processing (NLP) research , bias detection models , and media studies by offering a structured and annotated corpus of news articles covering diverse political perspectives and a range of bias types. MBBMD consists of 100 news articles sourced from multiple Spanish-language media outlets, annotated using a perspectivist approach (LeWiDi) at the document level. The dataset is divided into two main phases , each containing training, testing, and control subsets. MBBMD is structured into two levels of analysis: Level 1: Document-level binary classification This level determines whether a news article is biased or not, based on majority vote agreement among annotators. It includes percentage scores reflecting the degree of agreement. Level 2: Multilabel bias classification This level categorizes bias into five specific types: intentional bias, spin bias, statement bias, coverage bias, and gatekeeping bias. Each type includes binary majority vote annotations and percentage agreement scores from annotators. To enhance annotation robustness , Counterfactual Data Augmentation (CDA) techniques were applied to a subset of the dataset. These modifications involve outlet swaps, entity swaps, and terminological changes, allowing for the assessment of how these factors influence annotator perceptions of bias. Dataset File Structure and Field Descriptions The Multilevel Bias Detection Dataset for Spanish Media (MBBMD) is organized into two main directories, corresponding to the two annotation phases : Phase 1 (Document-level annotations) : Focused on binary and multilabel bias classification at the article level. Phase 2 (Sentence-level annotations) : Provides fine-grained sentence-level bias annotations. Each directory contains three subsets : train : The primary dataset for training models. test : A reserved subset for model evaluation. control : Contains articles modified using Counterfactual Data Augmentation (CDA) to analyze annotation robustness. The dataset files are stored in TSV (Tab-Separated Values) format , encoded in UTF-8 , ensuring compatibility with most data processing tools. Phase 1: Document-Level Annotations Files: phase_1/train_phase1.tsv phase_1/test_phase1.tsv phase_1/control_phase1.tsv Fields in Phase 1 files Each row represents a full news article annotated for bias. topic : The topic of the article (e.g., Pedro Sánchez Investiture , Barcelona Amnesty Protest ). text_id : A unique identifier for each article. title : The headline of the article. text : The full body of the news article. is_biased_majority_vote : Binary label (1 = biased, 0 = not biased), determined by the majority vote of annotators. is_biased_% : The percentage of annotators who classified the article as biased. Phase 2: Multilabel Bias Classification at the Document Level Files: phase_1/train_phase2.tsv phase_1/test_phase2.tsv phase_1/control_phase2.tsv Fields in Phase 2 files Each row corresponds to a news article , but with bias annotations broken down into five specific categories. topic : Topic of the article. text_id : Unique identifier for the article. title : Headline of the article. text : Full text of the article. is_intentional_bias_majority_vote : Binary label (1 = present, 0 = absent) indicating whether intentional bias is detected. is_spin_bias_majority_vote : Binary label indicating the presence of spin bias. is_statement_bias_majority_vote : Binary label indicating statement bias. is_coverage_bias_majority_vote : Binary label indicating coverage bias. is_gatekeeping_bias_majority_vote : Binary label indicating gatekeeping bias. is_intentional_bias_% : Percentage of annotators who identified intentional bias. is_spin_bias_% : Percentage of annotators who identified spin bias. is_statement_bias_% : Percentage of annotators who identified statement bias. is_coverage_bias_% : Percentage of annotators who identified coverage bias. is_gatekeeping_bias_% : Percentage of annotators who identified gatekeeping bias. These fields enable multilabel classification , allowing researchers to analyze different dimensions of bias simultaneously. Phase 3: Sentence-Level Bias Manifestation Annotations Files: phase_3/train_phase_3.tsv (1,231 sentences from 49 articles) phase_3/test_phase_3.tsv (529 sentences from 21 articles) phase_3/control_phase_3.tsv (588 sentences from 30 articles) Fields in Phase 3 files Each row represents a single sentence within an article, annotated for six specific bias manifestation types based on the taxonomy from Recasens et al. (2013) and Spinde et al. (2022) . topic : The topic of the parent article. text_id : Identifier linking the sentence to its parent article (matches Phase 1 and Phase 2). outlet : The media outlet that published the article. date : Publication date of the article. title : Headline of the parent article. sentence_id : Unique identifier for each sentence. sentence_text : The text of the sentence. previous_text : The preceding sentence (for context). next_text : The following sentence (for context). Failure_to_attribute_sources : Whether the sentence omits attribution for evaluative claims (True/False). Sensationalism/Emotionalism : Whether the sentence uses sensationalist or emotionally charged language (True/False). Mind_Reading : Whether the sentence attributes thoughts or intentions without evidence (True/False). Word_choice/labelling : Whether the sentence uses loaded terms or biased labeling. When present, includes the specific wording (e.g., {'value': True, 'wording': 'term'} ). Subjective_qualifying_adjectives : Whether the sentence contains subjective adjectives expressing opinion. When present, includes the specific wording. Opinions_as_facts : Whether the sentence presents opinions using declarative/factual framing (True/False). This phase enables sentence-level bias detection and fine-grained bias manifestation analysis , supporting hierarchical approaches that decompose document-level bias into sentence-level manifestations. Total: 2,348 sentences across 100 articles, covering 10 topics and 10+ media outlets.