Bias and Fairness in Natural Language Processing

1. What is?

1.1. **What:** Bias in NLP refers to results that reflect models that reflect and perpetuate historical human biases, negatively affecting society and the company that uses this system.

1.2. **Why:** NLP has existed for a long time, but only in recent years has it become essential to human life, integrating into systems and decisions that impact us daily. Every major new technology and discovery brings unintended consequences; our best approach is to identify, measure, control, and mitigate them as much as possible to ensure this technology remains beneficial to humanity.

2. Problems

2.1. **Type of problems**

2.1.1. **Ethical and Legal**

2.1.1.1. Copyrights

2.1.1.1.1. **Example**: A lot of artist suing companies for copyrights because different models were trained with their work, with no consent

2.1.1.2. Privacy

2.1.1.2.1. **Example:** Models trained on personal messages or user interactions might inadvertently reproduce personal information

2.1.1.3. Security

2.1.1.3.1. **Example:** Chatbots or language models might be manipulated to generate harmful instructions

2.1.2. **Bias**

2.1.2.1. Gender

2.1.2.1.1. **Example:** Problem when prioritize male candidates for a position just because the gender

2.1.2.2. Cultural

2.1.2.2.1. **Example:** Translation models may translate non-Western names or phrases incorrectly or associate them with Western terms.

2.1.2.3. Racial

2.1.2.3.1. **Example:** El sistema de reconocimiento facial suele confundir a los afroamericanos

2.1.2.4. Socieconomic

2.1.2.4.1. **Example:** NLP models used in risk assessment may disproportionately classify individuals from low-income backgrounds as likely to reoffend, leading to a higher likelihood of parole denia

2.2. **Real world problems:**

2.2.1. **Amazon hiring toool:** "Amazon scrapped an AI hiring tool that showed bias against women, illustrating how biased training data can lead to discriminatory outcomes"

2.2.2. **Translation problems:** Google Translate has faced criticism for gender biases, such as defaulting to masculine pronouns for certain professions

2.2.3. **COMPAS Recidivism Prediction:** In the US, the COMPAS algorithm used in courts for recidivism prediction has been shown to disproportionately label African American defendants as “high risk” for reoffending

2.2.4. **Twitter Sentiment Analysis (Cultural Bias)** African American Vernacular English (AAVE) expressions, such as “lit” or “bad,” were often classified as negative when they carry positive meanings in specific contexts

2.2.5. **OpenAI’s GPT-3 (Privacy and Data Leakage):** During its early releases, OpenAI’s GPT-3 occasionally reproduced sensitive information from the internet.

3. **Ethical Framwork (Microsoft)**

3.1. **Fairnes:**

3.1.1. Ensure that AI systems treat all individuals fairly and do not reinforce existing societal biases.

3.1.2. Develop processes for detecting and mitigating biases, such as fairness audits, and evaluate models to prevent discrimination in language processing applications

3.2. **Reliability and safety**

3.2.1. AI systems should perform reliably and safely, with predictable outcomes and minimal errors.

3.2.2. Continuously monitor NLP models to avoid harmful outputs, especially in sensitive areas like mental health chatbots or legal document processing, where incorrect outputs could lead to negative consequences.

3.3. **Privacy and security**

3.3.1. Respect privacy by protecting personal data and ensuring data security in all stages of development and deployment.

3.3.2. Apply differential privacy techniques and rigorous data anonymisation methods to protect users’ data, and avoid any outputs that could reveal private information.

3.4. **Inclusiveness**

3.4.1. Strive to build AI systems that are accessible and beneficial to all, considering the needs of diverse user groups.

3.4.2. Involve diverse teams in model development and validation, ensuring that NLP applications accommodate various languages, dialects, and accessibility needs, so they are usable by people with different backgrounds and abilities.

3.5. **Transparency**

3.5.1. Maintain openness about AI technologies, including the data and models used, so that stakeholders understand how decisions are made.

3.5.2. Provide clear explanations of how NLP models work and disclose limitations. Use interpretable models or explainability tools

3.6. **Accountability**

3.6.1. Establish mechanisms to hold AI systems and their creators accountable for any negative impacts.

3.6.2. Develop governance structures that include regular evaluations and updates of models

4. Source of problems

4.1. **Data:**

4.1.1. Models replicate what humans feed them; if we use biased input data, the model will replicate the same biases that were fed to it, as the popular saying goes, ‘garbage in, garbage out’.

4.2. **Algorithms**

4.2.1. Algorithms can amplify biases present in the data or introduce new ones. Techniques like word embeddings or certain neural network architectures may encode and magnify underlying biases.

4.3. **Modelling choise**

4.3.1. The choice of model, parameters, and settings affects the fairness and accuracy of NLP outcomes. Simplified models or certain architectures may not capture nuances, leading to oversimplified and biased predictions.

4.4. **Humman Annotation**

4.4.1. Humans have a history of having problems with bias, very much related to between-measurement data, if we feed a model with biased labels it will generate biases in the models.

4.5. **Solutions**

4.5.1. **Understand the Data:** comprehensive data audit to check for imbalances and bias within the data, ensuring diversity across gender, ethnicity, socioeconomic status, and other relevant demographics.

4.5.2. **Data augmentations:** Use techniques such as data augmentation and synthetic data generation to address underrepresented classes or demographics

4.5.3. **Algorithm Features:** Feed the algorithm with only the data that should be relevant to the algorithm, e.g. if it is a screening model to choose HV, there is no need to feed the model with data such as gender, age, economic stratum, etc.

4.5.4. **Transparent Models:** For tools with critical decision is imperative to understand how the model us working to audit if there is any problem with that