The Glossary of Human-Centered AI

Key terms you need to know, whether you're coming from an AI, HCI, or UX background.

Image generated by DALL-E

The terms in this glossary fall into three broad categories:

• HCAI: Mostly classic HCI concepts, such as mental models, applied to AI systems.

• Models, Ethics, and Fairness: Concepts originating from STS (and conferences such as FAccT) that consider the ethical considerations, biases, and fairness aspects of AI systems.

• Machine learning: A subset of the field of AI that leverages data to develop models for certain tasks and is the basis for most modern AI systems.

• Uncertainty: Machine learning is deeply connected to uncertainty. This section focuses on understanding and managing uncertainty in AI models, which is a critical aspect of their performance and reliability.

• Explainability (XAI): Methods and techniques for making AI models more transparent, interpretable, and understandable to humans. 

Where I had them on hand, I have included references for the definitions in the footnotes. For feedback, or to add new terms, please contact me at nsalehi@berkeley.edu. Special thank you to my Spring 2023 HCAI class and Jeff Bigham for their help in creating this glossary.

Human-Centered AI: The study and design of AI systems with a focus on the human and social aspects of these systems.

• Human-Computer Interaction: HCI is an interdisciplinary field that draws upon computer science, social sciences, design, human factors, and other related disciplines to create technology that is usable, useful, and enjoyable and meets the needs of its users. Researchers in HCI also study the impacts of technology on people and societies.

• Socio-technical systems: Socio-technical systems are characterized by the integration of social and technical elements, where the social aspects include human behavior, culture, organization, and communication, and the technical aspects include technology, infrastructure, and systems. The concept emphasizes the importance of considering both social and technical factors when designing and deploying complex systems.

• Value-sensitive design (VSD)¹: an approach to design that aims to incorporate ethical, social, and human values into the design and development of technology. VSD emphasizes the importance of considering the impact of technology on various stakeholders, including end-users, communities, and society at large, and seeks to align design decisions with these stakeholders' values and interests. The goal of VSD is to create technology that not only meets functional requirements but also considers the ethical implications and social consequences of technology, such as privacy, security, equity, fairness, sustainability, and inclusivity. An example of a modified VSD approach is Privacy by Design².

• Mental model: cognitive frameworks or representations that people use to understand and interact with the world around them. These models are formed through a combination of past experiences, knowledge, and beliefs, and they guide an individual's decision-making and problem-solving processes. The black-box nature of many AI systems, makes mental models especially important to understand. Understanding users' mental models of AI can help designers create more intuitive and effective AI interfaces, as well as identify potential sources of confusion or misunderstanding.

• Gulf of execution³: the difference between the intentions of the users and what the system allows them to do, or how well the system supports those actions (e.g. what can I do with this system? How hard is it to do what I want to do?).

• Gulf of evaluation: the difficulty of assessing the state of the system and how well the artifact supports the discovery and interpretation of the state.

The gulfs of execution and evaluation were first developed to explain the complexity of interacting with early technologies but are particularly relevant to AI systems. See for instance “The biggest bottleneck for large language model startups is UX” where the questions are: when should I ask the system for help? What can I expect the system to do for me?

• Expectation Confirmation Model (ECM)⁴: the theory that user satisfaction and acceptance in a system is directly related to the difference between initial expectation and actual experience. Based on this model steps can be taken in the design of AI systems to better align expectations, such as communicating accuracy, helping the user detect basic principles of the system through examples, and giving the user control over the AI system.

• Cognitive Artifacts: Cognitive artifacts are external tools that are designed to enhance or augment human cognitive abilities, such as perception, memory, attention, problem-solving, decision-making, and communication. Examples of cognitive artifacts include calculators, text editors, and machine learning models. Cognitive artifacts change the nature of the work done by the person.

• Human-AI collaboration: cooperation between humans and AI systems in performing tasks or solving problems. It involves combining the strengths and capabilities of both humans and AI to achieve better results than what each could achieve individually. The framing of “collaboration” has been critiqued for labor appropriation and credit reassignment⁵. Other phrasings have been proposed such as “AI-advised human decision making.”

• Mixed-initiative interaction: related to the previous concept, is an early term used to describe computer systems that are designed to be pro-active and make suggestions or ask questions from the user. Clippy⁶ is an infamous early example of mixed-initiative interaction design that was meant as an office assistant.

• Appropriate trust/reliance: Making informed judgments about when and how much to trust and rely on AI systems, considering the accuracy, performance, context, purpose, and potential consequences of their use.

• Machine learning as design material⁷: The process of incorporating awareness of ML capabilities and limitations in the design process. Key challenges in this work are abstracting ML capabilities from a UX perspective, and how to sketch and prototypes ML systems early and often.

Models, Ethics and Fairness: the ethical considerations, biases, and fairness aspects of AI systems.

• Measurement modeling⁸: the process of creating a statistical model to represent and analyze the underlying structure of observed data. Measurement modeling involves specifying the relationships among the latent constructs and their indicators, and estimating the model parameters using statistical techniques. The model parameters provide information about the strength and direction of the relationships between the latent constructs and their indicators, and can be used to assess the validity and reliability of the measurement instruments.

  • Latent Constructs: unobservable variables or constructs of interest that are hypothesized to exist based on theoretical or conceptual frameworks.

  • Construct validity: the extent to which a measurement instrument accurately measures the theoretical or conceptual construct it is intended to assess. In other words: is it correct?

  • Construct reliability: a measure of the consistency and stability of a measurement instrument in assessing a specific construct or concept. In other words: can it be repeated?

• Value alignment: Ensuring that AI models or systems are designed and trained to make decisions or take actions that are consistent with the intended values and expectations of the human users or the broader societal context in which the AI system operates. The “alignment problem” has some apparent overlaps with the concept of value sensitive design (VSD) but has different origination, orientation, and uses. VSD originates from the field of information system design and takes a human-centered approach to design by integrating the values of users, communities, and society throughout the entire design process. Common methods used in VSD include qualitative research with stakeholders and value-tradeoff analysis. Value alignment originates from the field of economics and aims to model human values and objectives so that AI systems can “align” with them. Common methods used are reward engineering, preference elicitation, and value aggregation.

• Group Fairness vs. Individual Fairness: Group Fairness refers to the fairness of an algorithm or decision-making process in treating different groups of individuals equally or without bias. It focuses on ensuring that the outcomes of a system are not systematically biased against certain groups based on protected characteristics such as race, gender, age, religion, or other demographic attributes. Individual Fairness, on the other hand, focuses on the fairness of an algorithm or decision-making process in treating individuals on a case-by-case basis. It emphasizes that similar individuals or cases should be treated similarly, regardless of the group to which they belong.

• Privacy-Preserving AI: the practice of developing and deploying AI systems while protecting the privacy and confidentiality of sensitive data used in the training, evaluation, or inference processes. The essential components of privacy-preserving machine learning include Federated learning, Homomorphic Encryption, and Differential Privacy. Privacy-Preserving AI techniques aim to strike a balance between the need for AI systems to access and utilize data for effective learning and decision-making, and the need to protect the privacy and confidentiality of sensitive data. 

• Audit: The systematic examination, review, or evaluation of processes, systems, or activities to determine their accuracy, completeness, reliability, and compliance with relevant laws, regulations, standards, or internal policies. Audits are typically conducted by trained and qualified professionals known as auditors, who follow established auditing standards and methodologies.

• Normative ethics: also known as prescriptive ethics, is a branch of philosophy that deals with the study of ethical theories and principles that guide human behavior and moral decision-making. 

• The Impossibility of Fairness: the notion that achieving perfect or absolute fairness in AI and machine learning models may be impossible to achieve in practice. This idea stems from the fact that different definitions and measures of fairness can sometimes conflict with each other, making it challenging to simultaneously satisfy all fairness criteria in certain situations.

Machine learning: A subset of the field of AI that leverages data to develop models for certain tasks. 

• Agent: a software system that is designed to perform tasks autonomously. Simple agents follow predetermined rules or algorithms and have no capacity to learn or adapt. More complex agents are capable of learning from experience, making decisions, and taking actions based on their understanding of their environment and their goals. Some systems combine multiple interacting agents to reach an outcome, these are called Multi-Agent Systems (MAS).

• Three main types of machine learning:

  • Supervised learning: A model is trained to make predictions or decisions based on labeled data. 

  • Unsupervised learning: A model learns from unlabeled data, without any explicit supervision or labeled examples to guide its learning process. The goal of unsupervised learning is to discover patterns, structures, or relationships in the data without any prior knowledge of the correct output labels.

  • Reinforcement learning: An agent learns to make decisions and take actions in an environment to maximize a cumulative reward signal. Reinforcement learning is characterized by a trial-and-error learning process, where the agent learns from the consequences of its actions.

• Reinforcement Learning with Human Feedback (RLHF): a type of reinforcement learning in which a human provides feedback to the agent to help it learn more quickly and accurately. The feedback can take the form of rewards or punishments, or more complex signals that provide information on the quality of the agent's actions such as editing the agent’s output to improve it. The human feedback can be used to supplement or replace the traditional reward signal in reinforcement learning. This can be particularly useful in situations where the reward signal is sparse or difficult to define, such as in complex real-world environments.

• Interactive Machine Learning (IML)⁹: Users iteratively provide feedback to a learner after viewing its output, in a process of rapid: train, feedback, correct. Research shows that interactive ML trains both the learner and the user over time (for instance by helping users develop more accurate mental models). The three main ways that a user can teach a model are: demonstration, feedback, and providing examples.

• Active learning: A machine learning paradigm in which the learner picks the examples to learn from. For instance, a model might process a set of data, pick the data points its the least sure about, and ask an “oracle” for the correct answer for just those data points. The oracle might be a paid data labeler or a user.

• Large Language Models (LLM): machine learning models that are trained on massive amounts of text data to generate human-like language. These models use deep learning techniques, such as neural networks, to learn the statistical patterns and relationships between words and phrases in the input text. Some of the most well-known large language models include GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers). Other names for LLMs have been proposed to better describe the statistical nature of word generators and resolve confusion around their apparent intelligence, these include Pattern Synthesis Engines¹⁰ and Language Calculators¹¹.

• Zero-shot and Few-shot Learning: Zero-shot learning refers to the ability of a machine learning model to perform a task without any prior training examples. Instead, the model is given a description or attributes of the task, and it uses its knowledge of the relationships between different concepts to make predictions. For example, a zero-shot learning model might be able to recognize a new type of fruit based on its description and its similarity to other known fruits. Few-shot learning, on the other hand, refers to the ability of a machine learning model to perform a task with only a small number of labeled training examples. Typically, the model is trained on a small number of examples from a few different classes, and it must learn to generalize to new examples from unseen classes. Few-shot learning is particularly useful in situations where obtaining large amounts of labeled data is difficult or expensive.

• Adversarial attacks: Deliberate attempts to manipulate or deceive machine learning models by introducing small, carefully crafted perturbations to input data. 

• Model cards¹²: A form of documentation that provides information about the performance, capabilities, and limitations of an AI model. They are typically used as a tool for transparency and accountability, allowing users and stakeholders to better understand the behavior and characteristics of an AI model.

• Datasheets¹³: documents or metadata that provide information about various aspects of a dataset. Datasheets are intended to provide transparency and documentation about the data used in machine learning models, and they are typically created by data providers, data scientists, or researchers to share information about the dataset with others, including users, stakeholders, and regulators.

Uncertainty: Machine learning is deeply connected to uncertainty. This section focuses on understanding and managing uncertainty in AI models, which is a critical aspect of their performance and reliability. 

• Aleatoric (aka statistical) uncertainty: refers to the notion of randomness, that is, the variability in the outcome of an experiment which is due to inherently random effects.

• Epistemic (aka systematic) uncertainty: refers to uncertainty caused by a lack of knowledge, i.e., to the epistemic state of the agent. As opposed to aleatoric uncertainty, epistemic uncertainty can in principle be reduced with additional information.

• Type I and Type II errors: a Type I error in machine learning, also known as a false positive, occurs when the model incorrectly classifies a negative instance as positive. For example, in a binary classification task, a Type I error occurs when a model predicts a sample as belonging to the positive class when it actually belongs to the negative class. Type II error in machine learning, also known as a false negative, occurs when the model incorrectly classifies a positive instance as negative. For example, in a medical diagnosis task, a Type II error occurs when a model fails to diagnose a disease in a patient who actually has the disease.

• High Recall versus High Precision (optimize for false positives vs false negatives): High recall in a machine learning context refers to the ability of a model to identify a high proportion of all relevant instances in the dataset. In other words, a model with high recall is able to correctly identify a large percentage of instances that belong to a certain class. For example, in a medical diagnosis system, high recall means that the system is able to correctly identify a high percentage of patients who have a certain disease, even if it may also include some false positives. High precision in machine learning refers to the ability of a model to correctly identify a high proportion of relevant instances among all the instances that it identifies as belonging to a certain class. In other words, a model with high precision is able to correctly classify most of the instances it predicts as belonging to a certain class, and has a low rate of false positives. For example, in a fraud detection system, high precision means that the system is able to accurately classify most of the transactions it flags as fraudulent, and has a low rate of false positives. In summary, high recall in machine learning focuses on identifying most of the relevant instances in a dataset, even if some false positives are included, while high precision focuses on correctly classifying most of the instances it identifies as belonging to a certain class, with a low rate of false positives. There is a tradeoff between optimizing for high recall vs. optimizing for high precision.

  

  The tradeoff between high precision vs. high recall. Source: https://medium.com/analytics-vidhya/precision-recall-tradeoff-79e892d43134

• Calibration: In machine learning, calibration refers to the agreement between the predicted probabilities of a classification model and the true probabilities of the events it is predicting. In other words, a calibrated model has predicted probabilities that accurately reflect the true probabilities of the events it is predicting.

• Error boundary: The regions in which a machine learning model is correct vs. incorrect. If the user’s mental model of the error boundary does not align with the true boundary, it can result in suboptimal decisions¹⁴. For instance, the person may trust the AI even when it provides an incorrect recommendation, or conversely, may not trust the AI even when it makes a correct recommendation. These types of decisions can negatively impact productivity and accuracy. Models with error boundaries that align with people's mental models can improve decision-making, as people are better able to discern when the model is wrong. People have better mental models of the error boundary (i.e. when is the model wrong?) for models where the error boundaries have:

  • Non-stochasticity: An error boundary is considered non-stochastic if it effectively separates all erroneous predictions from the correct ones. However, in real-world scenarios, the error boundary of a model may exhibit stochastic behavior due to several reasons. These include generalization, where the model's ability to make accurate predictions on new data is limited, representation mismatch between the AI and user, and inherent stochasticity in the outcome being predicted.

  • Parsimony: An error boundary is parsimonious if it is simple to represent.

Explainability (XAI): Methods and techniques for making AI models more transparent, interpretable, and understandable to humans. 

• Explainability gap: The challenge of understanding and explaining the decision-making process of complex automated systems, such as machine learning models, deep learning algorithms, and other AI systems. These systems often operate as "black boxes," making decisions based on intricate calculations and patterns that may be difficult to interpret or explain in human-understandable terms.

• Intelligibility gap: The disparity or disconnect between the level of understanding or interpretability of an AI or machine learning model by humans versus its actual functioning or decision-making process. In other words, it refers to the difficulty or challenge in comprehending or explaining the inner workings, decision-making process, or reasoning of a complex machine learning model, especially when it involves deep learning, black-box models, or complex algorithms.

• Glassbox ML models: models that are designed to be inherently interpretable, in contrast to "black box" models.

• Model-agnostic explanations: refer to explanations or interpretability techniques in machine learning that are not tied to any specific machine learning model or algorithm. 

• Plausibility: The degree to which an explanation or interpretation of a machine learning model's prediction or decision is reasonable, believable, and makes sense to a human user 

• Faithfulness: The degree to which an explanation or interpretation accurately reflects the actual behavior of the underlying machine learning model. 

• Post-hoc interpretability: also known as post-hoc explanation or post-hoc analysis, refers to the process of interpreting the decisions or predictions made by a machine learning model after it has been trained. Post-hoc interpretability techniques are typically used for black-box or complex machine learning models that are not inherently interpretable, such as deep neural networks, support vector machines, or ensemble models. 

• Adaptive Explainability: The ability of an AI system to dynamically adjust and provide explanations for its decision-making process based on various factors or context. It involves the system's capability to adapt its explanations in real-time or in response to changing conditions, inputs, or user requirements. For example, an AI system used in a healthcare setting may need to provide different explanations to a patient, a nurse, or a physician, taking into account their varying levels of expertise and understanding of the technical details.

• Fidelity: the approximate quality of explanation models.

• Fidelity gap¹⁵: the disparities in approximation quality between explanation models. (i.e. some groups receive better ML explanations than others).

• SHAP: SHAP, short for SHapley Additive exPlanations, is a popular and widely used method for explaining the predictions or outcomes of machine learning models. The Shapley value is a mathematical concept that originates from cooperative game theory and provides a way to fairly distribute a value among a group of individuals or players. In the context of machine learning, SHAP uses the Shapley value to attribute a prediction or outcome of a model to the input features or variables that contributed to that prediction.

• LIME: LIME, short for Local Interpretable Model-agnostic Explanations, is a popular and widely used method for explaining the predictions or outcomes of machine learning models. LIME works by approximating the complex model locally around a specific instance or prediction of interest with a simpler, interpretable model, such as linear regression or decision trees. 

• Counterfactual explanation: A counterfactual is a hypothetical scenario that involves changing one or more input features or conditions of a prediction while keeping the remaining features unchanged, in order to explore how the model's prediction would have changed in response to different inputs.

Please reach out with additional words or citations or any feedback you may have at nsalehi@berkeley.edu. Thank you!

1. Friedman, Batya, et al. "Value sensitive design and information systems." Early engagement and new technologies: Opening up the laboratory (2013): 55-95.

2. Hustinex, P. "Privacy by design: delivering the promises. Identity Inf Soc 3 (2): 253–255." (2010).

3. Norman, Donald A. "Cognitive artifacts." Designing interaction: Psychology at the human-computer interface 1.1 (1991): 17-38.

4. Kocielnik, Rafal, Saleema Amershi, and Paul N. Bennett. "Will you accept an imperfect ai? exploring designs for adjusting end-user expectations of ai systems." Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019.

5. Sarkar, Advait. "Enough With “Human-AI Collaboration”." Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 2023.

6. https://knowyourmeme.com/memes/clippy

7. Yang, Qian. "Machine learning as a UX design material: how can we imagine beyond automation, recommenders, and reminders?." AAAI Spring Symposia. Vol. 1. No. 2.1. 2018.

8. Jacobs, Abigail Z., and Hanna Wallach. "Measurement and fairness." Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.

9. Fails, Jerry Alan, and Dan R. Olsen Jr. "Interactive machine learning." Proceedings of the 8th international conference on Intelligent user interfaces. 2003.

10. I first heard of pattern synthesis engines on this blog https://redeem-tomorrow.com/what-if-bill-gates-is-right-about-ai

11. I first heard of language calculators on this podcast with Emily Bender and Timnit Gebru https://www.podchaser.com/podcasts/factually-with-adam-conover-853712/episodes/ai-and-stochastic-parrots-with-170946764

12. Mitchell, Margaret, et al. "Model cards for model reporting." Proceedings of the conference on fairness, accountability, and transparency. 2019.

13. Gebru, Timnit, et al. "Datasheets for datasets." Communications of the ACM 64.12 (2021): 86-92.

14. Bansal, Gagan, et al. "Beyond accuracy: The role of mental models in human-AI team performance." Proceedings of the AAAI conference on human computation and crowdsourcing. Vol. 7. 2019.

15. Balagopalan, Aparna, et al. "The road to explainability is paved with bias: Measuring the fairness of explanations." 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022.