CFG Seminar by Nigel Bosch: Construct Validity and Machine Learning
How do we know what a machine learning model predicts? The answer may seem obvious: it predicts whatever the labels are! However, the reality is more complicated. Machine learning models make predictions that are only an approximation of what we really want. Moreover, predictions may approximate things we do not want (e.g., demographic characteristics that are supposed to be orthogonal to predictions).
Consequently, it is worthwhile to understand what models actually predict. Fortunately, there is well over 60 years of research on a closely related topic: how do we know what an educational test measures?
This session provides an overview of some epistemological, ethical, and legal issues stemming from educational testing and how they apply to machine learning. The session will also describe methodological techniques that can be applied to assess machine learning models and educational tests alike to examine the degree to which they predict what we want to predict -- i.e., their "construct validity."
This event is sponsored by Conceptual Foundations Group