Topic: On the Opportunities and Risks of Foundation Models—Evaluation
Session Lead: Ming Jiang
Gathering the experience of over 100 AI research scholars from different backgrounds, this survey report provides a comprehensive overview of the opportunities and risks of foundation models (i.e., "any model that is trained on broad data at scale and can be adapted [e.g., fine-tuned] to a wide range of downstream tasks"). Rather than singly emphasizing the technical properties of such fashionable AI products, by taking the ecosystem of foundation models in society into account, this report hopes to "highlight all the interdisciplinary connections" of foundation models to improve the understanding of their sociotechnical nature and provide suggestions for future research.
The session will focus on the model evaluation for discussion because the process of evaluation is critical to understand how these fancy models work, and getting to know existing evaluation methods and their limitations is helpful for being aware of any uncertainties in the perception of foundation models’ properties.
Further details about can be found on the CFG website. For most up-to-date information about the group, please subscribe to our mailing list by subscribing online or sending an email with the subject line "subscribe cfg."
Questions? Contact Lan Li
This event is sponsored by Conceptual Foundations Group (CFG)