A Comparison of Language Model Performance vs. Human Performance on GLUE Tasks

Started Mar. 16, 2021

Abstract or project description

GLUE (Wang et al., 2019) is a set of popular language tasks in Natural Language Processing. However, models trained on GLUE tasks often outperform humans, showing clear training error. Thus, we seek a comparison of the most popular language models, which an be done with accuracy and model performance - i.e. what is it missing, what is it getting right, how does it relate to human performance.