Tony Wu

How realistic and accurate are our implementations of NLP models like GPT-3 when predicting text?

Started June 17, 2022

Abstract or project description

Natural Language Processing (NLP) is a field in Machine Learning where Artificial Intelligence (AI) works with human language- more specifically, the AI tries to understand, analyze, and generate text. Our research paper will focus more on the text prediction and generation of NLP. The way it works is that when someone types in a bit of text, the AI will try to predict what will be typed next. Text prediction has a variety of uses, like autofilling queries in search engines and improving ease of use of text applications like Word. Currently, there are many models like GPT-3 that can accurately predict text. In our research paper, we will look at models of text predictions like GPT-3 through prior research papers. Then, based on the information in the research papers, we will try to emulate and implement these models through the use of programming. We will also experiment with these models, changing hyper-parameters, weights, and biases to see how it affects the model. The models will be trained, and their relative accuracies will be noted. We will take note of the results of the experiment, as well as give a brief description and background of the models we implemented. Due to resource constraints like limited time and processing power, we do not expect the results of our models to be better than the ones we emulated. However, we do intend that our results provide an insight as to how far the quality of text predictions and NLP models have improved in previous years.