Data Collection in Psychology: The Ultimate Guide

In previous articles, I talked about how to conduct psychology research and how to how to write research paper outlines. Today I wanted to give additional details about how to collect data in psychology. There are two main types of data collection in psychology: quantitative and qualitative. Quantitative data collection focuses on numerical data, which allows researchers to analyze large amounts of data quickly by conducting statistical analyses. Qualitative data focuses on rich word-based information, such as what people express in interviews and open-ended response questions. This type of information contains emotional nuances and depths that cannot be captured through numbers.

Both types of data are extremely valuable methods to conduct primary research, which can form the basis of a research paper that you write. However, for better or worse, most psychologists focus on quantitative research. In this article, I will discuss the three main types of quantitative research, including pros, cons, and examples.

How do you Design a Psychology Experiment?

Experiments are the bread and butter of psychology because they allow psychologists to determine causation - i.e., that something causes another thing to happen. The “something” is called the independent variable. It is what the researcher “manipulates” or changes in the experiment. The dependent variable(s) are what a researcher measures to determine if they are affected by the independent variable. Make sure that when you are designing an experiment, you only include one independent variable! Otherwise, you can’t be sure which independent variable caused the effect. 

To demonstrate why it’s important to only have one independent variable, imagine that you’re having trouble sleeping and you want to identify a way to improve your sleep. You think that you might be having trouble because you spend too much time on your phone before bed or you drink caffeine too late in the day. If you reduce your screen time and caffeine consumption on the same day and you sleep better, then you don’t know which variable caused your sleep to improve! It may have been the screen time, the caffeine consumption, or the combination of the two. 

Thus, the key to an experiment is randomly assigning participants to carefully crafted conditions. Carefully crafted conditions (I liked the alliteration) should only vary in one dimension - the independent variable. In other words, all participants do the same experiment, but there is one difference in the experimental condition compared to the control condition. Random assignment means that neither the participants nor the researchers choose which condition the participant is in. A good psychology experiment is also double-blind, which means that neither the participants nor the researchers know which condition the participant was assigned to. 

What are the Different Types of Data Collection Methods in Psychology?

In-person Psychology Experiments

To make this idea more tangible, I’ll describe a classic psychology study about bystander intervention, which is whether people choose to get involved when they think that there might be a problem. Participants were supposedly in the “waiting room” for a psychology experiment. They didn’t know that the experiment had already begun. The participant was either waiting alone or with two fake participants who were part of the research team (psychologists refer to these fake participants as “confederates”). The room began to fill with smoke, but the confederates did not react to the smoke. The target participants were much more likely to get help when they were alone than when they were with the confederates. The independent variable was the number of people in the room and the dependent variable was whether the participant sought help.

That was an example of an in-person psychology experiment, which of course has its pros and cons. In-person psychology experiments allow researchers to observe participants’ behaviors. In the smoke study, they could see whether the participant did a certain behavior (i.e., got help) or not. However, these types of experiments are very time-intensive because each participant has to come into the study one by one. Additionally, the participants will likely be less diverse in terms of age, gender, race, socioeconomic status, etc. because researchers typically conduct in-person experiments in a single geographical area. Often, university students are used for in-person experiments because they are a convenient, low-cost (and sometimes free) sample. Researchers can also recruit participants for in-person studies via social media, online advertisements, and flyers.  

Online Psychology Experiments 

Because of the limitations of in-person experiments, psychologists like to conduct research online. Many researchers create their online experiments using Qualtrics, which is a powerful and easy-to-use research tool. Two common platforms researchers use to recruit online participants are Prolific and Amazon’s Mechanical Turk (mTurk). A major benefit of online experiments is that they are quick. I can usually finish data collection for an experiment within a week. I design my experiment, launch it, and simply wait for the responses to roll in. Because online data collection is less time intensive, researchers can recruit a larger sample size (total number of participants) for their study. Within reason, it’s generally better to have a larger sample size than a smaller one. Would you rather have 30 participants to verify that your experiment works or 200 participants? 

Researchers can also get more diverse participants from online platforms. They are not limited by the people who are near their experiment location. However, online recruitment does not entirely solve the issue of participant diversity. There are still some types of participants who don’t or who are less likely to use these platforms. Nevertheless, the benefit of online recruitment is that researchers can much more easily recruit a specific population of participants (such as participants within a certain age group) with online platforms. 

Of course, online experiments also have their difficulties. Some participants are bots who aren’t doing the researchers’ experiment. They are just responding randomly or according to an algorithm. To identify bots, researchers use attention checks. For example, participants have to click “Strongly disagree” as the answer to a question to identicate that they are paying attention. Researchers also check for participants who were particularly fast or slow at completing the experiment, or for participants who answer the same response for every question. I have found that there are more bots on mTurk than Prolific. 

The other con of online experiments is that it is more difficult, although not impossible, to measure behavior. For example, researchers were interested in how parents’ view of failure concerning learning helped or hindered their children’s learning. The researchers manipulated whether participants viewed failure as debilitating, something that causes negative emotions and thereby hurts learning, or enhancing, something that identifies room for improvement and thereby improves learning. For the dependent variable, participants indicated on a scale from 1 to 6 how they would react if their child failed a math quiz. Parents in the failure-as-enhancing focused more on improving their child’s learning than those in the debilitating condition. This study is limited in that participants indicated how they would feel, not how they behaved. However, the researchers also used multiple studies, including an in-person study, to research this idea in various ways. 

As you’ve seen, experiments are a powerful tool that psychologists use to understand causation. However, sometimes researchers want to investigate something that would be difficult or unethical to study in an experimental setting. In these cases, they may turn to survey-based research. 

Survey-Based Research

Survey-based research or self-report research is the most widely used method of data collection in Psychology. As I mentioned earlier, surveys often yield qualitative data, such as answers to open-ended response questions, but in this section, I will focus on quantitative surveys. Quantitative surveys typically ask Likert scale questions, which have responses on a scale from 1 to 5, 1 to 7, etc. My personal favorite Likert scale is a 7-point scale because it captures nuances about the ways that a participant can feel about a topic. On a 7-point Likert scale, participants can respond with 1 = Strongly disagree; 2 = Disagree; 3 = Somewhat disagree; 4 = Neither; 5 = Somewhat agree; 6 = Agree; 7 = Strongly agree. Researchers can then do statistical analyses on the responses, such as calculating the mean, median, and standard deviation. 

When psychologists cannot ethically experiment on the subject of interest, they often conduct surveys instead. For instance, they may ask participants questions about their mental health and factors that may be related to their mental health. Participants could answer questions about how sad or anxious they feel, along with questions about the number of close friends they have and the amount of time they spend on social media. Researchers may find that the amount of time the participants spend on social media is related to how sad they feel. “Related” is key. Since the researchers are not experimenting, they cannot determine causation. Remember: correlation does not equal causation!

Other times, researchers conduct surveys simply to gather information about a topic. For example, a researcher might want to know how much participants agree or disagree with certain political issues. The researcher may or may not want to experiment with analyzing the results.

Both experimental and survey-based research have one noticeable problem: demand characteristics. “Demand characteristics” are cues or signs that participants may get that may cause them to act differently because they know that they are being studied. They may give responses that they think the researcher is looking for. Thus, psychologists may use observational research because those participants do not know that they are in a study. 

Observational Research

In an observational study, psychologists notice and make notes about what people do in the real world without knowing that a researcher is around. For example, a researcher could go to a cafeteria and count the number of people who separate their recyclables and trash versus the number of people who throw everything into the bin. Not having to worry about demand characteristics is a major benefit to this type of research, but real-world data is messy! This is not an experiment, where participants were randomly assigned to conditions, or a survey, where you select your participants intentionally.

In the recycling example, there are several unknown variables at play. Do people who come on certain days have different recycling habits than other people? What about people who eat at different times? What if the researcher made their observations on a day that was different from normal for some reason? To mitigate these unknown variables, the researcher would make their observations on multiple days and at multiple times. They could also repeat the process at a different location. 

What Types of Data are Easier Analyze?

For all types of research, you must consider how to collect data that is “clean” (easier to analyze). Thinking about this beforehand will make your work much simpler in the future! During online experiments or surveys, make sure that participants give you responses in the format that you want. For example, when you ask participants for their ages, you can use Qualtrics to ensure that participants type in their names as numbers without any spaces before or after. For a question with “yes” or “no” as the only two response options, use a multiple-choice question rather than a written response. Participants could write “Yes”, “y”, or “Y”, and you would have to make sure that you recognize all of those as “yes”.

For an observational study, develop a system to make it easier to record data. Returning to the recycling example, you wouldn’t want to come to the cafeteria with a blank sheet of paper to take notes on. Instead, you could create a spreadsheet with three headers - the location, time, and recycling status. This spreadsheet will keep everything for later analysis.

Can I Explore Quantitative Research Topics as a High School Student?

It can be challenging to do research as a high school student. You may not have the resources to compensate participants for taking part in your study. However, one option is to analyze a free, publicly available dataset about your topic. For example, one of my mentees was interested in researching factors that predicted better or worse mental health outcomes. We eventually decided to use a dataset from the Substance Abuse and Mental Health Services Administration (SAMHSA), which conducts a national survey on mental health every year. My mentee used computer science and machine learning to analyze the dataset. The results were quite interesting - I don’t want to give anything away, but when it’s published I’ll link it here! 

There are a few considerations to keep in mind when choosing your dataset. Make sure to get the data from a source that you trust. The very helpful Polygence staff gave my mentee and me several options. The American Psychological Association has a list of reputable sources you can check out. You should also consider how long it will take to access the data. Do you need to get permission to use it or can you simply download it from the website? How recently was the data collected? How diverse are the participants? How clean is the data? Since you didn’t collect the data yourself, it likely won’t match up perfectly with what you had in mind. With perseverance, you can find something that will work!

Another option to do quantitative research as a high school student is scraping data from websites. Many websites have APIs that you can use to collect data from their websites. For example, my mentee scraped posts from Reddit and ran quantitative and qualitative analyses on them. (This work has been published!) Data scraping does require a bit of programming, but just like with observational research, a pro of this approach is that the participants don’t know that they are in a study. For this strategy, identify what website you want to scrape (e.d., Reddit) and simply do a Google search for “Reddit API”. If the website has an API, you’re in business! If it doesn’t, you will need to scrape a different website. It’s really difficult to scrape if there isn’t an API. 

Conclusion

In this article, you learned about data collection in psychology . Before collecting your data, you need to get approval to run your study if you want to write a research paper. After you’ve collected your data, you can work on outlining the research paper and then eventually writing the paper.

Related Content:

• Psychology Research and Passion Project Ideas for High School Students 

• Top 20 Psychology Summer Research Opportunities for High School Students

• Top Psychology Competitions For High School and Middle School Students

• Top 4 Psychology Journals for High School Students to Publish In