How to collect the most accurate data possible?
Question raised by Brenda Khor after watching the video about my critique on the article, "Questioning the Selective Exposure to News: Understanding the Impact of Social Networks on Political News Consumption": You mentioned that another research method the authors (2017) could use to determine if social media makes an impact on public opinion was to explore people’s social media posts. If using this method, what kinds of general results do you think this can yield, differing from what the authors found when doing surveys and focus groups?
I am currently working on Dr. Yoo Kyung Chang's research project about people's mobile news consumption behavior. The main objective is to validate whether our research method - to record people's mobile screen while they read the news - help with understanding people's mobile news consumption behavior. At first, there seemed to be no problem, really. When we discussed the idea in class, I was so confident our research method was strong and solid. I mean how can observing people's behavior not offer any insight, right? However, after running two experiments with people - that is, asking people to record their phone screens while they read mobile news, I have started to develop some doubts about the methodology.
When I meet with research participants for data collection, the process usually goes as follows: I first give them a brief intro of the research and ask if they have an interest in participating; if yes, I move on to provide more context and ask for their consent; once they agree, I start with initial survey questions, such as 'Where do you usually find news?' and 'Where are your go-to news sources?' After all these 'pre' stages are over, here comes the main part - we ask people to read the news on their phone for about 15 minutes and record the screen. Once they are done, they share the file with us via Google Drive. We close the entire process with closing remarks, thanking them for their time and effort.
[The recording of my own phone screen as I am reading my news]
The problem arises when I ask people to start reading the news. Up until then, when I explain the project and ask several opening questions, people are very interested and seem comfortable talking to me. But once I ask them to pull out their phone, read the news, record the screen within the given time, I can sense a hint of discomfort from them. 'You are suddenly asking me to read the news and record the screen?' But people do it anyway because they have already made their commitments at this point. They reluctantly pull up their mobile news apps, such as The New York Times or CNN or whatever that would be, but hey, those are not the sources you mentioned as your go-to news sources in our initial survey questions? Did not you just mention in our initial chat that you mostly catch up with news through Apple news alert, which pops up while you look at your phone screen and engage in other activities? Wait, so you open your news apps, because you are directed to read the news?
This 'unnatural' setting is what concerns me. Our goal of the research was to collect data that can offer insights about people's mobile news consumption behavior. However, during the experiment, people are forced in some ways. Their behavior, which is captured as our data, does not necessarily align with how they usually access the news in their daily lives. A lot of people said they get news updates from news app breaking news alert messages or from other media sources. They also mentioned they do not necessarily lookup mobile news voluntarily in their spare time. However, because we asked people to read the news, all of a sudden, people start opening the news apps, which they did not mention as their primary source of current events.
[Screenshot of my own phone screen bombarded with news app notifications. I also get news updates mainly from these unexpected updates, which can be considered as 'accidental exposure theory.']
The authors of "Questioning the Selective Exposure to News: Understanding the Impact of Social Networks on Political News Consumption" used surveys and focus groups to find data about people's news reading patterns on social media. I suggested the authors could have strengthened their research by examining each person's social media and the type of news people share. Doing so would provide more insights regarding people's behavior in a more natural setting. It would also allow the researchers to compare how do they actually behave and how they claim or think themselves to behave. This data could be really meaningful, because it might suggest the degree to which people actually read or not read the kind of news that are different from their political views. Anyways, data collection, or more like finding the right data will always be a struggle.
Hi Joanna, thank you for sharing your research experiences with us. I find the research topic very interesting. The question seems very intuitive to me at the start as well, but the hard part is to prove it. We, as researchers, want honest and spontaneous answers from the participants. However, when they aware of the research context, participants make unnatural judgments and decisions. I was wondering whether there is a better way to get the data we want? Maybe we can do a news quiz of the current event and record the participant's screen. In this way, we can see their "go-to" news application.
ReplyDeleteMoreover, because they are focusing on answering the questions, it might help with distracting their attention from "I'm being asked to read the news". Data collection is so crucial to research. It needs a thorough design and precaution of the possible problem, but you can always learn from it.