The ESA Gaming Report has Serious Problems

The ESA Gaming Report has Serious Problems


The entertainment software association – the ESA – came out with their annual report on Wednesday April 19. Essential Facts about the Computer and Video Game Industry, Sales, Demographic and Usage data is supposed to provide us with information about the American public like how many of us play video games for example but as a former serve a professional myself I immediately noticed a problem that calls into question the trustworthiness of the survey let’s talk about one of my favorite topics data in this game objective feature video first a little background the annual ESA survey is widely reported and cited not only does the media jump all over its well presented infographics and easy to access data but academia cites the report all the time as well whenever people refer to half of gamers being women they’re almost always referring to ESA statistics whenever you see an article on the survey it almost always presents the data with no context and without question that’s not what I do here at game objectives so I decided to take a closer look on the second page of the report I immediately noticed a problem take a look and see the ESA is reporting that ninety-seven percent of american households have a personal computer ninety-seven percent there’s no way that’s accurate the United States is wealthy yes but there’s a lot of poverty as well so I look for any possible collaborative data and I came upon Pew Research q is a respected research back tank quote unquote and they have indeed looked at computer ownership and guess what their numbers don’t match with the ESA since 2008 pew has reported between seventy-three percent and seventy-eight percent of americans own a computer that’s a good twenty percent difference with the ESA survey which is certainly outside the margin of error that leaves us with three possibilities here the ESA survey is wrong the Pew survey is wrong or both are wrong the next step in their looking at what’s going on here is to look at the methodology how the survey was conducted if you know how a survey is conducted then you can repeat the process and look at changes over time that’s why you see the Pew survey having a small range difference over and that’s what you would expect with computer ownership since it’s largely a stable product in a stable market so here we have the Pew methodology and we have five or six paragraphs talking about how it’s a phone survey done with both cell phones and landlines from a national sample and they talk about how it’s weighted and that this is how it should be done then we have the esa survey the only methodology that they will go on the record with is what they presented in their press release essential facts about the computer and video game industry is the most in-depth and targeted annual survey of its kind it sews media CT conducted the research for esa and surveyed more than 4,000 american households that doesn’t help at all I also talked to pew research and other industry insiders but unfortunately it’s very difficult to get someone to go on the record to criticize someone else’s survey so where does that leave us well we have pew telling us around three-quarters of Americans have a personal computer and they’re telling us exactly how they came to that number then we have the ESA telling us very nearly every single household in the United States has a personal computer and they won’t tell us how they arrived at that figure to put it plainly I don’t trust the ESA numbers if ninety-seven percent of their respondents on a personal computer then their sample is biased in favor of people who are more technologically inclined if that’s the case then it’s likely that these people are more likely to be gamers which means that the ESA estimate of the number of gamers may be too high this calls into question whether or not this survey accurately represents Americans people of lower socioeconomic status are less likely to own a computer are they fairly represented in this survey do they account for over females being more likely to opt into a survey and younger males being less likely we don’t know because they won’t provide their methodology and that’s the problem with a lack of transparency if you can’t trust one number in the survey how do you trust the rest of the numbers the fact is their numbers are screwy and their transparency is non-existent I have to question whether the ESA survey is actually helping the game industry or if it’s just passing along biased information now there are a couple more problems with the sir I need to address because I know my audience has talked about at least one of them first you’ll notice in their infographic that they list virtual reality devices at eleven percent market share that is of course somewhat misleading the vast majority of those devices are cell phones and I notice that pc gamer actually picked up on that and questioned the esa report well done pc gamer good job the second issue is the always fun talk of gender politics in the esa report they use their gender statistics and then unusual way they say quote women aged 18 and older represent a significant greater portion of the game-playing population than boys under age 18 it looks like they’re trying to contradict the notion that games are for young boys but this is an oddly presented statistic I mean if you think about it is it really worth mentioning that there are more women between the ages of 18 and 118 playing games then boys aged 0 to 18 playing game those are certainly two problems with the survey but the thing is if the survey is not representative and the sample has bias issues then all of that data is called into question anyway that’s how important proper sampling is it’s the foundation of everything so the moral of the story here is that this survey is unfortunately not a very useful tool for understanding the American Gaming population and that’s too bad the ESA needs to get good and do better that’s what I think what do you think as always if you liked this video hit the thumbs up and if you don’t like it hit the thumbs down and maybe leave me a nasty comment if you’d like to see more videos like this please make sure to subscribe to the channel and maybe consider donating to our patreon I’m sure you’ve heard of the problems with YouTube advertising lately so your donations are more important than ever you can find links to support game objective as well as links to all the sources that used in the video in the description below I’m Brad Glasgow thanks for watching you

Leave a Response

Your email address will not be published. Required fields are marked *

8 thoughts on “The ESA Gaming Report has Serious Problems”

  • It almost looks to me like they fucked up the infographic. Like it sposed to read that of the 67% of US households owning a device for playing video games (the stat above the graph); 97% of those homes (so, 97% of the 67% total) have a PC. Maybe its just incredibly sloppy wording?

  • yellin' at a toilet
    the people who are wrong aren't wrong by accident
    they're wrong because they want to be wrong, and to push further wrongness

  • Thank you for this video, I'm currently writing a research paper around sexualisation of females in video games, and you'd think that the statistics on female games would be a relatively easy thing to come across considering the massive scale of the video game industry. Unfortunately all I can write in my paper, in reference to these numbers, is that there are no figure to show them. I can now, at least cite this video for an explanation as to why there are no accurate figures.