Earlier this year, Matthew Groh instructed ChatGPT to create images of a man playing guitar with Bruce Springsteen and tennis with Rafael Nadal and Roger Federer.
At first glance, the two pictures looked pretty convincing. But Groh, an assistant professor of management and organizations at Kellogg, spotted a couple of tell-tale signs that they were fake: the guitar fret inlays weren’t positioned normally, and the tennis rackets’ strings looked oddly warped.
As an expert in identifying AI-generated images, Groh has developed the skills to spot these kinds of discrepancies with relative ease. But most people don’t have that level of expertise—and in these early days of generative AI, it’s not yet clear how easily the skill can be learned.
Groh and his colleagues set out to explore whether a brief training session could help people improve their ability to tell apart real and AI-generated images. They focused their attention on professionals who need to make this determination in very high-stakes situations: U.S. government intelligence analysts.
To a group of intelligence analysts, Groh delivered a single half-hour training session that pointed out patterns in real versus AI-generated images. And the results were reassuring. The training helped improve the analysts’ overall accuracy significantly.
“It’s possible to get better,” Groh says. “We’re not doomed not to be able to tell real from fake.”
Room for improvement
The latest AI models allow practically anyone to create a fairly realistic-looking picture by feeding them a text prompt. Because the models have improved so much over such a short period of time, “the conventional wisdom is that people are not good at detecting AI-generated images,” Groh says.
In a recent study, however, Groh and his colleagues tested how well ordinary people could distinguish real from fake images and found they correctly classified images about three-quarters of the time. And the longer they studied a picture, the more likely they were to spot the fakes. The participants were “far from perfect,” he says, but “also far from just random guessing.”
Still, there’s room for improvement, especially for professionals like intelligence analysts who handle sensitive, high-stakes work on a daily basis. If an analyst fails to correctly identify AI-generated images, severe consequences could follow. For instance, officials might authorize a drone strike without just cause if they fail to correctly screen a fake image that suggests a terrorist is in a particular location.
So, the researchers developed a 30-minute training session that points out patterns and tell-tale signs in 7 real and 50 AI-generated images.
For example, a common giveaway for AI images is that they often look overly perfect and cinematographic, featuring people with symmetrical, classically beautiful features—something that “looks too good to be true,” he says. Other common signs include waxy or glossy skin, missing teeth, or implausibilities such as a doctor carrying a stethoscope with the two earpieces merged into a loop.
Fast results
For their study, Groh and his colleagues worked with 32 intelligence analysts from various U.S. agencies and wrote daily briefs for the president and other senior White House officials.
The team asked the analysts to classify 40 images as real or AI-generated on a customized web interface. They could also write down their reason for each decision. The collection of real and fake images included portraits, full-body shots, posed group shots, and candids. And the images were paired based on content. For instance, the image set might have included both a real photograph of a female astronaut and an AI-generated one (though participants were shown each image one at a time).
As a whole, the analysts correctly identified 73 percent of the images, close to the level of accuracy that Groh’s team had found in their previous study of regular people. These AI models are so new that “even the people who professionally deal with this kind of content are not necessarily better,” he says.
Then Groh presented the 30-minute training slide deck about how to distinguish real from AI-generated images to the analysts, after which the analysts repeated the online classification exercise on another set of 40 real and AI-generated images.
Before the training, the analysts’ accuracy was about the same as that of the general public. But after the training, the analysts’ accuracy increased to 82 percent—a bump of 9 percentage points. And the participants’ ability to accurately classify both images in a pair, such as the real and fake images of an astronaut, improved by 11 percent.
Spotting what’s real
The analysts’ improvement was mostly driven by a 14 percent increase in accuracy at identifying real images. Or put another way, the training helped reduce the analysts’ false positives.
“We helped them better identify real images as real, which is actually the hardest thing to do,” Groh says. With an AI-generated image, you can be confident it’s fake once you spot the artifact, but “a real image is something that has the absence of artifacts.”
This skill is critical, because without it, people may favor believing that images are fake by default. In some of the previous trainings that other researchers have tried, for example, participants got better at identifying AI images but also became more likely to say that real photographs were created by AI.
“That’s kind of problematic,” Groh says. “Now you’re just leading everyone to think everything’s fake.”
What’s more, the analysts gave more-detailed reasons for their choices the second time they took the test than the first time, showing that they had a better understanding of common signs. Before the training, their comments tended to be vague: naming a body part (“hair” or “fingers”) or saying that “something looks off.” Afterward, they were more likely to pinpoint details that reflected key patterns, such as doors opening into empty space or dirt appearing too uniform.
“[After the training,] people knew to look for something like that,” Groh says.
Seeing the limitations
Beyond national security, the ability to distinguish images accurately is critical for everything from spotting fake news to making sound business decisions. “Trust is fundamental to business,” Groh says. For instance, investors might field pitches from budding entrepreneurs who provide photos of their products or backstory.
“The moment that you can’t tell the difference at all is the moment you can no longer trust any visual medium,” he says.
In an ideal world, policymakers would be putting up guardrails around AI image generation, such as requirements for when and how creators must disclose that images are fake. “But that policymaking isn’t here yet,” Groh says. “In the meantime, people need to build this skill themselves.”
People can boost their accuracy not only through prepared trainings but also by experimenting with AI models on their own. The more people do so, the more they’ll notice the signs that tend to show up in AI-generated images.
“The biggest thing is playing around with these tools themselves,” Groh says. “You start to see the limitations.”
