BOULDER, Colo. — Researchers in Colorado and Pennsylvania have developed a series of artificial intelligence tools that can scan through open-ended essays in college applications.
The study was published in October in the journal “Science Advances” and was developed by researchers from the University of Colorado Boulder and University of Pennsylvania.
“We advocate for transparency, where the AI provides explanations and clearly communicates when it is less confident in its decisions,” said Sidney D’Mello, professor in the Institute of Cognitive Science and Department of Computer Science at CU Boulder. “People can then decide for themselves how much they should trust it.”
In the study, D'Mello and his colleagues tapped into more than 300,000 anonymous applications that prospective students submitted between 2008 and 2009. Students who submitted the application essays wrote about their extracurricular activities or work experiences.
"We find that because those essays really reflect reality and lived experience, they're a really good avenue to convey what those kids' experiences are," D'Mello said.
Professionals scored the essays for evidence of seven traits that colleges might want to see in incoming freshmen. Traits included intrinsic motivation and what the researchers call "prosocial purpose," or the willingness to help others.
The insights were then fed into a series of AI platforms to train them to identify evidence of personal qualities going beyond simple word spotting.
Afterward, when the platforms read new essays, their results closely lined up with the judgment of the human readers.
"It's really blending what computers do best, which is finding patterns and large volumes of data, with really what humans do best, which is finding the best in each other. The computer is always guided by humans," D'Mello said.
Students who wrote essays showing evidence of leadership were more likely to graduate college in six years than those who didn't.
“I was amazed by how a 150-word open-ended response contained sufficient information on whether a student would graduate college six years later,” D’Mello said. “Language is really an amazing thing.”
SUGGESTED VIDEOS: Colorado Buffaloes