Human Teams Produced More Diverse Ideas Than ChatGPT in a 208-Person Study

Generative AI can produce a long list of suggestions quickly, but a recent experiment suggests that speed and volume don’t guarantee creative range. In a study of 208 people, participants working with another person came up with more creative and varied ideas than participants collaborating with ChatGPT. The result points to a practical distinction: AI can help generate material, while human interaction may help teams explore less familiar ground.

The experiment, conducted by cognitive scientists who study creativity, used the alternate uses task. In this exercise, people suggest different ways to use an everyday object, such as a brick or a book. Researchers can assess the responses by looking at how many ideas participants offer and how original those ideas are. The task measures one part of creativity, known as divergent thinking; it doesn’t capture every aspect of creative work.

Longer responses, fewer distinct ideas

Participants were randomly assigned to work with ChatGPT or another person. Those working with ChatGPT wrote longer responses than people in pairs taking turns. But the human pairs generated more creative ideas, and their suggestions were more distinct from one another.

That difference matters because a long answer can contain repeated or unnecessary material. A separate analysis of 12 chatbots, tested on thousands of factual questions, found that their answers were often much longer than human responses. The answer itself made up about 42% of the text, with the remainder largely consisting of repetition, extra background or detail that wandered from the question. This finding concerns factual answers, rather than the brainstorming experiment, but it illustrates why response length alone is a poor measure of usefulness.

AI can still produce an original-sounding suggestion. In the experiment, one example was using a book as a sound dampener. The concern was that ChatGPT offered this same idea across multiple sessions. A suggestion can be unusual on its own yet become less distinctive when it recurs among many people’s results.

Other research has reported a similar pattern in different creative tasks. Repeated AI-assisted image generation has been associated with generic-looking results, described by researchers as “visual elevator music.” And although AI support may help an individual produce a more creative story than their earlier work, writing produced with AI across a group of users has tended to be more similar, according to a separate study.

Why suggestions can start to sound alike

Large language models generate text by predicting likely next words based on patterns learned from large collections of writing. When different systems learn from similar material and use similar methods, their phrasing and suggestions can share recognizable patterns. Research comparing systems such as ChatGPT, Claude and Gemini has found similarities in their cadence, word choice and sentence structure, especially when set beside the work of different human writers.

That doesn’t mean every AI suggestion is unoriginal. It means a suggestion that seems fresh in isolation may be common across many AI-generated responses. As more people draw on similar systems, their ideas and writing could become more alike. The experiment’s finding about repeated suggestions supports that concern, though it doesn’t establish how often this happens in everyday workplaces or across every kind of creative project.

Human collaboration may widen the search over time

The gap between human and chatbot collaboration also appeared to grow as participants continued working together. Researchers assessed the distance between concepts, a measure of how different ideas are from one another. Over time, human collaborators’ ideas spread farther apart in meaning than ideas generated with the chatbot. The authors interpret this as people exploring a broader range of possibilities through continued interaction.

That pattern could matter for projects that take months or years, where a team needs to keep developing and revising an idea. Research cited by the authors estimates that developing a new product takes an average of 27 months from initial idea to finished product, and that more innovative products can take up to 54 months. The experiment didn’t study product development directly, so it can’t show how human or AI collaboration affects the success of a real launch. It does raise a useful question: across a long project, does a team keep finding new directions, or do its ideas gradually cluster around familiar ones?

AI personas add variety, with limits

One proposed way to prompt more diverse responses is to ask a chatbot to write from a specified persona, such as a college professor or a construction company founder. Research on persona prompting found that this approach increased the range of ChatGPT’s ideas. But another study found that the overall variety still fell short of ideas from human writers. Other work suggests that chatbots are limited simulators of human personality, so assigning a persona may not reproduce the perspective of a person with their own experiences.

My read is that these results support using AI as one source of input while keeping human conversation in the creative process. A colleague can bring personal knowledge and unexpected associations that a model’s typical response may not supply. For an idea that matters, discussing it with another person before turning to a chatbot may help widen the starting point; AI can then offer additional material to consider.

The evidence here comes from a specific brainstorming task with 208 participants, not a direct test of all creative work. Still, the results suggest a clear takeaway for teams: a fast, lengthy response can be useful, but sustained exchange with people may produce ideas that are more varied and feel more meaningful to the collaborators. People may take longer to respond. Their perspectives can give a project somewhere new to go.

Ben Affleck and Matt Damon co-wrote the screenplay for “Good Will Hunting,” which won the 1998 Academy Award for best original screenplay.

This article was republished from The Conversation. Read the original article.

The post appeared first on SingularityHub.