People don't distinguish ChatGPT poetry from the poetry of Shakespeare and Byron (study)

It seems that ChatGPT could be a future Nobel Prize laureate in literature.

A new study, published in the journal Scientific Reports, reveals that readers struggle to distinguish between poetry written by artificial intelligence and that composed by humans. The majority of participants in the experiment preferred the AI-generated works.

Researchers Brian Porter and Edward Machery from the Department of History and Philosophy of Science at the University of Pittsburgh (USA) tested the ability of 1,634 participants to differentiate between poems created by AI and those written by human poets.

Participants were randomly presented with ten poems: five authored by renowned poets such as William Shakespeare, Lord Byron, Emily Dickinson, and T.S. Eliot, and five poems generated by ChatGPT3.5 in the style of these poets. Participants were more likely to believe that the AI produced poems were written by a human, while the five pieces they found least resembling human work actually belonged to the real poets.

In a second experiment, another group of 696 participants evaluated the poems based on 14 characteristics, including quality, beauty, emotions, rhythm, and originality. The participants were randomly assigned to three groups, with one group informed that the poems were written by a human, another group told they were created by AI, and a third group given no information about authorship.

Participants who were aware of the AI authorship rated the poems lower on all criteria. Conversely, the group with no information about the origin of the works preferred the AI-generated poems.

The researchers attribute the results to the simplicity and accessibility of the AI's poetry compared to the works of classic poets. Readers mistakenly perceived this ease of understanding as a sign of human authorship.

Source: phys