University of Leicester
Browse

On the Power of Special-purpose GPT Models to Create and Evaluate New Poetry in Old Styles

Download (118.32 kB)
conference contribution
posted on 2023-11-06, 11:53 authored by Piotr Sawicki, Marek Grzes, Luis Fabricio Góes, Dan Brown, Max Peeperkorn, Aisha Khatun, Simona Paraskevopoulou

ThisstudyinvestigatesthepossibilityofusingGPT-3 modelstogeneratehigh-qualitypoemsinaspecificauthor’sstyle,throughfine-tuningondatasetsofpoems accompaniedbytheirmetadataandautomaticallygeneratedsummaries.Ourexperimentsshowthatadataset ofonly300poemsissufficienttogeneratenewpoems inthestyleofaspecificauthor.Theevaluationwasdone throughGPT-3modelsfine-tunedforbinaryclassificationofGPT-3outputsagainsttheworksoftheoriginal author.ToestablishtheaccuracyofGPT-3-basedbinaryclassifiers,wefirsttestedthemonavarietyoftexts andarangeofclasses,andfoundthattheirpredictive accuracyis99%onaverage.UsingthismethodforpoetryevaluationshowedthattheGPT-3generatedpoems wereindistinguishablefromtheoriginalworksofWalt WhitmanandRudyardKiplinginanaverageof30% and21%ofthecases,respectively.Thissuggeststhat GPT-3canbeausefultoolinassistingauthors,while furtherresearchisneededtoturnitintoanindependentcreator.Additionally,theworkflowusedinthis studycanbeappliedtoothertypesoftextandprovides awayofusingGPT-3modelsforgeneratingnewcontentfromuser-providedsummaries,whenpromptengineeringaloneisinsufficient.

History

Author affiliation

School of Computing and Mathematical Sciences, University of Leicester

Source

International Conference on Computational Creativity

Version

  • AM (Accepted Manuscript)

Published in

International Conference on Computational Creativity

Copyright date

2023

Available date

2023-11-06

Temporal coverage: start date

2023-06-19

Temporal coverage: end date

2023-06-23

Language

en

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC