On the Power of Special-purpose GPT Models to Create and Evaluate New Poetry in Old Styles
ThisstudyinvestigatesthepossibilityofusingGPT-3 modelstogeneratehigh-qualitypoemsinaspecificauthor’sstyle,throughfine-tuningondatasetsofpoems accompaniedbytheirmetadataandautomaticallygeneratedsummaries.Ourexperimentsshowthatadataset ofonly300poemsissufficienttogeneratenewpoems inthestyleofaspecificauthor.Theevaluationwasdone throughGPT-3modelsfine-tunedforbinaryclassificationofGPT-3outputsagainsttheworksoftheoriginal author.ToestablishtheaccuracyofGPT-3-basedbinaryclassifiers,wefirsttestedthemonavarietyoftexts andarangeofclasses,andfoundthattheirpredictive accuracyis99%onaverage.UsingthismethodforpoetryevaluationshowedthattheGPT-3generatedpoems wereindistinguishablefromtheoriginalworksofWalt WhitmanandRudyardKiplinginanaverageof30% and21%ofthecases,respectively.Thissuggeststhat GPT-3canbeausefultoolinassistingauthors,while furtherresearchisneededtoturnitintoanindependentcreator.Additionally,theworkflowusedinthis studycanbeappliedtoothertypesoftextandprovides awayofusingGPT-3modelsforgeneratingnewcontentfromuser-providedsummaries,whenpromptengineeringaloneisinsufficient.
History
Author affiliation
School of Computing and Mathematical Sciences, University of LeicesterSource
International Conference on Computational CreativityVersion
- AM (Accepted Manuscript)
Published in
International Conference on Computational CreativityCopyright date
2023Available date
2023-11-06Temporal coverage: start date
2023-06-19Temporal coverage: end date
2023-06-23Language
enUsage metrics
