Bits of Grass: Does GPT Already Know How to Write like Whitman?

This study examines the ability of GPT-3.5,  GPT-3.5-turbo(ChatGPT) and GPT-4 models to generate poems in the styleof  specific  authors  using  zero-shot  and  many-shot  prompts(which use the maximum context length of 8192 tokens). Weassess the performance of models that are not fine-tuned forgenerating poetry in the style of specific authors,  via auto-mated  evaluation.   Our  findings  indicate  that  without  fine-tuning,  even  when  provided  with  the  maximum  number  of17 poem examples (8192 tokens) in the prompt, these modelsdo not generate poetry in the desired style.