1. Introduction

Nowadays generative Artificial Intelligence(AI) has been widely used in every part of people’s daily lives, whether it’s academic research area or entertainment area, it can help people to complete tasks efficiently. In this age of diversity, with the development of artificial intelligence increasingly mature, new media is an emerging field that can keep pace with AI, and its development prospect is also bright. Generative AI can be widely used in network new media, such as virtual anchors, personalized recommendation data analysis, AI creates network new media copy and so on. When many people post photos and videos on the new media, they choose to use AI to generate copy to attract more attention. However, the copy generated by different AI tools has differences, these differences are directly related to whether AI can promote the development of new media on the internet.

In addition, paperwork plays a very important role in social media. The main task of paperwork is to deliver simple and creative language. Besides, paperwork also delivers a brand message to motivate the actions of the audience [1]. In writing marketing copy, emotional appeal and brand identity are key points. Meanwhile, creators need to catch the needs of audiences and create compelling content [2]. At the same time, the special of social media copywriting is interactive quality and flexible form. For example, fans have interaction can produce" The Butterfly Effect" and it can enhance communication effectiveness [3].

Thus, this study not only aims at reveal and understand the performance of different generative AI tools in marketing copy creation, but also to delve into underlying algorithms and language models that drive the output of these tools. This not only means to help fine-tune AI for specific writing styles or marketing needs in the future but also to deeply find out AI’s application potential in new media and the feasibility of human-AI collaboration in this field. Through this research the authors are going to verify: first can different AI software generate the same network new media copywriting to show emotional empathy and keep up with what's a hot and important element; second, this copywriting has differences in language style, such as exaggeration, humor, irony, colloquial, etc; third which type of AI software have the best performance in copywriting generation. The authors expect that this study will not only provide key insights for future improvements and development of Al, but also provide effective support for marketers and content creators.

2. Literature Review

2.1. The Role of AI As a Generative Tool and Its Application in New Media

Generative AI plays an important part of role in the production of content, social media art and AI technology mix in particular. AI through data analysis and machine learning and other Technologies can provide social media art with strong innovation support and assistance [4]. In short video creation field, AI technology not only optimize video editing and video creation, but also it can promote the viewing experience of the audience and the communication effect of content [5]. In addition, AI achieves remarkable evolve in the dissemination of news and digital restoration of movies and other field [6].

2.2. Status Quo of We-media and AI or Short Videos

In we media content creation, AI writing techniques gradually become strong auxiliary tools gradually. AI can provide more precise vocabulary recommendations and correction functions. AI helps we media creators push large-scale content production, and it also can optimize based on audience needs to enhance content dissemination effectiveness and consumer engagement [7]. In the short video field, copywriting is a core factor, it through accurate planning and writing satisfies different users' needs [8]. However, there will still be quality differences in the content AI generates, so AI not can satisfy high-quality content output requirements [9].

2.3. The Application and Impact of AI and Copywriting

AI plays an important role in copywriting create. Through generative AI-driven storytelling technology, brands can create more lively and attractive storytelling copywriting, and satisfy consumers' personal needs [10]. AI’s large language model(LLMS) and natural language generate(NLG) beyond simple copyrighting generation, show stronger human-computer interaction ability, especially in new media industry copyrighting generation, this further upgrades the accuracy and attention of the copyright [11]. Even so, the copyright creation and quality generated by AI are different from human, especially on copyright which has a high level of creativity and personal expression [12]. Therefore, AI is a powerful tool for copywriters, which can improve efficiency and quality, but creators need to keep it unique, and avoid relying on AI completely [1].

2.4. Future Development Trends of AI and People in the New Media Industry

Along with improvements in AI technology, the new media industry is facing more and more challenges and opportunities. Generative AI software provides copywriting generation and efficient promotion for new media companies, at the same time raises communication of intellectual property rights and copywriting creation [9]. In the future, AI and human beings integration of creation is becoming increasingly close, it is expected to play a greater role in information dissemination, content production and data analysis. But human creativity and judgement cannot be replaced [11, 13].

3. Methodology

3.1. Preparation of Visual Stimuli

The authors created a collection of five images from a restaurant visit (all captured by the authors) in order to thoroughly examine the potential of generative AI techniques in producing marketing text. These pictures show the storefront, the interior design, and the food from a well-known restaurant. The photo is popular and attracts widespread public interest, making it an ideal subject for testing AI-generated marketing copywriting. These visual cues will act as a unified basis for various AI techniques, guaranteeing that the text outputs produced by them are equivalent.

3.2. Selection of AI Tools

The authors selected four AI tools for this study: ChatGPT 4o, Image-To-Prompt-AI (Poe), Tiangong 3.0, and Doubao. These tools were chosen due to the fact that, although there is a wide range of AI tools available, technologies primarily focused on generating text from images are still in the early stages of development. However, these four AI tools represent highly advanced and mature options. As Table 1 shows Tiangong 3.0 and Doubao represent top domestic AI tools, while ChatGPT 4o and Image-To-Prompt-AI (Poe) were selected as representatives of frequently used foreign AI tools. This selection not only enhances the credibility of the comparison but also provides valuable insights into the performance differences of AI tools developed in different regions.

Table 1: Reasons for Selecting the Four Tools

3.3. Application of the BROKE Prompt Framework

To ensure consistency in the output of all AI tools, the authors have adopted the BROKE instruction framework: B stands for Background, R stands for Role, O stands for Objective, K stands for Key Results, E stands for Evolve.

This structure is uniformly applied across all four AI tools, with each selected photo serving as the primary input for each tool. Providing direction to AI tools so they may produce marketing material that complies with social media posting guidelines is the aim. By using this methodical process, the authors make sure that the text that is produced is both comparable and in line with the particular marketing goals associated with the images.

The reasons for selecting the BROKE framework are as follows:

1. Accuracy: It clearly defines the role of AI, ensuring that the generated text is more targeted and accurate, aligning with the intended marketing objectives.

2. Ease of debugging and maintenance: The framework's simple structure and clear logging make the debugging and maintenance process more intuitive and efficient, reducing the user's workload.

3.4. Collection of Data and Feedback

After the marketing copywriting is generated by each AI tool, the authors will design a related questionnaire to collect data and feedback from a broad audience. The authors mostly disseminate it online, and the instrument we use for this is the WeChat Mini Program Questionnaire Star. To help participants compare, the questionnaire has five photos with associated AI-generated sentences. The copywriting evaluation criteria are attached to each text and presented in the form of a Likert scale, facilitating data collection.

3.5. Standard for Assessing Copywriting Quality

The quality of the generated marketing copy will be evaluated based on the following seven criteria, see Table 2.

Table 2: Copywriting Quality Evaluation Criteria

4. Results

The majority of the 53 questionnaire respondents, who were drawn from throughout the nation, were from Jiangsu and Shaanxi provinces. They gave the marketing copywriting produced by four AI tools a lot of criticism.

Participants evaluated the marketing copywriting generated by each AI tool based on Table 2, with a rating scale ranging from 1 mark (Strongly Disagree) to 5 marks (Strongly Agree). The survey data are as follows:

Table 3: Quality Evaluation Data Table for Copywriting Generated by ChatGPT 40

According to Table 3, it can be seen that the overall average score of the copy generated by ChatGPT 4o is 4.11, indicating that the performance of the copy is good overall, especially in the category of "heartfelt and eye-catching titles," which received a higher average score of 4.32. Furthermore, ChatGPT 4o achieved outstanding results with average scores of 4.23 and 4.30 in the categories of "logical coherence and clear structure" and "material that is relatable to readers and close to life," respectively. However, ChatGPT 4o scored relatively low in the "presence of a twist" category, with a rating of only 3.81. Overall, the copy generated by ChatGPT 4o performed well in attracting readers and maintaining content structure, but it fell short in terms of language conciseness and crafting memorable phrases.

Table 4: Quality Evaluation Data Table for Copywriting Generated by Image-To-Prompt-AI (Poe)

With average scores of 4.15 and 4.21 in "simplicity and strength" and "logical coherence and clear structure," respectively, Poe's work excels in these areas. This suggests that Poe has an edge in upholding logical structure and is capable of conveying information in a clear and impactful way (see Table 4).

Table 5: Quality Evaluation Data Table for Copywriting Generated by Tiangong 3.0

Table 5 illustrates that Tiangong 3.0 performed exceptionally well in several rating categories, particularly in "a unique perspective with a clear and distinct topic" and "a logically coherent and clearly structured framework," with average scores of 4.28 and 4.30, respectively. This indicates that Tiangong 3.0 is capable of offering fresh perspectives and demonstrates strong organizational skills in content structure. Additionally, Tiangong 3.0 scored highly in the category of "a compelling and attention-grabbing headline," with an average score of 4.25. However, its lower score in the "presence of a twist" category reflects its shortcomings in creating unexpected elements.

Table 6: Quality Evaluation Data Table for Copywriting Generated by Doubao

Doubao received fairly even evaluations for his copywriting in a number of categories. With average scores of 4.13 and 4.08 in "a logically coherent and clearly structured framework" and "concise and impactful language," respectively, it did well in these categories. This suggests that Doubao can retain both expressiveness and logical coherence to some extent. Doubao, however, received the lowest score of 3.75 for the question "whether there is a twist," indicating a rather poor capacity to include unexpected components in the content (see Table 6).

To gain a more accurate understanding of the respondents' preferences for copywriting, a relevant question was included in the questionnaire. The statistical results are presented in Table 7.

Table 7: Participant Preference Data Analysis

A thorough examination shows that Tiangong 3.0 does remarkably well in all areas, but it excels most in original viewpoints and logical reasoning. Poe and ChatGPT 4o also show strong skills in content structure and conciseness, but they struggle with title appeal and surprising the reader. Doubao didn't excel in any one category, but it did perform consistently across the board. When it came to the final choices made by the participants, ChatGPT 4o came in first with 41.51%, followed by Tiangong 3.0 (20.75%), Image-To-Prompt-AI (18.87%), and Doubao (18.87%). This indicates that Tiangong 3.0 excels in content distinctiveness and structural coherence, whereas ChatGPT 4o's copywriting has a stronger overall appeal to audiences. Doubao and Poe each have advantages in particular domains. In the end, foreign AI tools such as ChatGPT 4o and Image-To-Prompt-AI (Poe) generally perform better than domestic AI tools such as Tiangong 3.0 and Doubao. But local AI programs are more adept at grasping the nuanced details of regional culture.

5. Conclusion

The authors evaluated the quality of numerous copywritten works created by social media users through a poll to see if they adhered to the requirements for excellent online new media copy. The results of the study show that although the copy produced by different AI tools typically satisfies the requirements for language structure and content logic, there are some content variations. For instance, ChatGPT 4o tends to generate longer copy, while the other three AI tools produce more appropriately sized content. However, ChatGPT 4o excelled in other key metrics, making it more favored by participants.

Research indicates that there are differences in the emotional resonance and presentation of key elements across copy generated by different AI software. ChatGPT 4o and Tiangong 3.0 exhibit superior emotional resonance and remain relevant to important elements and current trends when compared to other AI technologies. When it comes to language style, ChatGPT 4.0 seems verbose in its linguistic complexity and expression, whereas Tiangong 3.0 shines with its own viewpoint and logical coherence. Doubao demonstrated a balanced and conversational style. In the end, there was some discrepancy between the copy's popularity and its compliance with all evaluation criteria, according to participant input. All things considered, ChatGPT 4o was the most popular and performed the best overall.

Although the research has yielded some results, there are still certain limitations. Initially, there are restrictions on the evaluation criteria. The study's seven criteria were mostly derived from a synthesis of existing research and the authors' own personal experience, which may have compromised the assessment's thoroughness and accuracy in the absence of authorized standards. Furthermore, the questionnaire feedback's subjective textual judgments depend on the participants' individual tastes and levels of knowledge, which could provide biased findings. The uniqueness of the copy type is another drawback. This study did not cover additional copywriting scenarios; instead, it concentrated only on one particular sort of marketing copy (in the restaurant business). Therefore, it's possible that the conclusions won't hold true for all fields.

Generative AI will progressively permeate several industries in the new media era, especially the new media industry. In order to further optimize and reinvent content creation, future development will concentrate on the collaborative synergy between AI and human producers. This will combine the efficiency of AI with the creative powers of humans. Simultaneously, AI necessitates enhancements in content development and application, requiring continuous investigation and refinement across technological, legal, and ethical dimensions. This study provides an initial validation of AI's application in copywriting and lays a foundation for further in-depth research in related fields.

Authors Contribution

All the authors contributed equally and their names were listed in alphabetical order.