Download pdf
LLM-Based Web Generation Quality Assessment
- 1 Lanzhou University, Chengguan District, Lanzhou City, Gansu Province, China
- 2 Lanzhou University, Chengguan District, Lanzhou City, Gansu Province, China
* Author to whom correspondence should be addressed.
Abstract
Large Language Models (LLMs) have demonstrated powerful capabilities in the field of code generation, with a deep understanding of the semantics and functionality of code. Building websites is one of the most important tasks in software development, as it utilizes rich frontend displays and backend processing to achieve various service functions. It is one of the most widely used interactive software models. Although there have been some efforts in Web website generation, these efforts have been limited to the automation of generating Web pages. The advent of LLMs provides a new approach to Web site generation tasks. However, there is currently a lack of comprehensive evaluation of the generation performance of LLMs in this context, making it difficult to optimize and improve the generated results in a targeted manner. To address this issue, this paper conducts a multi-angle investigation and analysis of the performance of LLMs in Web site generation tasks. Firstly, Web generation requirements are collected, and effective prompt engineering is designed. These prompts are then input into different LLMs to initiate the self-iteration process. Next, the generated code is fed back into the LLM for security self-iteration, where the model performs vulnerability detection and repair on the code it has generated. The security-enhanced code is subsequently subjected to manual review, where it is evaluated using predefined quantitative metrics to generate indicator values. Finally, through testing, the quality, security, and code defects of the generated front-end and back-end Web code across different LLMs are analyzed, providing a comprehensive evaluation of the generation results. The experiments demonstrate that LLM systems perform well in completing and implementing the functions and layouts of pages in prompts for Web generation tasks, but there remains room for improvement in the security of the Web code.
Keywords
LLM, code generation, Web security, defect detection, software development
[1]. Jiang, J., Wang, F., Shen, J., et al. (2024). A survey on large language models for code generation. arXiv preprint arXiv:2406.00515.
[2]. Sharma, M., & Angmo, R. (2014). Web-based automation testing and tools. International Journal of Computer Science and Information Technologies, 5(1), 908-912.
[3]. Kaluarachchi, T., & Wickramasinghe, M. (2023). A systematic literature review on automatic website generation. Journal of Computer Languages, 75, 101202.
[4]. Lin, F., & Kim, D. J. (2024). When LLM-based code generation meets the software development process. arXiv preprint arXiv:2403.15852.
[5]. Mu, F., et al. (2023). ClarifyGPT: Empowering LLM-based code generation with intention clarification. arXiv preprint arXiv:2310.10996.
[6]. Ugare, S., et al. (2024). Improving LLM code generation with grammar augmentation. arXiv preprint arXiv:2403.01632.
[7]. Huang, D., et al. (2023). Bias assessment and mitigation in LLM-based code generation. arXiv preprint arXiv:2309.14345.
[8]. Liu, J., Xia, C. S., Wang, Y., et al. (2024). Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36.
[9]. Gu, Q. (2023). LLM-based code generation method for Golang compiler testing. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023) (pp. 2201-2203). Association for Computing Machinery. https://doi.org/10.1145/3611643.3617850
[10]. Nam, D., Macvean, A., Hellendoorn, V., et al. (2024). Using an LLM to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (pp. 1-13).
[11]. Paolone, G., Marinelli, M., Paesani, R., et al. (2020). Automatic code generation of MVC web applications. Computers, 9(3), 56.
[12]. Stocco, A., Leotta, M., Ricca, F., et al. (2017). APOGEN: Automatic page object generator for web testing. Software Quality Journal, 25(3), 1007-1039.
[13]. Calò, T., & De Russis, L. (2023). Leveraging large language models for end-user website generation. In International Symposium on End User Development (pp. 52-61). Springer Nature Switzerland.
[14]. Tóth R., Bisztray T., Erdődi L. (2024). LLMs in web development: Evaluating LLM-generated PHP code unveiling vulnerabilities and limitations. In International Conference on Computer Safety, Reliability, and Security (pp. 425-437). Springer Nature Switzerland.
[15]. Gur, I., Nachum, O., Miao, Y., et al. (2022). Understanding HTML with large language models. arXiv preprint arXiv:2210.03945.
[16]. López-Gil, J. M., & Pereira, J. (2024). Turning manual web accessibility success criteria into automatic: An LLM-based approach. Universal Access in the Information Society, 1-16.
Cite this article
Gong,Y.;Wang,X. (2025). LLM-Based Web Generation Quality Assessment. Theoretical and Natural Science,100,60-68.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 3rd International Conference on Mathematical Physics and Computational Simulation
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).