1. Introduction

In today’s intelligent era, the development of artificial intelligence (AI) has become a critical focal point of international competition. The Fourteenth National People’s Congress emphasized the role of the fourth and fifth factors of production, with AI as the fifth factor playing a key role in promoting education and talent cultivation.

The quality of a curriculum underpins the quality of education, with educational goals realized through the curriculum. As a crucial component of curriculum instruction, AI textbooks for primary and secondary schools are vital tools for teachers and essential mediums for students to learn AI course content [1]. Currently, there is a variety of AI textbooks for primary and secondary schools in China, but their quality varies significantly. Therefore, a comparative analysis of these textbooks holds substantial theoretical and practical significance. This study utilizes content analysis and quantitative curriculum difficulty models to systematically compare 18 AI textbooks across different grade levels and versions. The aim is to provide insights and references for the development of AI textbooks and curricula in primary and secondary schools.

Different scholars have various perspectives on textbook analysis. For instance, Gao Lingbiao believes that textbook analysis should at least cover four aspects: design and compilation, teacher usage, student behavior patterns, and academic performance [2]. Pingel, through a “standard list for analysis,” studies textbooks by module, emphasizing both quantitative and qualitative methods in text analysis [3]. A professor from Sussex University proposed a model for textbook analysis and evaluation, following a sequential order of purpose, goals, content, pedagogy, and assessment, to evaluate and analyze different elements of textbooks for qualitative and quantitative research [4].

Focusing on the field of AI, there are currently few comparative studies on AI textbooks, and most are qualitative. Wang Dongli and others refined the dimensions of textbook analysis into five aspects: textbook positioning, core content, writing style, value orientation, and teaching activities, systematically analyzing 45 published AI textbooks for primary and secondary schools. Shi Wenjing and others used content analysis to analyze textbooks in three steps: identifying valid sentences, extracting keywords, and summarizing macro categories, then summarizing and concluding the findings based on textbook content, structure, writing principles, and student capability development [5]. This research differs from other studies by employing both quantitative and qualitative methods to more objectively compare and analyze AI textbooks for primary and secondary schools.

2. Research Design

2.1. Research Subjects

This study follows the principles of considering both the professionalism and accessibility of the textbooks. Through research, we identified the mainstream AI textbooks for primary and secondary schools available on the market. A total of 18 textbooks were selected, with basic information as shown in Table 1.

Table 1: Basic Information of AI Textbooks for Primary and Secondary Schools

2.2. Research Methods

2.2.1. Basis for Comparison

The “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” published in 2021 in China emphasizes that textbook writing should reflect scientific and forward-looking perspectives; the content of the textbooks should be moderately difficult and presented through project-based activities and specific cases; the arrangement of the textbook content should balance teaching modules for teachers and learning modules for students, and the teaching content should have a certain degree of openness and expansiveness. The “Standards for Information Technology Curriculum for Compulsory Education (2022 Edition)” mentions that the curriculum philosophy should reflect the correct direction of education in the digital age, construct logically connected curriculum structures, select curriculum content that emphasizes both scientific principles and practical applications, advocate authentic learning, and strengthen diverse evaluations oriented towards competency. The revised “Information Technology Curriculum Standards for General High Schools (2020 Edition)” states that the curriculum should adhere to the value of fostering virtue, set a curriculum structure that meets the diverse needs of students, choose curriculum content that reflects the times and foundational knowledge, cultivate a teaching and learning relationship centered on learning, and build an evaluation system based on core competencies in the discipline.

2.2.2. Content Analysis Method

The content analysis method is a rigorous, systematic approach researchers use to systematically analyze relevant research literature, journals, national documents, etc. In the field of comparative analysis of AI textbooks for primary and secondary schools, existing dimensions for textbook comparison are shown in Table 2.

Table 2: Analysis of Existing Dimensions for Comparing AI Textbooks for Primary and Secondary Schools

It can be seen that existing research often compares textbooks based on aspects such as teaching positioning, structure, and content. This study, building on existing research dimensions and considering the characteristics of the 18 selected textbooks, conducts a content analysis of primary and secondary school artificial intelligence textbooks from four perspectives: textbook arrangement logic, textbook content, textbook usage conditions, and textbook evaluation.

2.2.3. Quantitative Model for Course Difficulty

Determining the difficulty of a course is very challenging, and people often make intuitive evaluations based on their own experiences. This study focuses not only on the qualitative analysis of textbook content but also emphasizes quantitative analysis. Therefore, this study also adopts the quantitative model for course difficulty adapted by Shi Ningzhong [10] for quantitative analysis of textbooks. This model studies the depth, time, and breadth of the curriculum to calculate the difficulty level of the textbooks.

The calculation method of the quantitative model for course difficulty is shown in Formula (1):

N = ɑS/T + (1-ɑ)G/T (1)

Where: N represents the course difficulty, S represents the course depth, T represents the course time, G represents the course breadth, S/T represents comparable depth, G/T represents comparable breadth, ɑ is the weighting coefficient, satisfying 0<ɑ<1 [11], reflecting the emphasis of the course on “comparable depth” or “comparable breadth.” Obviously, the greater the comparable depth and breadth of the course, the more difficult the course. This study focuses on static text analysis, and the value of ɑ in the calculation model is taken as the average value of 0.5.

Course difficulty (N) refers to the absolute course difficulty, i.e., static course difficulty—this study only analyzes, compares, and evaluates the textual content of the textbooks without including the real situation of applying the textbooks in the classroom.

Course depth (S) refers to the extent to which the textbook covers the knowledge points, including conceptual knowledge points and related reasoning knowledge content. This study quantifies the weighted average of the cognitive learning objectives, skill-based learning objectives, and experiential learning objectives in the classification of learning objectives [12], as shown in Table 3.

Table 3: Assignment of Values for Learning Objective Categories

Course Breadth (G) refers to the extent of knowledge points covered in the textbook. This is quantified by analyzing the number of knowledge points contained in each chapter.

Course Time (T) refers to the number of class hours required for each chapter’s content. In this study, the number of sections in each chapter of the textbook is counted as the number of class hours.

3. Textbook Comparative Analysis

Based on the analysis of existing research, this paper conducts a comparative study of the 18 selected AI textbooks for primary and secondary schools from both qualitative and quantitative dimensions. The qualitative dimensions include the logic of textbook organization, textbook content, usage conditions, and textbook evaluation. The quantitative dimension uses the quantitative model for course difficulty to calculate and analyze the difficulty of the textbooks.

3.1. Textbook Organization Logic

The logic of textbook organization refers to the order and manner in which the textbook’s knowledge content is presented. The relevant course standards for AI in primary and secondary schools mention that a logical structure meeting the diverse needs of students should be established. This study compares the structure of the textbooks based on their chapter directories.

The organizational sequence of the 12 AI series textbooks is mostly written in the logical order of “initial understanding of AI—typical applications of AI—experiencing internal algorithms of AI—future development of AI,” showing strong logical coherence. Among them, each section in the BNU series textbooks sets learning objectives; the SE series textbooks use “modules” instead of “chapters” to highlight the subject characteristics of AI. The sequence of the 6 elective AI textbooks is: “overview of AI—understanding the technical principles of AI—applying AI technology—future development of AI.” The difference between these 6 textbooks and the series textbooks lies in presenting the technical principles of AI more logically and simply, helping students progress from understanding AI, internal algorithms, and applications to future prospects, step by step. Additionally, compared to the 12 AI series textbooks, the elective textbooks place more emphasis on the construction of AI ethics and morality. Furthermore, the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” mentions that textbooks should present knowledge through project-based activities and specific cases. Among the 18 textbooks, a total of 10 mainly use project-based learning activities to advance the chapter logic.

3.2. Textbook Content

The main function of a textbook is to convey knowledge, and the content of knowledge is the core component of the textbook [13]. The relevant course standards for AI in primary and secondary schools mention that scientific principles and practical applications should be selected equally, reflecting the timeliness and fundamental nature of the course content. This study statistically analyzes the core content and types of activities in the textbooks to understand how different versions of AI textbooks for primary and secondary schools demonstrate the process of acquiring knowledge and the methods of learning to students [14].

The statistics of the core content of the textbooks are shown in Table 4.

Table 4: Statistical Analysis of Core Content of Textbooks

Combining the above analysis of the textbook organization logic, this paper categorizes the core content of the 18 AI textbooks into four main categories: “Overview of Artificial Intelligence,” “Principles of Artificial Intelligence Technology,” “Applications of Artificial Intelligence,” and “Future Development of Artificial Intelligence.” Some textbooks have incomplete knowledge content; for example, the “Principles of Artificial Intelligence Technology” category lacks coverage of core knowledge points like “Expert Systems,” and the “Future Development of Artificial Intelligence” category lacks predictions of “Future Development Trends.” Overall, most textbooks integrate both scientific and contemporary elements, meeting the relevant curriculum standards.

In terms of activity design, the study selected 10 high school textbooks to analyze the types of activities and compare the differences between series textbooks and elective textbooks at the same educational level. The activities are categorized into six types: Thinking and Discussion, Extension, Experiential Inquiry, Project Practice, Exercises, and Summary Reflection. “Thinking and Discussion” involves students analyzing, discussing, and synthesizing problems; “Extension” supplements the subject with additional knowledge; “Experiential Inquiry” allows students to deeply understand and apply AI-related knowledge using relevant technical software; “Project Practice” involves students designing and completing practical activities based on projects or tasks; “Exercises” are problems aimed at deepening understanding of knowledge; “Summary Reflection” involves summarizing chapter content and reflection questions to help students assess their learning outcomes.

Analysis of the frequency distribution of activity types shows that the 10 textbooks have diverse activity designs, including Extension and Experiential Inquiry activities. However, compared to elective textbooks, series textbooks have fewer types of activities and frequently exhibit a count of zero for some types. Thus, elective textbooks 13-18 have more diverse and richer activity arrangements: Textbook 13 has the most activities; Textbooks 13-15 focus primarily on Project Practice, emphasizing project-based teaching and student experience with AI technology applications; Textbooks 17-18 focus on Experiential Inquiry, encouraging more student thinking and experience with AI technology; Textbook 16 includes more Exercises, advocating effective review of knowledge points. The combination of these six types of learning activities can effectively enhance students’ learning outcomes and help them better understand the AI textbook content. However, Figure 1 indicates that Textbooks 9, 10, and 12 lack Project Practice activities; Textbooks 10 and 11 lack Exercises; Textbooks 11 and 12 lack Thinking and Discussion activities; Textbook 18 lacks Summary Reflection. In comparison, Textbook 13 has a higher overall number of activities, diverse types, and a more balanced distribution, with a focus on Project Practice and Extension, indicating more reasonable activity design.

3.3. Textbook Usage Conditions

Textbook usage conditions are the foundation for ensuring that textbooks can be effectively utilized in the classroom. Since AI textbooks for primary and secondary schools involve various AI technology software, hardware, and platforms, this study focuses on comparing the technical conditions for textbook use. The specific statistics are shown in Table 5.

Table 5: Analysis and Statistics of Textbook Usage Conditions

Most textbooks require the use of artificial intelligence-related software and hardware such as computers; the majority of textbooks utilize accompanying or typical AI platforms; and a few textbooks come with digital resource packages. It is evident that textbooks often use popular AI software and platforms, reflecting their timeliness and relevance to the field. However, there are few textbooks with complete digital resource packages, resulting in a lack of systematic and comprehensive learning resources.

3.4. Textbook Evaluation

Teaching evaluation is an important measure to improve teaching quality [15]. The “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” mentions that a diversified evaluation with a focus on competencies should be strengthened, and an evaluation system based on core subject competencies should be constructed. Therefore, this study compares and analyzes textbook evaluations from three aspects: evaluation methods, evaluation subjects, and evaluation content. The specific statistics are shown in Table 6.

Table 6: Analysis of Textbook Evaluation

From the table, it is evident that the subjects and methods of textbook evaluation are diverse, and the evaluation content is varied. With the exception of the East China Normal University series, other textbooks all include instructional design evaluations. However, it is concerning that only five textbooks fully consider the development of students’ abilities and values in their evaluation content. Textbook evaluation should not only focus on the transmission and mastery of knowledge but also on the comprehensive development of students’ abilities and moral values. Therefore, when evaluating textbooks, it is important to consider how task design, question setting, and practical activities can stimulate students’ critical thinking, innovative spirit, and teamwork awareness. Additionally, the development of students’ emotional attitudes and values should be considered to guide them in establishing correct life views, worldviews, and values, thus promoting overall development.

3.5. Textbook Difficulty

The “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” mentions that textbook content should be of moderate difficulty. This study uses the course difficulty quantitative model to calculate and analyze the difficulty of the textbooks. Since different versions of textbooks have varying content designs, and the AI courses for primary and secondary schools primarily aim to introduce and understand basic knowledge related to AI, this study selects the “Overview of Artificial Intelligence” chapters from 17 textbooks as the research object for this dimension of textbook difficulty (excluding Textbook 12, Tsinghua University Edition “Artificial Intelligence (High School Version)”).

3.5.1. Course Breadth

Based on the knowledge points specified in the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” and the characteristics of the selected textbook content, 11 core knowledge areas are identified: meaning of AI, basic features, generation principles, origin and development of AI, mainstream AI technologies, AI products and applications, future development prospects, social value, social responsibility, risks and threats, and security and privacy (ethical) issues. Therefore, the maximum course breadth coefficient is 11.

3.5.2. Course Depth

Referring to the quality level content in the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)”, the learning objectives for different knowledge points in the AI textbooks are divided into four categories: “Recognition”, “Understanding”, “Comprehension”, and “Mastery”. Combined with the learning objective classification values described in Table 3, the course depth coefficient can be calculated. For example, for Textbook 1, there are 7 “Recognition”, 4 “Understanding”, 3 “Comprehension”, and 2 “Mastery” objectives in the learning goals. Using the classification values from Table 3, the calculation is as follows: 7*1+4*2+3*3+2*3, which totals 30. The final course depth coefficient is obtained by averaging this value based on the number of knowledge points, resulting in a coefficient of 30/16, so the course depth coefficient for Textbook 1 is 1.875. Similarly, the course depth coefficients for the remaining 16 textbooks can be calculated.

3.5.3. Course Difficulty

Based on the course breadth coefficient (G), course time coefficient (T), and course depth coefficient (S), and using the course difficulty quantitative model formula (1) described above, the course difficulty coefficient (N) for the “Overview of Artificial Intelligence” chapters in the 17 primary and secondary school AI textbooks can be calculated. This serves as the difficulty coefficient for the textbooks, as shown in Table 8.

Table 7: Difficulty Coefficients for the “Overview of Artificial Intelligence” in Textbooks

In this study, course difficulty is used to represent textbook difficulty. Thus, a higher course difficulty coefficient indicates greater textbook difficulty. From the data in the table, it can be seen that the difficulty coefficients for primary school textbooks are lower compared to the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)”. This is primarily because these textbooks allocate a larger proportion of class time to relevant content, and some textbooks have a lower breadth of content. In the case of junior high school textbooks, Textbook 5 and Textbook 8 have difficulty coefficients higher than the curriculum standards, while Textbook 6 and Textbook 7 have relatively lower difficulty coefficients, mainly due to the longer duration of the course time. For senior high school textbooks, the three series textbooks have lower course depth coefficients, resulting in less comprehensive horizontal and vertical extension of knowledge content, which leads to lower course difficulty coefficients. In contrast, the six elective textbooks have course depth coefficients that fully align with the curriculum standards, with clear learning objectives for knowledge content, resulting in difficulty coefficients close to the curriculum standards, especially for Textbook 17.

Due to the lack of a unified primary and secondary school curriculum standard, different textbooks emphasize different aspects of the content and cover varying knowledge points. Although there is a high similarity in content design among textbooks available on the market, differences in authors’ perceptions of artificial intelligence education can lead to variations in the breadth and depth of the same knowledge points. Therefore, the results of this study should be considered as reference only.

4. Research Conclusions and Discussion

Evaluating textbooks from a value perspective not only helps to identify and select suitable textbooks but also facilitates the improvement and development of textbooks [16]. Based on the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial),” the “Standards for Information Technology Curriculum for Compulsory Education (2022 Edition),” and the “Information Technology Curriculum Standards for General High Schools,” combined with the analysis above, this study finds that the 18 selected textbooks each have unique characteristics and have contributed to the development of artificial intelligence textbooks in primary and secondary schools in China to varying extents. However, there are still some shortcomings.

4.1. Textbook Organization is Systematic, but Some Lack Learning Objectives

In terms of textbook organization, most textbooks are well-structured and complete, demonstrating strong logic and systematicity. However, some textbooks lack learning objectives, leading to weak instructional guidance. This deficiency may arise from multiple factors: On one hand, textbook authors may focus more on the completeness of knowledge points and diversity of teaching methods, without paying enough attention to defining clear learning objectives. On the other hand, the absence of learning objectives also reflects broader challenges in the education sector. For example, reforms in the education system and adjustments to curriculum standards may make it difficult for authors to establish clear learning objectives within a limited time. Additionally, the complexity of the educational environment and the shortage of teaching resources may also limit authors’ efforts and depth in this area.

The organization and allocation of educational resources should place greater emphasis on the clarity of learning objectives, as this directly affects the effectiveness and guidance of teaching and learning. To improve the quality and effectiveness of textbooks, authors should focus more on setting and articulating learning objectives during the design phase to ensure that the textbook is not only logical and systematic but also clearly instructive. Moreover, educational institutions and policymakers should provide clearer educational directions and standards to offer better guidance and support to textbook authors. In summary, clear learning objectives are fundamental and central to textbook organization, directly impacting the effectiveness and depth of student learning.

4.2. Textbook Content is Contemporary but Lacks Comprehensive Knowledge Systems

Regarding textbook content, some artificial intelligence (AI) course textbooks cover innovative and engaging AI application examples and rich project-based activities, aiming to connect AI knowledge with real-life situations and encourage students to engage in hands-on AI technology experiences and explorations. However, some textbooks still fall short in terms of knowledge systems, particularly lacking modules on AI ethics and morality. Firstly, textbook authors may be more inclined to highlight the practicality and cutting-edge aspects of AI technology, neglecting the importance of ethical and humanistic considerations. Secondly, the lack of a comprehensive knowledge system also reflects limitations within the education system and teaching resources. Additionally, teachers may lack relevant knowledge and resource support during course instruction, making it challenging to effectively teach AI ethics and morality.

To address these issues, textbook authors should consider the social impact and ethical issues of AI technology more comprehensively when designing textbooks, incorporating these elements into the teaching system to ensure thorough and in-depth education. Educational management departments should optimize curriculum design and teaching resources, providing more training and support on AI ethics and morality education to help teachers guide students in recognizing and exploring the multifaceted impacts of AI, fostering a thoughtful and responsible attitude towards technology application.

4.3. Textbook Usage Conditions are Diverse, but Some Schools Lack Implementation Environments

Regarding textbook usage conditions, most AI course textbooks are equipped with corresponding learning platforms and digital resources, which facilitate the implementation of experiential and exploratory activities. However, some schools are unable to fully utilize these resources due to disparities in supporting resources and inadequate school facilities. This phenomenon has multiple causes: Firstly, textbook authors may face technical and financial constraints when designing supporting resources, leading to varying quality and coverage. Secondly, there is an imbalance in the allocation of educational resources among schools; some schools may lack sufficient funds and technical support to provide advanced teaching equipment and network environments. Additionally, differences in educational policies and development levels also impact the teaching conditions in different regions.

To enhance the diversity of textbook usage conditions and implementation environments, joint efforts are needed from the government, educational institutions, and schools. On one hand, educational departments should increase investment in educational technology and equipment to promote the balanced allocation of educational resources. On the other hand, textbook authors and publishers should strengthen the development and optimization of supporting resources to ensure that textbook content aligns well with the actual teaching environment, improving the practicality and applicability of textbooks. Through reasonable teaching design and resource integration, overcoming equipment limitations, and creating more inspiring and practical AI classrooms can be achieved.

4.4. Textbook Evaluation is Diverse, but Some Lack Evaluation Design

In terms of textbook evaluation, most textbooks use a variety of evaluation methods such as learning evaluation scales, project evaluation scales, and test questions. Evaluation content is not limited to core knowledge but also focuses on students’ AI values and learning abilities, which benefits their overall development. However, some textbooks lack evaluation design. The constraints of the educational system and teaching management mechanisms are significant reasons for the absence of evaluation design. Educational reform requires comprehensive progress from curriculum design to teaching evaluation, and evaluation design and implementation need supporting policies and resources. Additionally, teachers may lack professional training and methodological guidance in evaluation design, leading to a limited variety of evaluation methods and suboptimal effectiveness.

To address these issues, it is recommended to introduce diverse evaluation methods in textbook design, incorporating evaluation activities that align with experiential and project-based activities to assist teachers and students in real-time evaluation during the learning process. Furthermore, providing professional development training for AI teachers in primary and secondary schools is advised to enhance their ability to design diverse teaching evaluations. Additionally, regular review and updates of textbook content should be conducted to ensure that evaluation activities align with AI development trends and practices, thus improving the scientificness and effectiveness of evaluations.

4.5. Textbook Difficulty is Moderate but Differs from Curriculum Standards

Regarding textbook difficulty, six elective high school textbooks closely match the “AI Curriculum Development Standards for Primary and Secondary Schools in China (Trial)” in terms of course breadth, course time, and course depth, especially with the course depth coefficients fully aligning with the quality level descriptions in the curriculum standards. However, there is still a certain gap between the difficulty of all versions of textbooks and the curriculum standards. The reasons for these issues include: Firstly, textbook authors may have considered students’ capacity and the feasibility of course implementation when designing textbook difficulty, opting to lower the difficulty to ensure more students can successfully complete learning tasks. Secondly, the influence of educational policies and teaching management systems may also contribute to the gap between textbook difficulty and curriculum standards. During textbook development, authors need to follow guidelines and policy requirements from educational departments, which may result in adjustments to textbook difficulty to fit the current educational environment and teaching reality.

To address these issues, it is recommended that textbook authors align their designs with relevant curriculum standards, regularly review and adjust course content to ensure it matches students’ knowledge base and cognitive characteristics, and avoid discrepancies between textbook difficulty and actual classroom needs, preventing overly easy or difficult content. Additionally, a continuous feedback mechanism among textbook authors, teachers, and students should be established to promptly address feedback and make necessary adjustments and updates to textbooks and teaching content, thereby better assisting students in learning AI knowledge and building an AI knowledge system.