1. Introduction

With the rapid development of artificial intelligence, large-scale pre-trained models have shown significant success, particularly in educational technology [1]. Traditional educational systems often rely on static textbooks and fixed teaching processes, which fail to address individual learner needs [2]. In response, personalized educational platforms using Natural Language Processing (NLP) and deep learning have emerged, such as intelligent teaching systems based on generative models like GPT, which can automatically generate code, explain complex concepts, and enhance learning through dynamic demonstrations [3].

The “ZhaoXi” project, a large-model-based educational application, aims to generate code and produce video animations from natural language input. Its core structure includes four steps: concept interpretation, example analysis, animated demonstration, and resource recommendation. Key challenges include selecting appropriate examples from a vast database and recommending relevant video content from online resources [4].

Traditional recommendation systems often rely on user behavior or content similarity but fail to integrate multimodal information such as text, video, and images [5]. In education, where diverse content and teaching methods are involved, a single recommendation approach is inadequate. Deep learning, particularly multimodal learning, offers a solution by processing multiple modalities, enabling more precise and personalized recommendations [6].

This paper proposes a multimodal recommendation system for the “ZhaoXi” project. It integrates user queries, video titles, and descriptions into embedding representations, then refines recommendations using a contrastive loss function. The system first generates embeddings with pre-trained models, followed by feature fusion and matching through a deep neural network, ultimately providing the most relevant recommendations [7,8].

2. Fundamental principles

This study uses the OpenAI text-embedding-3-small model to convert video titles, descriptions, and user inputs into 1536-dimensional embedding vectors for building the recommendation system.

The dual-tower model serves as the core architecture in the recall phase of recommendation systems, with its design concept originating from the DSSM model proposed by Microsoft Research [9].

Figure 1: Dual-tower model architecture

As shown in Figure 1, the model consists of symmetric User Tower and Item Tower, which process user-side features (user profile, behavioral sequence, context information) and item-side features (item attributes, content descriptions, statistical metrics), respectively. The two towers perform nonlinear mapping through independent multi-layer perceptrons (MLP) [4].

\( u={f_{θ}}({X_{user}})=MLP([e_{1}^{(u)},...,e_{m}^{(u)}]) \) (1)

\( u={g_{ϕ}}({Y_{item}})=MLP([e_{1}^{(v)},...,e_{n}^{(v)}]) \) (2)

Where \( e_{i}^{(u)} \) and \( e_{j}^{(v)} \) represent the feature embedding vectors of the user-side and item-side, respectively, and \( θ \) and \( ϕ \) are the network parameters. The model measures semantic similarity through cosine similarity [10].

The core advantage of the dual-tower model lies in its decoupled computation, as shown in Table 1. This architecture completely isolates the processing of user-side and item-side features, enabling the precomputation of item embeddings for massive catalogs, which significantly reduces the online service latency. However, this also leads to the issue of late-stage feature interaction—user and item features only undergo dot product operations at the top-level embeddings, lacking the ability to learn fine-grained cross-feature interactions. Research has shown that this can reduce the recall accuracy for long-tail items [11].

The dual-tower model’s two encoders process the interaction features of \( q \) and \( (t,d) \) respectively, and introduce a cross-modal attention mechanism:

\( α=softmax{(}{W_{q}}q\cdot ({W_{t}}t+{W_{d}}d)) \) (3)

Table 1: Analysis of dual-tower model characteristics

3. Deep collaborative dual tower model

In the recommendation system designed in this study, the core issue is how to provide personalized recommendations for users based on their queries and multimodal data (such as video titles, descriptions, tags, etc.). Specifically, as shown in Figure 2, this study aims to provide precise video recommendations by using deep learning methods, combining the embedding representations of video content (title, description) and user queries.

Figure 2: Deep learning-based video recommendation pipeline

Query tower: responsible for mapping the input query q to an embedding vector q_em. This tower includes a linear transformation layer, an activation function (ReLU), and a dropout layer to enhance generalization.

\( q\_emb={f_{q}}(q)=Re{L}U({W_{q}}q+{b_{q}}) \) (4)

Video tower: responsible for mapping the input video title t and description d to their respective embedding vectors t_emb and d_emb. This is also achieved through a similar process.

\( t\_emb={f_{t}}(t)=Re{L}U({W_{t}}t+{b_{t}}) \) (5)

\( d\_emb={f_{d}}(d)=Re{L}U({W_{d}}d+{b_{d}}) \) (6)

This study fuses query text and multimodal data, such as video titles and descriptions, using concatenation and dot product operations. In the feature fusion layer, the embeddings of the query, video title, and description are concatenated to preserve information integrity and enhance the model's ability to capture complex semantic relationships. In the interaction layer, the dot product calculates the similarity between the query and video content, allowing the model to assess their matching degree. Together, these layers optimize the recommendation system's performance, enabling accurate video recommendations based on the user's query.

By applying the contrastive loss function, the system can not only distinguish between positive and negative samples but also precisely select the most relevant video content from a large pool of candidate videos based on the user query. This optimization process helps the model gradually converge to a state where it can effectively handle different modal information and perform accurate matching.Contrastive loss function is as follows:

\( L(y,{x_{1}},{x_{2}})=\frac{1}{N}y{‖{x_{1}}-{x_{2}}‖_{2}}+(1-y)max{(}m-{‖{x_{1}}-{x_{2}}‖_{2}},0{)^{2}} \) (7)

where \( ‖{x_{1}}-{x_{2}}‖={\sum _{i=1}^{P}(x_{1}^{i}-x_{2}^{i})^{\frac{1}{2}}} \)

4. Experiments and results

4.1. Model comparison

This study uses a self-constructed linear algebra teaching video dataset with 46 professional teaching samples, which includes video titles, descriptions, and corresponding 1536-dimensional embedding vectors. The embedding vectors are generated using the OpenAI text-embedding-3-small model and L2 normalized for better cosine similarity calculation. The dataset is split into training (32), validation (9), and test (5) sets in a 7:2:1 ratio. A negative sampling strategy is applied, where the top 30% most similar samples from unrelated videos are selected as negative samples to enhance the model’s discriminative ability. The recommendation system is built on the PyTorch 2.0 framework and trained on an NVIDIA RTX 3090 GPU, using the AdamW optimizer with a learning rate of 0.001 and cosine annealing scheduling. The batch size is 16, with 200 epochs and an early stopping mechanism (patience=15). Several models, including collaborative filtering, BM25, single-tower, and DCDT, are pre-trained using the same dataset, with evaluation metrics such as Accuracy, Recall, F1, and MMR provided(Table 2).

Table 2: Model comparison

The DCDT model outperforms traditional collaborative filtering [12], improving the F1 score by 19.8% and MRR by 55.6%, indicating its effective use of multimodal data [13] (e.g., video titles, text, descriptions, and embedding vectors) for feature learning and enhanced ranking performance. This leads to better video relevance and higher user satisfaction. The model’s success is attributed to its decoupled computation feature [14], enabling large-scale precomputation of item embeddings, which improves real-time performance and accuracy. Additionally, the dynamic hard negative sampling strategy further enhances the model's discriminative ability, improving recommendation quality [15].

4.2. Ablation analysis

This section will verify the contribution of different modality interaction features to the model through ablation experiments, as shown in Table 3. The DCDT model concatenates the title, description, and query, and then treats the concatenated content as a whole tensor for feature fusion. On the other hand, the single-tower model fuses the query with the title and description separately. Figure 3 shows the distribution of matching scores on the test set for both architectures.

Table 3: Ablation experiment

The training loss curve shows that the final loss of the two-tower model is 0.0845, higher than the 0.0435 loss of the single-tower model. However, the two-tower model's loss decreases more rapidly in the early stages of training, indicating that it can effectively optimize model parameters in a shorter period. Therefore, despite the slightly higher final loss, the two-tower model demonstrated stronger learning capabilities during training. In practical applications, the model involves complex and large-scale cross-modal interactions, and the two-tower model can integrate more information to improve the model's expressiveness and predictive performance. In summary, the two-tower model outperforms the single-tower model in the learning process, especially in handling more complex multimodal data, where it holds a clear advantage.

Figure 3: Loss function curve and similarity scores

The similarity score curve reveals distinct differences in the three models' score distributions. The Dual-Tower model (DCDT) shows a concentration of matching scores between 0.45 and 0.51, with sparse scores above 0.63. This can be attributed to the model's decoupled feature encoding, which enhances consistency between title and description and improves recall for low similarity samples by 8.6%. The negative sampling strategy also contributes to a 10.9% improvement in MRR, enhancing ranking ability. However, the Dual-Tower model exhibits a conservatism bias, leading to more concentrated scores in the lower similarity range. Despite lower matching scores, this design helps the model capture subtle differences between the query and content, making it more effective for complex queries and real-world applications.

5. Conclusion and future outlook

This research presents three key innovations in education recommendation systems with significant practical impact. First, it constructs the first fine-grained recommendation benchmark dataset for linear algebra teaching, offering valuable data for future research. Second, it validates the transferability of pre-trained embeddings in educational multi-modal tasks, expanding the applicability of existing models across various educational domains [16]. Third, it builds a scalable recommendation framework, improving user satisfaction by 31% on the "ZhaoXi" platform, showcasing real-world effectiveness. Future work will focus on integrating text, formulas, and charts into a unified representation space using CLIP pre-training to enhance multi-modal understanding [17], developing a hierarchical attention mechanism to improve recommendation accuracy and personalization, and building a federated framework for distributed updates under privacy protection, advancing education systems toward greater intelligence and fairness [18].