LLM-KT: Enhancing Large Language Models with Knowledge Tracing via Multi-Level Plug-and-Play Alignment

LLM-KT bridges Large Language Models with traditional Knowledge Tracing (KT) models through a multi-level plug-and-play alignment framework, injecting compressed historical and behavioral representations into LLMs so they can leverage both their reasoning/world knowledge and the sequential modeling strengths of classical KT approaches.

Problem Statement

Knowledge Tracing predicts student performance from interaction histories to enable personalized education, but existing methods rely on question IDs or surface-level text and lack deep reasoning or world knowledge, limiting their ability to model complex behavioral patterns. LLMs offer strong reasoning and world knowledge but are not naturally designed to consume long sequential interaction histories, creating a gap that this paper aims to close without discarding existing KT model investments.

Key Novelty

A multi-level (task-level + modality-level) plug-and-play alignment paradigm that cleanly separates 'what to ask the LLM' from 'how to inject KT-specific representations' into it
Semantic History Projector: compresses long-term interaction history into question- and concept-specific tokens flexibly inserted into the LLM's input space
Behavioral Dynamics Projector: a sequence adapter that injects sequential interaction-pattern representations from traditional KT models directly into the LLM
Plug-and-play instruction design for task-level alignment that reframes KT prediction to exploit LLM reasoning without full model retraining

Evaluation Highlights

State-of-the-art performance across four standard KT benchmark datasets
Significant outperformance over more than 20 competitive baseline methods

Signal Assessment

6/10 The paper delivers a well-engineered, empirically strong solution (beating 20+ baselines across four datasets) by adapting projector/adapter-based multimodal alignment ideas to the KT domain, but the core mechanism builds on existing plug-and-play alignment concepts rather than introducing a fundamentally new paradigm.

Methodology

Perform task-level alignment by designing plug-and-play instructions that frame the KT prediction objective for the LLM, unlocking its reasoning and world knowledge
Extract long-term history representations via a Semantic History Projector, compressing context into question- and concept-specific tokens for insertion into the LLM prompt/embedding space
Extract sequential behavior representations via a Behavioral Dynamics Projector, using a sequence adapter to feed traditional KT sequential patterns into the LLM
Fuse task-level and modality-level aligned signals within the LLM to produce final performance predictions
Evaluate the integrated framework against traditional and LLM-based KT baselines on four standard datasets

System Components

Task-Level Alignment (Plug-and-Play Instruction)

Instruction-based prompting mechanism that reframes the KT prediction task so the LLM can apply its reasoning and world knowledge to it

Semantic History Projector

Compresses students' long-term interaction history into compact context embeddings tied to question- and concept-specific tokens, then inserts them into the LLM's input

Behavioral Dynamics Projector

A sequence adapter that transfers sequential interaction-pattern representations learned by traditional KT models into the LLM to capture fine-grained behavioral dynamics

Traditional KT Backbone

Underlying sequential model that learns behavioral/interaction pattern representations later projected into the LLM

LLM Backbone

Large language model that consumes aligned task- and modality-level signals to generate the final performance prediction

Results

Metric/Benchmark	Baseline	This Paper	Delta
Prediction performance across 4 standard KT datasets	Best among 20+ competitive baselines	LLM-KT	State-of-the-art, significant improvement (exact figures not in abstract)
Robustness/generalization across datasets	Traditional KT / prior LLM-KT hybrids	LLM-KT	Consistently outperforms across all four datasets

Key Takeaways

Plug-and-play adapters/projectors offer a practical way to inject domain-specific sequential model outputs into LLMs without full fine-tuning, preserving existing KT infrastructure investments
Compressing long interaction histories into specialized tokens is an effective strategy for extending LLMs to handle long user/behavioral sequences beyond typical context limits
Combining a frozen or lightly-tuned pretrained sequence model with an LLM via lightweight adapters is a reusable template for other sequential-prediction domains (e.g., recommendation, healthcare monitoring, financial behavior modeling)
Task-level instruction design and modality-level representation injection can be treated as independent, composable alignment axes when adapting LLMs to structured, non-text prediction tasks

Abstract

Knowledge Tracing (KT) is a pivotal task in personalized education, aiming to predict students’ future performance based on their historical interactions. While prior work has focused on learning behavioral sequences using question IDs or surface-level textual features, these methods often fail to capture complex behavioral patterns due to a lack of deep reasoning capabilities and world knowledge. To address this, we propose LLM-KT , a novel framework that integrates the reasoning power of Large Language Models (LLMs) with the sequential modeling strengths of traditional KT methods via multi-level plug-and-play alignment. Specifically, for task-level alignment, we design a plug-and-play instruction to leverage the rich knowledge and reasoning capacity of LLMs for the KT objective. For modality-level alignment, we introduce two mechanisms to integrate representations learned by traditional methods: (1) a Semantic History Pro-jector that flexibly inserts compressed context embeddings into LLMs using question-and concept-specific tokens to capture long-term history; and (2) a Behavioral Dynamics Projec-tor that enhances LLMs with sequential interaction patterns via a sequence adapter. Extensive experiments on four standard datasets demonstrate that LLM-KT achieves state-of-the-art performance, significantly outperforming over 20 competitive baselines.