Optimizing course assignment in higher education using natural language processing and semantic similarity

Mathematical approaches to professor assignment

Rakgomo, Mhalapitsa & Seboni (2023) describe the assignment of professors to courses as an extension of the transportation problem, where the goal is to allocate resources efficiently—in this case, professors—while meeting constraints such as classroom capacity, contact hours, course credits, and the number of courses per professor. The authors address the problem in a structured manner using integer linear programming techniques and empirical validation. According to Ongy (2017), course-to-professor assignment is recognized as an essential administrative task that must be carried out each semester in academic departments. This approach models the problem as a mixed-integer programming task, where decisions must be based on multiple objectives, including professor competence and scheduling preferences, and meeting institutional workload requirements. In Wulan et al. (2020), the authors tackle the problem of unbalanced assignment using a linear programming approach based on the Kotwal-Dhope method, a combination of the Hungarian method and the matrix one’s assignment (MOA) technique, balancing the cost matrix and making systematic adjustments to achieve an optimal solution that minimizes costs in the development of a massive open online course (MOOC). In their work, Cutillas & Díaz (2022) propose a solution to the course assignment problem using a MILP model, which accounts for specific organizational characteristics, irregular time schedules, different types of credits and groups, faculty ranks and teaching loads, and preferences. The model is validated using real cases with two datasets from the Universitat Politècnica de València (UPV): the first for faculty assignment in the linguistics department, evaluating solution quality and computational efficiency, and the second for faculty assignment in industrial management engineering, assessing the scalability and speed of the methodology. In Szwarc et al. (2020), the main problem addressed by the authors is the assignment of professors to courses so that professor satisfaction is maximized while meeting various operational constraints, such as the competencies required for each course, professors’ minimum and maximum workloads, and the balanced distribution of tasks among available faculty. To achieve this, they create a competency framework linking course requirements with professor competencies. Optimization is carried out through a mixed-integer linear programming model. In the study by Caselli, Delorme & Iori (2022), the authors develop an integer linear programming (ILP) model to maximize tutor preferences when assigning workshops, subject to constraints such as tutor skills and competencies, maximum tutoring hours based on employment contracts, simultaneous workshops, and tutors’ preferences for teaching specific workshops. The model was tested in the 2019–2020 academic year, resulting in a 29% increase in tutor preferences compared to manual assignments.

Artificial intelligence approaches

In Ngo et al. (2021), the problem of assigning professors to courses is addressed using a genetic algorithm that balances student needs with professor workload, ensuring a degree of robustness in teaching schedules. This approach involves identifying links between professor competencies and those required by the curriculum courses. The algorithm was implemented and tested using data from the Spring 2020 semester of the Department of Computer Fundamentals at FPT University. In Laguardia & Flores (2022), the authors propose a solution based on a Tabu Search algorithm, a metaheuristic that iteratively improves solutions through minor changes and avoids local optima using an adaptive memory called the Tabu list. They define an objective function that considers the essential qualities of an assignment while controlling computational complexity, allowing for multiple iterations within a reasonable timeframe. Several tests were conducted using data from the Department of Exact Sciences at the Technological University of Panama, covering 205 courses and 71 professors.

Natural language processing approaches

The use of NLP techniques in academic activities is fundamentally student-centric, potentially significantly enhancing efficiency and personalization in higher education. A systematic review by Sunar & Khalid (2024) delves into how NLP techniques are applied to analyze student feedback to professors, highlighting deep learning algorithms such as long short-term memory (LSTM) and bidirectional encoder representations from transformers (BERT) to improve the accuracy of sentiment analysis and feedback categorization. The development of chatbots like Infobot, capable of instantly responding to student queries across various social platforms, has significantly reduced professors’ workload, fostering interaction and personalized assistance in university courses (Lee et al., 2020). Moreover, applying advanced NLP techniques and co-occurrence network analysis in the analysis of open-ended responses has deepened our understanding of students’ perceptions of sustainable development, as Yamano et al. (2022) demonstrated in their study on education for sustainability. NLP has also made significant advances in the field of educational management, with the most commonly reported use being the automation of tasks involving the processing of large amounts of text, such as creating assessments, topic analysis, course recommendations, and student support. Alhawiti (2014) presents an overview of how NLP improves educational processes, including writing assistance, assessment feedback, and subject analysis. Focusing on management, Chandrapati & Koteswara Rao (2023) employed topic modeling and text clustering to empirically evaluate professor effectiveness empirically, thereby eliminating subjectivity and exploring latent topics that are difficult to perceive individually.

For her part, Giannopoulou & Mitrou (2020) proposes a methodology for the automatic classification of a collection of e-books of various types, using index information. Her work demonstrates how NLP can be used to improve access to and organization of large amounts of text. Nikolić, Grljević & Kovačević (2020) report on sentiment analysis in higher education reviews, highlighting the advantage of NLP in capturing latent perspectives in the opinions expressed in the texts. Strousopoulos et al. (2024) used NLP strategies to explore the design of a smart online virtual university.

One strength of NLP is the tools for calculating semantic similarities in texts. In this regard, NLP, utilizing similarity measures, is employed to enhance and adapt teaching dynamics to new educational technologies. Susanti et al. (2018) used semantic similarity to generate distractors for multiple-choice questions. Wulff et al. (2023) found that BERT-generated embeddings outperform other traditional architectures in classifying short educational texts. This research reinforces our decision to use semantic similarity measures in conjunction with advanced transformer-based models, such as multi-qa-mpnet-base-dot-v1.

In the development of large language models (LLMs), the approach of using NLP in management tasks that often require manual processing is validated. It can facilitate the implementation of solutions through application programming interfaces (APIs). Parker et al. (2025) used the GPT-4 and GPT-3.5 models to analyze feedback data from student surveys. They used semantic representations to extract information at the course level. The work demonstrates the application of language models in the semantic analysis of academic content written by students.

Topic modeling

Internet technology and Big Data provide abundant information that has significantly transformed society and the economy, and their application in education has attracted considerable attention. Shaik et al. (2022) studied artificial intelligence in personnel management, exploring opportunities and challenges for the higher education sector.

Top2Vec

This method does not require traditional preprocessing steps and automatically determines the number of topics, generating vectors representing documents and words in a shared semantic space. The process follows four phases: creation of a semantic embedding, identification of the number of topics through density-based clustering, calculation of topic vectors as centroids of dense document regions, and hierarchical topic reduction to facilitate interpretation (Angelov, 2020).

BERTopic

BERTopic is an unsupervised machine learning model that leverages pre-trained language models such as BERT or robustly optimized BERT pretraining approach (RoBERTa) to generate document embeddings, clusters these embeddings, and creates topic representations using class-based term frequency-inverse document frequency (TF-IDF). It effectively uncovers the underlying thematic structure in large textual datasets and facilitates topic exploration (Grootendorst, 2022; Reimers & Gurevych, 2019; Naghshzan & Ratte, 2023). The process begins by creating embeddings from a document set, producing a 384-dimensional document matrix (D) using BERT. A specific embedding represents each document (Dn). Dimensionality reduction is then performed using uniform manifold approximation and projection (UMAP), followed by clustering with hierarchical density-based spatial clustering of applications with noise (HDBSCAN). After completing these steps, the model can extract topics from the clusters and generate visualizations.

Latent Dirichlet allocation

Latent Dirichlet allocation (LDA) is a generative probabilistic model that assumes documents are composed of a mixture of topics, where a distribution over words characterizes each topic. It helps discover latent thematic structures in extensive collections of text in an unsupervised manner (Wang et al., 2021; Chen et al., 2011; Harrison & Sidey-Gibbons, 2021; Udupa et al., 2022).

Coherence metrics

UMASS. Based on the conditional co-occurrence of words within the corpus, this metric helps evaluate the internal cohesion of topics. However, its performance heavily depends on the corpus size (Mimno et al., 2011).

Cv measures topics’ semantic coherence by combining cosine similarity with text segmentation strategies, providing a more robust measure than other traditional metrics (Röder, Both & Hinneburg, 2015).

NPMI. This metric assesses the statistical relationship between words within the same topic, normalizing co-occurrence probability and reducing the bias introduced by frequent words (Bouma, 2009).

Dataset

The dataset used in this study was extracted from administrative systems and databases at the Faculty of Computer Science, Universidad Técnica de Manabí. Three primary datasets were utilized.

• (1)

  Course contents and professor profiles: documents for 35 professors and 42 courses. Each professor has three documents detailing training, experience, and publications.

  • (a)

    Professor profile attributes.

    • (i)

      Training: description of subject matter or content.

    • (ii)

      Experience: Name of the subject, minimum contents of the subject.

    • (ii)

      Publications: title

• (2)

  Administrative records of historical assignments from six academic periods between 2020 and 2023, comprising 288 records.

  • (a)

    Historical assignments: course, Professor

• (3)

  There are 1,470 professor preference records, indicating binary values (0 or 1) for each course per professor, reflecting whether it is preferred

  • (a)

    Attributes of preference records: Course, Professor, Selection (0|1)

These datasets were selected for their practicality in course management and the availability of historical data.

Preprocessing

Text preprocessing followed standard practices in NLP To ensure practical analysis and performance of the topic-generation models.

• Conversion of texts to lowercase.

• Removal of special characters, punctuation, and irrelevant numbers.

• Normalization of whitespace.

• Remove Spanish-language stopwords using standard stopword lists from NLTK and spaCy.

• Removal of recurrent administrative terms lacking semantic value for topic detection.

• Tokenization of texts using spaCy.

• Lemmatization to reduce words to their base form (e.g., “procesando”→ “procesar”).

Generation of semantic representations

A comparative study was conducted between three topic-generating models–LDA, BERTopic, and Top2Vec–which are widely used in NLP tasks. The evaluation focused on analyzing the semantic coherence of the generated topics concerning the training dataset. For this comparison, three coherence metrics recognized in the literature were used–UMASS, Cv, and NPMI–each with different statistical and linguistic foundations that allow for assessing the interpretive quality of the extracted topics.

Since BERTopic internally uses contextual embeddings, its superior performance on NPMI and Cv when creating representations of the dataset suggests a more accurate semantic structuring for our task. It justifies its selection as the model for creating our semantic representations.

Inlays generated with BERTopic were used to represent course and Professor profiles. To obtain the embeddings, three BERTopic models were created, each model using an all-MiniLM-L6-v2, all-mpnet-base-v2 and multi-qa-mpnet-base-dot-v1 sentence transformer respectively. Documents were fed into the created model and the embeddings were obtained. Subsequently, semantic similarities were evaluated. The multi-qa-mpnet-base-dot-v1 sentence transformer was selected as it obtained the highest semantic similarities in the evaluations.

Similarity calculation

The similarity matrix was constructed to determine the optimal course assignment to professors based on the semantic analysis of course descriptions and professor profiles. Text embeddings generated with the multi-qa-mpnet-base-dot-v1 model were used, and cosine similarity between the representative vectors of each course and the professor was calculated.

Cosine similarity is defined as follows:

(1) $$CosineSimilarity\left(A,B\right)={\displaystyle \frac{A\cdot B}{\Vert A\Vert \Vert B\Vert }={\displaystyle \frac{{\sum }_{i=1}^n{A}_i{B}_i}{\sqrt{{\sum }_{i=1}^n{A}_i^2}\sqrt{{\sum }_{i=1}^n{B}_i^2}}}}$$where:

• A is the embedding representation vector of the course.

• B is the professor’s embedding representation vector.

The cosine similarity results range between −1 and 1. Values closer to 1 indicate higher semantic similarity between the course and the professor. Since the embeddings used are generated from representations in high-dimensional vector spaces, cosine similarity is an appropriate metric because it measures the orientation between vectors without being influenced by their magnitude.

The cosine similarity was calculated for each course-professor pair, resulting in a similarity matrix of dimensions A × B, where A = 42 courses and B = 35 professors. This matrix served as the basis for course assignments, optimizing allocations through the Hungarian algorithm and applying a similarity threshold to enhance the model’s accuracy.

Previous explorations determined the factors and weights for each factor. The final similarity between courses and professors was calculated by combining training, experience, and publication factors with an professor preference factor using the Equation:

(2) $$FS=\left(SS\times 0.7\right)+\left(PP\times 0.3\right)$$where:

• FS: Final Similarity

• SS: Semantic Similarity

• PP: Professor Preference

Assignment methods

Three assignment methods were tested. (1) The greedy method assigns each course to the professor with the highest similarity in each iteration. (2) the Hungarian algorithm identifies the optimal assignment by minimizing the cost matrix derived from similarities. (3) The similarity-threshold-based method assigns courses to professors only if their similarity exceeds a predefined threshold. The operation of each method is detailed below.

Greedy method

The Greedy method is a heuristic strategy that selects the best local assignment at each iteration without considering the impact on future assignments. Its goal is to assign each course to the professor with the highest similarity available at that moment (Brassard & Bratley, 1996).

The algorithm follows these steps:

• All possible course-professor assignments are sorted in descending order according to Final Similarity.

• The course is assigned to the professor with the highest available similarity, avoiding repeated assignments.

• The process is repeated until all courses are assigned or no viable assignments remain.

The assignment criterion is formally defined as:

(3) $$Assignment(A,B)=arg{max}_{(a,b)\in P}\left(FinalSimilarit{y}_{(a,b)}\right)$$where:

• A is the set of courses.

• B is the set of professors.

• P represents all possible course-professor assignments.

Hungarian algorithm

The Hungarian algorithm (Kuhn, 1955; Munkres, 1957) is a combinatorial optimization method designed to solve assignment problems optimally. It finds the minimal-cost matching in a cost matrix or, in this context, determines the optimal assignment of courses to professors based on similarity.

Algorithm steps

• Represent the problem as a cost matrix C of size N × M, where C_ij denotes the cost of assigning course i to professor j.

• Subtract the minimum value from each row and each column of the matrix.

• Identify optimal assignments through iterative matrix reduction steps.

• Assign courses to professors by minimizing the cost or, in this scenario, maximizing similarity.

Given a set of courses A and a set of professors B, the assignment problem is formally defined as:

(4) $$max\sum _{i=1}^N\sum _{j=1}^M{X}_{ij}\cdot Finalsimilarit{y}_{ij}.$$

Subject to:

(5) $$\sum _{j=1}^M{X}_{ij}=1,\mathrm{\forall }i\in A$$

(6) $$\sum _{i=1}^N{X}_{ij}=1,\mathrm{\forall }j\in B$$

(7) $$X_{ij}\in \{0,1\}$$where X_ij = 1 if course i is assigned to professor j, and X_ij = 0 otherwise.

Similarity-threshold-based method

The similarity-threshold-based method assigns courses to professors only if their similarity exceeds a predefined threshold. This approach avoids assignments with low similarities, thereby enhancing the quality of the allocation (Ricci, Rokach & Shapira, 2011).

Algorithm operation:

• Define a similarity threshold α; in this study, α = 0.65.

• Select the highest similarity available for the course i.

• If the selected similarity is greater than or equal to the threshold α, assign course i to professor j (X_ij).

• Repeat this process for all courses.

Mathematically, the assignment is constrained as follows:

(8) $$X_{ij}=\{\begin{array}{c}1ifFinalSimilarit{y}_{ij}\\ 0ifFinalSimilarit{y}_{ij}\end{array}\begin{array}{c}\ge \\ <\end{array}\begin{array}{c}\alpha \\ \alpha \end{array}.$$

Evaluation metrics

The precision, recall, and F1-score metrics were used to evaluate the quality of the assignments, taking the professor preference matrix as the absolute truth. In this study, the calculation of recall and F1-score is used not as measures of interactive response (as is done in recommender systems) but as metrics of comparison between the models’ assignments (Dalianis, 2018; Yacouby & Axman, 2020) and the professors’ explicit preferences, considered as the reference truth. These metrics allow us to quantify how many of the assignments made by the model coincide with the professors’ preferences, as well as the balance between hits and misses.

Metrics relevance

Precision indicates the percentage of courses correctly assigned out of all assignments calculated by the model. In academic management, a high accuracy value indicates that the model avoids suboptimal assignments. It minimises conflicts with institutional policies or between the alignment of course-professor profiles.

Precision (P) refers to the percentage of assignments generated by the model that correctly aligns with the professors’ preferences.

(9) $$Precision={\displaystyle \frac{Correctassignments}{Totalassignmentsmadebythemodel}.}$$

Formally defined as:

(10) $$P=\frac{TP}{TP+FP}$$where:

• True Positives (TP): Correct assignments according to professor preferences.

• False Positives (FP): Incorrect assignments according to professor preferences.

Recall assesses the portion of courses that the model has assigned correctly and that are preferred or suitable for a professor. A high value in this metric indicates the model’s ability to comprehensively calculate professor suitability, ensuring that all profile dimensions and their corresponding weights have been considered and accounted for.

Recall (R) was calculated, measuring the percentage of professor-preferred assignments correctly identified by the model:

(11) $$Recall={\displaystyle \frac{Correctassignments}{Totalinstructorpreferredassignments}.}$$

Mathematically defined as:

(12) $$R=\frac{TP}{TP+FN}$$where:

• TP are assignments made by the model matching professor preferences.

• False Negatives (FN) are professor-preferred assignments not made by the model.

F1-score, the harmonic mean of precision and recall. It balances the assignments that were matched and the preferences that were covered by the assignment strategy. It is essential for institutional efficiency and professor satisfaction, as it is crucial to achieve an assignment that avoids incorrect matches (precision) of unwanted or misaligned courses with the professor, while maximizing the correct ones (recall) of the most significant number of aligned courses within the correct assignments made by the model.

The F1-score (F₁) was also calculated to evaluate the overall quality of the model, as it balances precision and recall:

(13) $$F_1=2\times \frac{P\cdot R}{P+R}$$where P identifies precision, and R represents recall.

The F1-score is particularly useful in course assignments since a high precision with low recall implies very accurate assignments but leaves many courses and professors unassigned, whereas a high recall with low precision results in numerous assignments that do not align with professor preferences.

To define the metrics in practical terms, evaluation with these metrics directly reflects the balancing act faced by the academic decision-maker in each period. For example, an assignment with high precision but low recall can result in very accurate but incomplete assignments, while a balanced F1-score ensures that assignments are reliable and reasonably complete.

Statical analysis

In this study, several statistics methods were used to verify that the model inference, based on the evaluation parameters and the composition of the experimental data. The usual standard evaluation metrics for classification and information retrieval models such: accuracy, recall, and F1-score are limited, non-negative, and commonly biased. These metrics also tend to violate the assumptions of normality and homoscedasticity required for parametric tests; in this sense, the Shapiro-Wilk test was applied, which is highly sensitive with small samples, as in this case.

Taking into account the results of the Shapiro-Wilk test (Table 2) and, the Kruskal-Wallis H test was selected as a nonparametric alternative to one-way analysis of variance (ANOVA) to compare independent groups without assuming Gaussian distributions. In order to make pairwise comparisons and identify which pairs differed Dunn’s post hoc test with Bonferroni correction was used.

Computational resources

The following computational resources were employed for the implementation and execution of the course-to-professor assignment model:

Hardware:

• Processor: Intel Core i7-12700K (12 cores, 20 threads)/AMD Ryzen 9 5900X

• RAM: 32 GB DDR4 3200MHz

• Graphics Card (for embedding acceleration): NVIDIA RTX 3060 (12 GB VRAM)/NVIDIA A100 on Google Colab

• Storage: 1TB NVMe SSD

Software and libraries:

• Programming Language: Python 3.9

• Execution Environment: Google Colab/Local Jupyter Notebook

• Libraries:

  • ○

    Transformers and SentenceTransformers (multi-qa-mpnet-base-dot-v1)

  • ○

    Gensim (for topic generation using LDA)

  • ○

    BERTopic (for embedding-based topic generation)

  • ○

    scikit-learn (for similarity computations and evaluation metrics)

  • ○

    SciPy and NumPy (for optimization using the Hungarian algorithm)

Execution time:

• Embedding generation: Approximately 5–10 min on graphics processing unit (GPU), ~40 min on central processing unit (CPU).

• Cosine similarity calculation and assignment matrix: Approximately 2–5 min.

• Hungarian algorithm execution: Less than 1 s for standard-sized matrices (~50 courses, ~30 professors).

These resources ensured efficient processing times, allowing for the creation and testing of the proposed model. Distributed execution in cloud computing environments is recommended in environments with lower computational capacity.

The following section describes the Proposed Model in detail, complementing the previously described methodology. This section is justified by the need to provide a clear and structured explanation of how specific components of the model interact, including semantic embedding generation, integration of professor preferences, and algorithms utilized for optimal course assignments. This division facilitates understanding and reproducibility, highlighting the general methodological approach and the specific structure of the developed model.

Proposed model

The model developed in this study uses NLP to address the course-to-professor assignment problem. It integrates semantic similarity between course contents and professor profiles, as well as explicit preferences provided by professors.

As input, the model receives records of course content, professor profiles, and assignment preferences previously selected by the professors. In the following stage, this data undergoes standard preprocessing procedures in NLP:

• Conversion of text to lowercase.

• Removal of special characters, punctuation, and irrelevant numbers.

• Normalization of whitespace.

• Remove Spanish stopwords using stopword lists from NLTK and spaCy.

• Elimination of recurrent administrative terms with no semantic value for topic detection.

• Tokenization of text into words using spaCy.

• Lemmatization.

Once the data is preprocessed, a pivotal phase for the model’s effectiveness, we move on to semantic representation. Here, we employ the state-of-the-art BERTopic and the high-performing multi-qa-mpnet-base-dot-v1 embedding model from SentenceTransformers. This tool choice underscores our model’s cutting-edge nature, ensuring it is equipped with the best resources for semantic representation tasks.

Cosine similarity between the generated embeddings is used to evaluate compatibility between courses and professors. Cosine similarity is defined by Eq. (1), as indicated in the methodology section. A similarity matrix of size N × M is obtained, where N represents the number of courses and M is the number of available professors. This matrix is referred to as semantic similarity.

Subsequently, to explicitly incorporate professor preferences, semantic similarity is weighted with the explicit assignment preferences using Eq. (2) to compute the Final Similarity matrix. In this study, semantic similarity is weighted at 70%, while professor assignment preferences are weighted at 30%.

Finally, the assignment is performed using the Hungarian algorithm and a similarity threshold filter set at 0.65. The output obtained is a list of courses assigned to professors. Figure 3 graphically illustrates the complete architecture of the model.

Scoring function and rol of preferences

For each course-professor pair (i, j), we define a composite scrore.

$$S_{i,j}=\propto \cdot Si{m}_{sem}\left(C_i,D_j\right)+\beta \cdot P_{i,j}\alpha +\beta =1,\alpha >\beta$$where Sim_sem is the cosine similarity between embeddings derived from multi-qa-mpnet-base-dot-v1 and P_ij indicates explicit professor preference. The assignment is obtained by optimizing ∑_i,j S_ij A_ij under one-to-one matching and institutional constraints. Preferences are not treated as labels to predict but as a secondary criterion that modulates semantically suitable matches.

Ethical considerations

No human subjects were used in this research, nor was there any direct interaction with individuals. All data used in the analysis were obtained, with prior authorisation, from administrative records of the Faculty of Computer Science at the Technical University of Manabí. The dataset did not contain any personal, sensitive, or identifiable information, as all teacher identifiers were anonymised prior to analysis. The research procedures complied with institutional guidelines for the handling of administrative data, and therefore, no formal ethical approval was required. The study poses no risk to individuals, preserves privacy, and complies with standard ethical practices in educational and computational research.

Topic modeling evaluation

Among the three topic models tested—BERTopic, LDA, and Top2Vec—BERTopic achieved the highest semantic coherence according to the UMASS, Cv, and NPMI metrics. The comparison was conducted using a representative corpus of course and professor documents, evaluated across different numbers of keywords and multiple random repetitions to ensure stability.

For embedding generation, the multi-qa-mpnet-base-dot-v1 model from the SentenceTransformers library was selected, as it consistently outperformed other tested embedding models, including all-MiniLM-L6-v2 and all-mpnet-base-v2. This transformer model was integrated into the BERTopic pipeline to enhance the semantic granularity of topic representations. Its superior performance in intrinsic and extrinsic similarity tests confirmed its suitability for capturing domain-specific content. The selected model also computed the cosine similarity between course and professor embeddings in downstream assignment evaluations, ensuring consistency throughout the NLP pipeline.

Shapiro-Wilk tests were performed to assess whether the values for each metric in the different models followed a normal distribution. These tests indicated that LDA, BERTopic, and Top2Vec did not follow a normal distribution (p < 0.05) for any metric. Since at least one model did not comply with normality, parametric tests such as ANOVA were ruled out. Table 2 shows Shapiro-Wilk test values.

The Kruskal-Wallis tests, a significant part of our analysis, yielded an H-statistic for each metric: UMASS = 44.2798, Cv = 38.6335, NPMI = 47.6393, with p-values of UMASS = 2.43e−10, Cv = 4.08e−09, NPMI = 4.52e−11. These results indicate that not all models generate topics similarly. We then used Dunn’s test with Bonferroni correction to identify which models were significantly different, further underlining the importance of our findings.

The results of the Dunn test indicate that in the UMASS metric, the three models differ significantly from each other (p < 0.01), with LDA standing out as the model with the most significant difference compared to BERTopic and Top2Vec.

In the Cv metric, BERTopic showed highly significant differences compared to LDA (p < 0.000001) and also compared to Top2Vec (p = 0.00025), while LDA and Top2Vec did not present statistically significant differences between them.

Regarding the NPMI metric, BERTopic had an advantage, with significant differences compared to LDA (p = 0.001675) and Top2Vec (p = 1.536895e−11). Reinforcing its superiority in terms of semantic coherence, which has clear real-world implications for the practical application of these models.

Comparison assignment method

The historical human-made assignments were restricted to a single academic term to ensure fairness in the comparative evaluation of assignment strategies. This approach aligns the evaluation horizon with the automated strategies, each generating assignments for one period. By doing so, we prevent the inflation of recall that may arise when aggregating assignments over multiple historical cycles.

Five assignment strategies were compared:

• 1.

  Greedy method

• 2.

  Hungarian algorithm

• 3.

  Similarity-threshold-based method (threshold = 0.65)

• 4.

  Hungarian algorithm + similarity threshold (0.65)

• 5.

  Human-made assignments

Table 3 compares the precision, recall, and F1-score assignment methods, taking professor preferences as ground truth. The Hungarian algorithm, followed by applying a 0.65 similarity threshold and the threshold-based method, achieved the best precision score of 1.0000 (100%), indicating that all assignments were correct. Figure 4 illustrates the performance metrics among the tested assignment methods.

It is worth noting that recall and F1-score display similar trends across strategies in Fig. 4. This effect is expected because several NLP-based strategies (threshold and Hungarian+threshold) achieve near-perfect precision values (≈1.0). Under these conditions, the F1-score, defined in Eq. (13), becomes dependent mainly on recall, resulting in comparable value differences between the two metrics. This behavior reflects the mathematical relationship between the two measures under high-precision scenarios rather than a calculation artifact.

The Hungarian algorithm achieved a precision of 0.8286 (82.86%) and an F1-score of 0.4113 (41.13%), surpassing both the Greedy method 0.7143 (71.43%) with an F1-score of 0.3546 (35.46%) and historical human-made assignments 0.0227 (2.27%) with an F1-score of 0.0133 (1.33%). The similarity-threshold-based method achieved a precision of 1.0000 (100%) and an F1-score 0.4525 (45.25%).

The Hungarian algorithm, combined with applying a 0.65 similarity threshold to select only assignments with a high degree of semantic and preferential compatibility, achieved a precision of 100%, significantly improving assignments compared to traditional methods and matching the performance of the similarity-threshold-based method. This strategy achieves a recall value of 0.2736 (27.36%) and an F1-score of 0.4296 (42.96%).

These findings highlight the advantage of integrating semantic similarity and professor preferences into the assignment. The hybrid approach combines optimization and filtering, offering accuracy while meeting institutional constraints.

To investigate whether significant differences existed among the assignment strategies across key evaluation metrics, a Kruskal–Wallis H-test was conducted separately for each metric. The test revealed significant differences among precision, recall, and F1-score strategies when analyzed by course and professor (p < 0.001 for all comparisons).

Post-hoc pairwise comparisons using Dunn’s test with Bonferroni correction confirmed that the threshold-based and Hungarian+threshold strategies significantly outperformed the Historical and Greedy methods in both precision and F1-score (p < 0.001). Comparisons such as Greedy vs. threshold and Greedy vs. Hungarian+threshold also reached statistical significance (p < 0.05), though with smaller effect sizes.

The graphical summary in Fig. 5 shows the strength of these differences based on the negative logarithm of p-values, highlighting the superiority of the threshold-integrated strategies. These results provide strong statistical evidence that integrating semantic similarity and faculty preferences yields measurable improvement in assignment quality.

Comparison granular assignment method

Results per course

Table 4 summarizes the mean and standard deviation for each metric per course. The results show that the threshold-based method and the Hungarian method and subsequent filtering with a threshold consistently achieve high accuracy for most courses and Professors, while Greedy and human-made assignments have lower accuracy and higher variability. Figure 6 shows the distributions of the F1-scores. Figure 7 shows the distribution of accuracy and Fig. 8 is the graphical representation of the recall measures. We can observe that the NLP-based methods stand out, generating tighter and higher distributions compared to the other methods, while in the human-made assignments we observe a lower median and a larger variability.

Table 4:

Comparison of average and standard deviation of metrics.

Each data point represents a granular analysis of task results across the entire dataset for each metric and strategy. This analysis assesses whether the observed improvements in model performance are consistent across courses, rather than relying on aggregate results.

	Precision		Recall		F1-score
Strategy	Mean	SD	Mean	SD	Mean	SD
Human-made	0.0125	0.0791	0.0063	0.0395	0.0083	0.0527
Hungarian	0.8056	0.4014	0.3321	0.2958	0.4342	0.3067
Greedy	0.6757	0.4746	0.2645	0.2845	0.3517	0.3132
Threshold	1	0	0.434	0.2853	0.558	0.2498
Hungarian+threshold	0.9355	0.2497	0.3856	0.2841	0.5042	0.2706

DOI: 10.7717/peerj-cs.3557/table-4

Results per professor

Table 5 summarizes the mean and standard deviation for each metric per professor. The results show that the threshold-based method and the Hungarian method and subsequent filtering with a threshold consistently achieve high accuracy for most courses and professors, while Greedy and human-made assignments have lower accuracy and higher variability. Figure 9 shows the distributions of the F1-scores. Figure 10 shows the distribution of accuracy and Fig. 11 is the graphical representation of the recall measures.

Table 5:

Comparison of assignment strategies for courses based on performance metrics.

Each data point represents a granular analysis of task results across the entire dataset for each metric and strategy. This analysis assesses whether the observed improvements in model performance are consistent across professor, rather than relying on aggregate results.

	Precision		Recall		F1-score
Strategy	Mean	SD	Mean	SD	Mean	SD
Human-made	0.0152	0.0870	0.0061	0.0348	0.0087	0.0497
Hungarian	0.8286	0.3824	0.2989	0.2400	0.4145	0.2628
Greedy	0.7000	0.4570	0.2217	0.1777	0.3278	0.2441
Threshold	0.7000	0.4661	0.2797	0.2169	0.3907	0.2857
Hungarian+threshold	0.9667	0.1826	0.3487	0.2226	0.4836	0.2154

DOI: 10.7717/peerj-cs.3557/table-5

Kruskal-Wallis tests confirmed statistically significant differences between strategies for all metrics, both at the course and level (p < 0.001). Dunn’s post hoc tests also show that the threshold-based strategies and the hybrid Hungarian+threshold strategy achieve significantly higher scores than the greedy and human-made assignments (p < 0.001). Suggesting that the observed improvements are maintained across the entire data set and are not limited to aggregate averages.

The proposed course assignment model

Based on NLP techniques, the proposed course assignment model integrates semantic similarity based on embeddings with faculty preferences and significantly improves human-made assignments. Specifically, the strategy combining the Hungarian algorithm with a similarity threshold (0.65) and preference weighting achieved a precision of 1.0000 and an F1-score of 0.4296, compared to the 0.0227 precision and 0.0133 F1-score obtained from human-made assignments. These results confirm the effectiveness of the proposed model and underscore its potential to enhance teaching load distribution in higher education institutions.

It is worth noting that the historical assignment strategy was evaluated based on data from a single academic term rather than aggregating multiple periods. This decision was made to ensure temporal fairness in the comparison with the model-generated strategies, which produce course assignments for one term at a time. Including historical assignments from multiple periods would have artificially inflated the recall and F1-score metrics due to the increased probability that at least one historical assignment aligns with a professor’s stated preferences. Consequently, the relatively low precision (0.0227) and F1-score (0.0133) observed for the historical strategy reflect a more equitable and rigorous evaluation scenario.

Assignment comparison with weighted and unweighted preference

A comparative analysis was conducted using two model versions to evaluate whether incorporating faculty preferences into the similarity-based assignment model improves the quality of course assignments. One version was based solely on semantic similarity, and another combined semantic similarity with weighted faculty preferences. Both models were run under identical conditions, without applying a similarity threshold, to isolate the effect of preference integration. Table 6 compares the assignment regarding precision, recall, and F1-score.

The results provide clear empirical support for the claim that integrating faculty preferences into the course assignment model would improve performance. Compared to the baseline model based solely on semantic similarity, the preference-weighted model achieved significantly higher scores in all evaluation metrics: precision increased from 0.20 to 0.83, recall increased from 0.066 to 0.27, and F1-score increased from 0.099 to 0.41.

Strengths and weaknesses of assignment strategies

To guide future research and practical adoption, it is important to highlight the strengths and limitations of each evaluated assignment method.

The Hungarian algorithm generates an optimal overall assignment based on an inverted similarity score matrix, ensuring that each professor is assigned a course and that the distribution is equitable. However, it can generate assignments with low similarity.

Greedy algorithm: a high-speed and straightforward method, computationally convenient when working with large volumes of data. However, it can generate low-similarity, suboptimal assignments because it calculates local solutions per course rather than considering the overall context, where all courses must be taken into account.

Threshold-based assignments: This method is the simplest to implement and therefore also the fastest. Once the course-professor similarity has been calculated, it filters out assignments greater than the threshold value, resulting in perfect accuracy. On the other hand, it can create an overload for professors who teach several courses with high similarity.

Hungarian algorithm + threshold: combines the advantages of the Hungarian algorithm and the threshold method. This results in a balanced distribution with high accuracy. However, courses that are not assigned due to low similarity must be assigned manually or through an alternative strategy. Even so, it is an excellent option for implementation in real-world systems under institutional constraints.

Human assignments reflect one of the real-world assignment practices, guided by institutional policies and the knowledge of the person executing the process. It is very flexible and adapts quickly to institutional changes. Its disadvantage is that the decision-maker’s experience conditions it and may not be objective, resulting in suboptimal assignments. It also requires a significant amount of time to perform.

Limitations

• 1.

  The benchmarking data used was from a single academic period, to ensure temporal equity between historical and model-generated assignments and to avoid recall inflation. This decision may limit the generalizability of the results across different academic periods with variable subjects or where content has been updated.

• 2.

  The accuracy of the model depends on high values of semantic similarity to recommend valid assignments. In cases where no Professor reaches a score above the threshold, the model may not assign the course, leaving gaps that require manual intervention or alternative strategies.

• 3.

  The model does not explicitly consider operational constraints such as academic load limits. Similarity and preferences guide the allocation, yet the model needs to be integrated into a broader system to make institutional policies feasible.

• 4.

  Although the model uses NPMI to assess the semantic relationship between the course and the professor, no qualitative assessments by human experts were incorporated to assess the relevance of the assignments. Assignments may be technically correct, but not aligned with the professor’s academic growth interests.

• 5.

  It is assumed that the texts used are in the same language. Texts with errors and multilingual data may impact lemmatization and, consequently, embeddings. Future implementations should incorporate processes to detect and normalize the language automatically.

The dataset, comprising 42 courses and 35 professors, was used to demonstrate the feasibility and compare the performance of different assignment strategies. However, it should be recognised that the data is limited by the specific academic model in which the study was conducted. Institutions with different academic organisations, numbers of professors, course offerings, and cultural contexts may have different allocation strategies. For this reason, we emphasize that subsequent studies should consider expanding the dataset to include multiple degree programs, various institutions, and intercultural and multilingual contexts in order to validate the generalization of the proposed model fully.

Future directions include expanding the dataset across diverse academic institutions and cultural settings to improve model generalizability. Incorporating teaching load constraints and availability will be essential for practical deployment. Exploring more advanced embedding models like E5-large, InstructorXL, or GPT-4 embeddings could enhance semantic representation quality. Implementing automatic language detection during preprocessing is also recommended to improve performance in multilingual environments.

While speculative, integrating this NLP-based assignment model into institutional decision-making systems can automate and personalize academic planning at scale, offering substantial improvements over conventional, manual processes.