Accessible interactive learning of mathematical expressions for school students with visual disabilities

Screen readers technology

Screen reader technology converts the on-screen math text into synthesized speech. Despite its importance, screen reader technology serves limited purposes when it comes to interpreting and presenting mathematical expression structure (da Paixão Silva et al., 2017). There are many mathematical symbols, such as superscripts, subscripts, and specialized characters, that pose a major challenge to screen readers because they may have difficulty conveying these elements accurately, resulting in incomplete or inaccurate auditory presentations (Maćkowski et al., 2018). As a result, screen reader output does not capture the hierarchical and spatial relationships inherent in mathematical expressions, such as fractions or matrices, which depend on the spatial layout to be understood (Miesenberger et al., 2022). There are challenges associated with tools that convert MathML and LaTeX into accessible formats, such as formatting inconsistencies and incomplete support for advanced mathematical expressions (Greiner-Petter, 2023; Klingenberg, Holkesvik & Augestad, 2020). These tools are Mathjax, NVDA With Math player, JAWS, Voice over, and Speech Rule Engine (SRE) (Mejía et al., 2021a). However, those tools offer the straightforward generation of Math-ML representations of the mathematical formulas in the formats of MathML, but they have some disadvantages, for instance, showing the structure of the document and having changeable navigation (Maćkowski et al., 2023b). As a consequence, this study proposed a solution to address these challenges to making mathematical expression more accessible, present the structure overview of math expressions and provide flexible navigation to understand mathematical expressions.

MathML and LaTeX-based tools

MathML is one of the W3C recommendations that offer a clear representation of a mathematical notation, as well as making the notations machine-readable, so a screen reader can translate it into speech or braille (Manzoor et al., 2019). The conversion of MathML and LaTeX-based content into accessible formats requires intricate formatting steps. According to Raman & Gries (1995), Soiffer (2005), Stevens & Edwards (1994), Math Player converts MathML content into a voice for the browser. Although Math Player is useful, it has limitations, such as limited navigation and manipulation features, mobile support, and platform support (Al-Razgan et al., 2021). Wang et al. (2021) also describe converting LaTeX and TeX code to MathML. However, MathML and LaTeX-based tools can present accessibility challenges due to formatting requirements (Schmitt-Koopmann et al., 2024).

Using MathML support in screen readers, students with visual disabilities can perceive mathematical concepts in a machine-readable format by converting MathML code into speech or braille (Miesenberger et al., 2022). In addition to syntax highlighting and audio feedback, students can use a MathML editor with accessibility features to create and modify MathML code (Frankel, Brownstein & Soiffer, 2017). Additionally, LaTeX-to-MathML conversion tools address the challenge of existing LaTeX content by converting it into accessible MathML format (Stamerjohanns et al., 2009). Interactive platforms that support MathML provide these students with enhanced learning opportunities, increased independence in math education, and better access to mathematical content (Brzostek-Pawłowska, Rubin & Salamończyk, 2019). MathML and LaTeX are used to create these tools (Greiner-Petter, 2023). However, MathML still faces challenges such as limited adoption by educators and publishers, possible complexity of MathML code, and the need for ongoing development to keep pace with technological advancements and evolving pedagogy (National Research Council, Division on Engineering and Physical Sciences, Board on Mathematical Sciences and Their Applications, Committee on Planning a Global Library of the Mathematical Sciences, 2014). There are several directions underway in the future, including standardizing and integrating MathML, developing user-friendly MathML authoring tools, and developing pedagogical approaches to maximize the learning potential of these tools (Thai, Bang & Li, 2022). Our proposed solution offers technological approaches to this limitation to assist students with visual disabilities in learning mathematical expressions.

Braille and E-book formats

Math contents can be presented in braille through refreshable braille displays which enable students with visual disabilities to have touch-friendly content (Russomanno et al., 2015). However, many of these displays can only display simple mathematical expressions and require braille proficiency, so they are not widely used (Brzostek-Pawłowska, Rubin & Salamończyk, 2019). Students with visual impairments can access literary content more easily with the DAISY format, which has structured navigation, text-to-speech, and compatibility with assistive technology. For inclusion education and equal access to literature, irrespective of print disabilities, it is essential to address challenges and spread awareness of the DAISY format (Akbar et al., 2024). The DAISY format, however, is limited in its ability to display complex mathematical notation, navigate equations seamlessly, and provide tactile representation, which requires additional assistance (Ali & Khusro, 2024). Our proposed solution is user-friendly, requires no additional training, and is cost-effective.

Computer algebra systems

Mathematical problems, especially algebraic ones, can be solved with advanced tools like computer algebra systems (CAS), but they can be difficult to access for students with visual disabilities (Mejía et al., 2021b). Complex mathematical computations are handled efficiently by popular CAS platforms like Mathematica, Maple, and Maxima (Shingareva & Lizárraga-Celaya, 2010). Further, these systems generally lack navigation capabilities, have inaccessible interfaces, have steep learning curves, and have inaccessible graphic outputs. Users with severe visual disabilities are often unable to access their outputs. Similarly, IrisMath, a web-based computer algebra system, addresses mathematical problem-solving needs but faces its own set of challenges (Zambrano et al., 2023). Input mechanisms are complex, and navigation and analysis features are limited, making it hard to use. Although Mathematica, Maple, Maxima, and IrisMath are powerful computational tools, their usability and accessibility issues make them unappealing to a wide audience (Maćkowski et al., 2023a).

AI-based tools for math accessibility

The challenge of mathematics for students with visual impairment persists especially in teaching-learning environments that are heavily invested with visualization (Lemessa, 2023). However, the landscape of these technologies changed in recent years due to new advancements in artificial intelligence (AI) and large language models (LLMs) which seek to provide new solutions that could improve accessibility (Kshetri, 2024). Computer vision technologies including optical character recognition (OCR) and speech-to-text translation can fill the existing gap left by most learning methods between conventional and innovative experience-based approaches (Sakshi & Kukreja, 2024). OCR technologies assist in digitizing texts of mathematical print so that the screen readers can easily translate them and make them more easily navigable for a student; speech recognition allows students to solve math problems verbally making the learning process more engaging (Luo et al., 2024).

Similarly, current state-of-art LLMs like OpenAI’s GPT-4, have shown the ability to both generate and comprehend mathematical content. It must be noted that some of these models can turn mathematical computations into the normal language to enhance learning for these students (Collins et al., 2024). In addition, LLMs like ChatGPT can solve math problems in several steps, come up with a second version of the explanation or explanation in their own way, and adapt the answers to a particular learner’s needs and preferences, which makes them useful tools in learning personalization contexts (Alto, 2023). But despite that, there are disadvantages of LLMs including interaction with non-text content, high wordiness, and accessibility issues for users with disabilities via screen readers and braille devices displays (Kuzdeuov et al., 2024; Sohail et al., 2023). Despite their potential, artificial intelligence-based tools still have critical issues in interaction and navigation, limiting their effectiveness (Mejía et al., 2021a). These limitations are mitigated in our proposed system as it provides non-redundant interaction and compliance with screen reader-based interfaces making for a far more efficient experience for the students with visual disabilities.

Analysis of tools and studies supporting mathematical expressions

To make mathematics accessible for students with visual disabilities, several technological tools are available, however, they differ in their features and accessibility. Our proposed solution is compared to existing tools in Table 1 below, emphasizing key features such as structural overview, elements identification, and navigation.

Table 1 comparing the tools demonstrates that although current technologies offer different degrees of assistance when it comes to navigation, structure overview, and comprehension of mathematical equations, there is a lack of an effective, easy-to-use system that takes into account the multiple needs of students with visual disabilities.

Altogether, math content is complex and notated so screen readers cannot accurately convey spatial and hierarchical relationships. Conversion tools for MathML and LaTeX have challenges in formatting, navigation, and platform support (Maćkowski et al., 2023b). Although MathCAT is an interactive tool, it suffers from compatibility problems and lacks the flexibility to solve complex problems. Despite providing tactile access, braille displays and e-books do not support complex mathematical expressions and require braille proficiency (Neuper, Stöger & Wenzel, 2022). Due to their complex interfaces and outputs, computer algebra systems such as Mathematica and Maple are largely inaccessible. In comparison to our proposed solution, which incorporates, interactive navigation through keyboard input with speech feedback, collaborative support, semantic parsing of mathematical expressions, and broad accessibility via Python-based implementation, the existing multi-sensory augmented reality (AR) system is limited by region-specific braille inconsistencies, reliance on specialized hardware, lack of collaborative functionality, absence of semantic structuring for mathematical content, and limited empirical validation (Mikułowski & Brzostek-Pawłowska, 2020). The proposed solution addresses this issue and designed an algorithm for presenting a structured overview of mathematical expression with a flexible navigation key for comprehension to students with visual disabilities.

Architecture of the proposed solution

There are five essential modules in the proposed solution, the first converting mathematical expressions into content MathML. The second module parses the MathML content, the third module extracts semantic information, and the fourth module creates interactive keys. In the fifth module, the output speech is generated. Students with visual disabilities can use the solution to gain a deeper understanding of mathematical expressions’ actual meaning and structures. This makes it easy for users to navigate within the expression. This process involves making the proposed solution compatible with assistive technology such as screen readers, audio feedback, and results. Figure 1 provides a schematic explanation of the proposed solution.

XML tree construction

The parsing process was initiated utilizing SymPy, a Python library intended for symbolic mathematics. To convert MathML representations into SymPy expressions, the sympy.parsing.mathml module used the parse_mathml() function. This function understood and correctly transformed the supplied MathML structure into a symbolic format that allowed SymPy to perform additional mathematical operations and analysis. This conversion then changes the input into an XML tree form where the form elements are placed in a hierarchy based on a tag such as, <math>, <apply>, <plus>, and other operators of the expression as shown in Fig. 2. The following are the tree representations of expression.

(1) $${\displaystyle \frac{1}{x}+x^2.}$$

• <math> is the root node, which contains a single apply> element.

• The <apply> element has three children: <plus>, another <apply> (for the fraction), and yet another <apply> (for the exponent).

• The first child <plus> has no further children, making it a leaf node.

• The second <apply> (representing the division) has two children: <cn> (with the value 1) and <ci> (with the variable x).

• The third <apply> (representing the power operation) has <ci> (the variable x) and <cn> (the exponent 2) as children.

Design algorithm

The following are the steps of the designed algorithm.

The algorithmic flow for parsing mathematical expressions in Content MathML is depicted in Fig. 3. The following is the step-by-step procedure of the whole process, such as mapping an expression, parsing through the XML structure, getting semantic parts of the expression like terms and variables and last but not least providing navigational and auditory assistance.

The algorithm’s complexity is in three cases: best, average, and worst. In the best case, the input expression is simple and requires minimal operations, resulting in a time complexity of O (1). The average case depends on the size and complexity of the input expression, with nested loops and regular expression operations contributing to a time complexity of O (n). The worst-case scenario is a large and complex input expression, resulting in a time complexity of O (n²) due to numerous nested loops and the use of multiple lists to store intermediate results, which can also lead to increased memory usage.

Design navigation keys

The function of interactivity with the help of keys in the navigation was designed for the user with visual disabilities. With the help of the ‘keyboard’ library, and “pyttsx3”, library was able to handle ‘keypress’ types that let users easily move through mathematical equations. These keys enabled users to traverse the expression level, term level, term relation level as well as level of factors, constants or variables. When commands were entered into the system, it read back the needed piece of information in a spoken format. Features such as keyboard control, labelling for screen readers, logical flow of navigation, and simple auditory descriptions were maintained according to WCAG 2.1. A more detailed list of key functional areas within the detailed navigation is included in Table 2.

The designed keyboard shortcuts provide a streamlined user experience, ensuring predictability through consistent Ctrl use and ease of navigation with left-right arrows. Adaptability offers shortcuts that fit diverse needs, while clear functions make task execution intuitive.

Output speech

The objective of this phase is to provide real-time speech feedback to enhance user understanding and engagement. Math expressions were converted into speech using a Text-to-Speech (TTS) engine, specifically the Python library gTTS (Google Text-to-Speech) navigation keys allowed users to explore individual terms, factors, and more with iterations through the whole expression, as well as adjust speech output settings.

Implementation of the proposed solution

The proposed was implemented using Python, in conjunction with the SymPy library which served as the core tool for symbolic mathematics. Additional XML parsing libraries were also used for traversing the Content MathML structure. With the implementation, users with visual impairments will be able to access the system using a screen reader and also text-to-speech facilities.

Use case of the proposed solution

Students with visual disabilities can learn mathematical expressions with the proposed system using this use case. Through auditory feedback, the system allows students to navigate through an interactive learning environment with flexible navigation based on mathematical expressions converted into Content MathML.

Actors: Students with visual disabilities are the primary actors in this use case, while educational instructors and support staff are secondary actors.

Preconditions: Students must possess a foundational understanding of assistive technologies and install the proposed solution on their devices.

Post conditions: Students gain an enhanced understanding of mathematical expressions after successfully interacting with the system.

(2) $$\mathrm{T}\mathrm{h}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{f}\mathrm{l}\mathrm{o}\mathrm{w}\mathrm{o}\mathrm{f}\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}2x+4=0\mathrm{i}\mathrm{s}\mathrm{a}\mathrm{s}\mathrm{f}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{w}\mathrm{s}:$$

a. System initialization: The student power on the application. The auditory interface informs the client concerning the options of the functional capabilities, as well as how commands may be made.

b. Input of mathematical expression: The student types in the linear equation in the form 2x + 4 = 0 using a keyboard or a braille reader instead of a mouse. The system translates the given input into an intermediary form called content MathML.

c. Parsing the expression: The system will first analyze the equation to identify the semantic components of the equation such as 2x, the constant 4, and the relational operator =.

d. Interactive navigation: With an optimized keyboard key, the student navigates through the equation. The system provides vocal feedback for each term:

“Term 1: Two times x.”

“Term 2: Plus four.”

“Relational operator: Equals zero.”

e. Understanding structure: The student requests elaboration on 2x. The system responds with a detailed explanation: “Two times x means the coefficient 2 is multiplied by the variable x.”

f. Modifying expressions: The student independently chooses the coefficient 2 and substituted it by 3. The system updates the equation in real time and reads the updated expression: “Three times quantity x plus four amounts to zero.”

Participant’s demography

The study was conducted with 15 instructors eight male and seven female and 94 students 50 male and 44 female from government institutes for the blind at Peshawar, Abbottabad, and Swat Khyber Pakhtunkhwa Pakistan. The study involved students of both genders aged between 11 and 20 years and was focused on the grade 6 to 10 students. The participants were chosen deliberately to comprise a good sampling of the target group of students with visual disabilities. The sample size was deemed sufficient to assess the solution’s efficiency while addressing the concerns regarding the feasibility of the study and statistical significance. Demographic data are provided in Table 3.

Research design

The solution was assessed by the Technology Acceptance Model (TAM) by Davis (1989) and this model measures the level of technology acceptance with perceived usefulness and perceived ease of use. The TAM framework is especially appropriate for this study since it is used to delineate its acceptance by the students and instructors who are part of the impaired vision population. Questionnaire data were collected concerning user experiences, and the TAM-based questionnaire used was reviewed and developed in consultation with a subject-matter expert in the field to ensure that the collected data are reliable. The reliability of the self-developed researcher-administered questionnaire was determined using Cronbach’s alpha, which established internal consistency.

Potential participants were explained the purpose of the study and any participant interested in participating in the study duly signed a consent form. After a 15-day experimental immersion in the proposed solution, they responded to the questionnaire, and the results showed a comparison to existing similar tools like NVDA and Access8Math.

Evaluation criteria and process

The proposed solution’s effectiveness was assessed based on the following criteria:

Accuracy: Tested participants’ knowledge on how to label terms, factors, constants and variables in mathematical expression.

Usability: Based on a TAM-based survey, found out the perceived ease of use (PEOU) and perceived usefulness (PU).

Comprehension: Evaluated according to respondent’s explanations and suggestions given after interaction with the system.

Navigation Efficiency: Controlled the time that it took and the number of mistakes made when trying to navigate through equations using an icon-like keyboard.

Random participants were supposed to use the specially designed system, so they went through the procedure to become trained with the system before the evaluation. They then applied the proposed solution on mathematics problems of different degree of difficulty and examined the results that came from it and other facilities like NVDA and Access8Math. Collected data measures included questionnaire data in the form of performance logs and error rates paired with interviews. Objective data related to various problem-solving levels were statistically examined using the ANOVA test, while the qualitative data were analyzed using the thematic analysis in order to establish usability problems and to provide recommendations.

Data collection and analysis

Measurement and data collection included accuracy measures, the time taken to complete the tasks, the rate of mistakes, surveys, and interviews with the participants. Descriptive statistics were used in analyzing quantitative data to determine if there was a significant difference in performance with increasing problem difficulty and open-ended responses were coded to generate themes by usability and participant recommendations.

Ethical considerations

Each participant was given information about the study and its aim, methods, and possible dangers involved in the study, and each of them signed an informed consent before they could participate in it. To ensure the security of the participants, their data was only shared between the researcher and the participant and all data was kept confidential. The research conformed to ethical guidelines for data collection that were provided by the University of Peshawar, Pakistan institutional review board reference No. 105 dated 15^th May 2024.

Quantitative analysis

Accuracy

The effectiveness of the proposed solution was evaluated based on its capacity to navigate through terms, factors, constants as well as variables of the mathematical expressions and formulas. From the results obtained it became clear that the proposed solution had an accuracy of 85% which compared well with 70% of NVDA and 65% of Access8Math as shown below in Table 4 and Fig. 4A. This can be evidenced as significant progress in the proper interpretation of mathematical context which will help in the advancement of improving learning among the students with visual disabilities.

Table 4:

Comparison of the proposed solution with NVDA and Access8Math.

Tool	Accuracy (%)	Time (min)	Error rate (%)	Usability score (1–5)
Proposed solution	85	10	5	4.5
NVDA	70	15	15	3.5
Access8Math	65	20	20	3.0

DOI: 10.7717/peerj-cs.2599/table-4

The results of the assessment of the accuracy show that the suggested approach offers higher recognition of mathematical characters as compared to the state-of-the-art tools enhancing the learning process for students with visual disabilities.

Qualitative analysis

Qualitative feedback was collected from instructors and students to gain insights into their experiences with the proposed solution. This data highlights the tool’s unique contributions to navigation, comprehension, and contextual analysis of mathematical expressions, in comparison to existing tools like NVDA Screen Reader and Access8Math. The qualitative data were categorized into key themes, including Ease of Navigation, Comprehension, Contextual Analysis of Mathematical Expressions, Comparison with NVDA and Access8Math, and Areas for Improvement.

• Ease of use: Several participants mentioned the proposed solution’s user-friendly interface and accessibility features. For instance, one instructor shared, “The proposed solution is simple to navigate, even for students with limited experience in technology. It was easy to integrate into our teaching routines”. A student commented similarly “Similarly, a student remarked, “I found it straightforward to access the lessons and solve problems without needing constant guidance”. The feedback from this suggests that the adoption was indeed helped by the intuitive design. But a few students suggested that an introductory tutorial or user manual for first time users would be appropriate.

• Improved focus and comprehension: The proposed solution was used both by instructors and students found a big improvement in both engagement and in understanding content. An instructor stated, “This solution provides a complete learning experience by integrating navigation and analysis of math expressions, something the other tools don’t offer in the context of mathematical learning”. Another comment from a student highlighted how the solution helped bridge gaps in their learning: “The proposed goes beyond just reading; it helps me navigate, understand, and analyze mathematical expressions, which makes learning math much easier compared to NVDA and Access8Math”. Other students also highlighted “The proposed solution helps me understand what the equations mean, not just read them”. These responses align with the quantitative findings of improved performance metrics.

• Areas for improvement: The overall feedback was positive, but some challenges were identified. Students expressed initial difficulty adapting to the tool’s format, with one student stating, “I needed extra time to get used to this new system.” Instructors suggested additional training sessions or technical support might help ease the transition. For example, one instructor proposed, “A brief training module for teachers and students would make the implementation smoother”.

The qualitative feedback shows that the proposed solution is capable of enhancing the performance of navigation, comprehension and contextual analysis over the already existing tools as stated in the literature review and also supports the mathematical learning of the visually impaired students. Participants noted the directions for improvement, yet there are plenty of innovative concepts regarding the use of the tool in a bigger picture. This study is an important contribution to accessible education, introducing a new tool to bridge the gap in learning mathematics for visually impaired students while acting as a consolidation of usability and comprehension concerns. Unlike other assistive technologies, this proposed solution is unique in targeting mathematical comprehension. At the very beginning, adaptation faces challenge and this is where robust training modules need to be for scalability.

Statistical evaluation

The instructor and student question responses were analyzed by Analysis of Variance (ANOVA) which yielded significant differences in mean responses. With regard to the instructor data, the p value was as follows 0.00000022; this gives a picture where the means of at least two groups are significantly different with an F value of 11.22 and degrees of freedom of 6 for between and 98 for within group variations. Likewise for the student data, test of between-subjects effect indicated the ANOVA test yielded an F value of 2.94 and a p-value of 0.00638, much less than 0.05 thereby suggesting significant between-group differences in the means. These results have enabled me reject the null hypothesis in both the circumstances a circumstance. According to the results obtained, 91.43% of instructors and 83% students perceived the proposed solution as easy to use, thus concluding that PEOU was high. But a few of the students complained that they encounter some difficulties in using the solution, and some said that it does not greatly improve the amount of time it takes for them to learn mathematics.

Discussion

As per the findings of this research, the proposed solution is quite beneficial in the enhancement of mathematical learning to the students with visual disabilities. The proposed solution has achieved a higher success rate of 85% when compared to the present tools NVDA-70% and Access8Math-65%. By improving the visibility and influencing the way that traditions refer to mathematical expressions, it improves learning.

Usability feedback was positive, with 91.43% of instructors and 83% of students finding the proposed solution beneficial and easy to use, supported by high internal consistency (Cronbach’s alpha: 0.81 and 0.91). The overall mean estimate of effect size was 0.94 for the instructor group and 0.91 for the student group. The system also enhanced task completion and reducible error rates, providing a faster and less erroneous way to engage with material.

Finally, combining qualitative and quantitative data gives a view of the proposed solution effectiveness. Measurable improvements in learning outcomes were demonstrated with quantitative data, and qualitative insights revealed the experiences for users, such as improved navigation, comprehension and engagement. Taken together, these findings show that the tool fills in gaps with traditional assistive technologies and provide useful guidelines for future improvements. This study focused on algebra within a specific context; future research should explore other mathematical domains and settings. A reliable, efficient, and portable solution for teaching mathematics to visually impaired students, the proposed solution could be expanded with future advancements and made compatible with other adaptive technologies.