Design and evaluation of an e-governance ICT solution to empower the non-literate vegetable market consumers

Universal design guidelines and considerations

Recent surveys have synthesized decades of research on user interface design for people without literacy skills and semi-literate populations. For instance, Islam et al. (2023) conducted a systematic literature review encompassing 45 studies focused on designing interfaces accessible to users with limited or no literacy. From this body of work, they identified 16 recurring design considerations, which they categorized into five thematic areas: text usage, interaction patterns, pictographic or multimedia integration, audio and video implementation, and information architecture.

Among the most frequently cited design principles were the following:

Minimize or eliminate textual content: since many people without literacy skills are unable to read, systems should aim to be entirely “text-free” or use minimal, contextually relevant language. Eight of the reviewed studies explicitly advocated for interfaces devoid of text, while others recommended using only localized languages when necessary.

Incorporate audio and speech-based features: nearly half of the studies (20 out of 45) integrated voice input or output in some form. Voice prompts, spoken menus, and auditory navigation were consistently highlighted as critical components for usability. Interfaces employing voice-based modalities such as voice-guided instructions or spoken feedback were found to significantly enhance intuitiveness. For example, farmers in India were able to access real-time market prices by interacting through voice commands in their native language (Somnath & Sunil, 2022).

Utilize pictorial and iconic elements: all reviewed systems employed visual components such as images, icons, or photographs. In every case analysed by Islam et al. (2023) graphical elements including sketches, illustrations, and photographs, were integral to the design. Research repeatedly demonstrates that such visuals often convey meaning more effectively than text. Notably, hand-drawn sketches were found to be more comprehensible to people without literacy skills compared to abstract icons or photographs (Khan et al., 2017; Rahman, 2019). Moreover, cartoon-like or culturally familiar imagery was shown to improve interpretability over generic stock graphics. Cultural relevance emerged as a key factor; designers were advised to use imagery aligned with users live experiences and environments (Rahman, 2019; Shah et al., 2015).

Prioritize local language and mnemonic aids: when textual or auditory content is included, it must be presented in the user’s native language (Islam et al., 2023). One study reported the successful use of mnemonic pictograms linked to familiar local cues such as a village landmarks (Rahman & Fukuda, 2015). This use helped in assisting people without literacy skills in self-identification without reliance on written text.

Taken together, these findings support the development of multi-modal user interfaces, combining spoken prompts with intuitive icon-based or animated menus, while minimizing or eliminating textual instructions. However, authors across the literature caution against overloading interfaces with excessive multimedia features, noting that overly rich designs can lead to confusion (Rahman, 2019). Instead, the ideal interface is described as clean, simple, and purposeful, with each screen presenting few items and clearly defined navigational signposts (Islam, Ahmed & Islam, 2020). Furthermore, user involvement through participatory design methods is emphasized throughout the literature. Engaging people without literacy skills directly in iterative prototyping cycles has been shown to surface critical usability issues and result in more effective, user-centred designs.

In summary, prevailing design guidelines converge on two core recommendations: the use of audio-visual interfaces and continuous usability testing. Restyandito & Chan (2015) further noted that people without literacy skills may exhibit differences in cognitive processing, including reduced memory capacity and challenges with abstract reasoning. As such, interfaces should avoid overwhelming users with excessive choices or demands on recall. Preferred approaches include direct manipulation and linear interaction flows, supported by multimodal cues such as voice, imagery, and sound. Ideally, interfaces should require little to no formal training, enabling users to learn through exploration and minimal orientation.

Voice and speech interfaces

Speech represents a natural and accessible interaction modality for individuals with limited or no literacy. Several studies have explored the use of voice-driven systems to bridge the digital divide. For example, a spoken dialogue system was developed in India to enable farmers to inquire about real-time commodity prices using voice input (Somnath & Sunil, 2022). Users could articulate their queries in Hindi or regional languages and receive vocal responses, thereby eliminating the need for any reading ability (Restyandito & Chan, 2015).

Research consistently highlights the potential of voice-based user interfaces (VUIs), including interactive voice response (IVR) systems and mobile voice menus, to make digital services accessible to populations lacking formal literacy. However, several challenges remain, particularly in rural settings where background noise can affect speech recognition accuracy and where users often rely on low-end mobile devices. Despite these limitations, pilot implementations have demonstrated promising results: field trials indicate that farmers are able to effectively retrieve market prices through spoken queries.

The application of voice-based interfaces extends beyond agricultural contexts. Traditional text-based communication tools, such as SMS, have also been reimagined to accommodate people without literacy skills. For instance, Iqbal (2021) redesigned an SMS application by replacing textual elements with audio recordings and graphical icons. In usability testing, participants found this audio-visual interface significantly more usable than a conventional text-based SMS app. The system allowed users to listen to voice clips and select messages via pictorial menus, enabling them to send and receive communications without typing. All participants expressed a strong preference for this “text-free” design. Islam et al. (2023) observed that nearly 44% of the systems reviewed in their systematic analysis incorporated voice input or output functionalities, underscoring the widespread adoption and perceived effectiveness of audio-based modalities in low-literacy contexts.

Based on such findings, best practices for designing effective voice-user interfaces include:

• Using simple, clear spoken prompts at each step of the interaction.

• Incorporating fallback mechanisms such as options to repeat instructions or slow down playback.

• Providing auditory feedback for every user action.

• Delivering content in a familiar dialect or accent to enhance comprehension.

• Complementing audio with touchscreen buttons or soft keys labelled with intuitive icons, so that auditory instructions and visual symbols reinforce one another.

Graphical and pictorial interfaces

Where voice-based interfaces alone are insufficient, graphical elements serve as essential complements. User interfaces designed for people without literacy skills frequently employ icons, symbols, photographs, and sketches in place of textual content. Instead of traditional text-based menus, users are presented with grids of images that represent available actions or categories. For example, a job-search website for people without literacy skills used hand-drawn illustrations of different workplaces, such as farms, shops, and offices to classify job opportunities rather than relying on written descriptions (Khan et al., 2017; Rahman, 2019). At each interaction step, users simply selected the image that best matched their intent by tapping on it.

Empirical studies emphasize that both the choice and style of visual representations significantly affect usability. Research indicates that simple black-and-white line drawings often outperform detailed photographs, as they eliminate extraneous visual complexity and focus attention on core meaning (Khan et al., 2017; Jan et al., 2020). Animated icons or short video sequences i.e., an outline of a person performing an action can further enhance intuitiveness.

Some designs draw inspiration from familiar mobile phone paradigms. For instance, visual phonebooks have been implemented where each contact is represented by an image of the person, or menu layout resemble TV channel guides (Hayat, Rextin & Bilal, 2021). In a review of SMS user interfaces tailored for people without literacy skills, researchers found that graphic elements produce favourable outcomes particularly when combined with audio support (Iqbal, 2021).

However, it is important to recognize that icons are not universally interpretable. Cultural context plays a critical role in determining how visual symbols are understood. For example, an icon representing “medicine” might be depicted as a red cross in one country but may signify “hospital” in another (Kalyan, Rani & Shouri, 2024). Therefore, many studies stress the importance of co-designing interfaces with target communities to ensure cultural relevance and clarity.

In one e-commerce project targeting shopkeepers in Pakistan, researchers conducted ethnographic studies to identify locally meaningful imagery. The resulting interface included simple sketches such as a gas stove to represent cooking fuel, or a mango to symbolize fruit vendors, rendered in a style familiar to local users (Khan et al., 2016). The system was touchscreen-based to simplify input, and incorporated audio guidance in Urdu and Pashto. As a result, users were able to navigate and make purchases independently, without requiring external assistance.

In summary, pictorial user interfaces play a central role in making digital systems accessible to people without literacy skills. Studies recommend the use of “icons, digital photographs, and hand-drawn sketches with audio/video support” as preferred modalities. Data consistently show that text-free layouts, especially those featuring large, clear icons, significantly improve usability compared to any text-only design. When textual elements must be included, they should be displayed in large font size, presented in the user’s native language, and accompanied by audio cues for clarity and accessibility.

Interaction and navigation patterns

Beyond content representation, interaction style plays a crucial role in the usability of digital interfaces for people without literacy skills. Research indicates that such users perform most effectively with simple, linear navigation structures and large interactive elements. Chaudry et al. (2012) conducted an evaluation of various graphical user interface (GUI) widgets with low-literacy adults in a health-related context. The study found that users could interact successfully with non-textual elements such as icons and images. However, they achieved the highest efficiency with large radio-button-style selectors. Furthermore, participants preferred a shallow, linear navigation flow. Such flow is analogous to progressing sequentially through pages rather than navigating through deep multi-level menu systems. These findings underscore the importance of uncluttered screen layouts and step-by-step guidance in interface design for this user group.

Mobile interfaces often avoid complex interaction patterns. For instance, an SMS application tailored for people without literacy skills replaced scrolling text with a hierarchical picture-based menu system. Similarly, banking ATMs designed for low-literacy populations have adopted button-driven navigation flows, where each screen presents a limited set of pictorial options. Subsequent choices are based on prior selections, thereby minimizing cognitive load and simplifying decision-making (Cremers, de Jong & van Balken, 2008).

Touch-based input methods such as tapping large buttons or icons on a touchscreen are universally favoured over typing or executing complex gestures. Islam et al. (2023) observed that touch-button navigation was a common feature across mobile user interfaces targeting people without literacy skills, while menu-based designs were generally avoided due to their complexity. When menus cannot be eliminated, labelling strategies and interface constraints significantly improve usability. For example, in the design of an e-government portal for low-literacy users, screens were limited to a small number of topic icons accompanied by voice labels, and advanced search fields were disabled to prevent confusion. In another study involving ATM interface design, the use of standardized symbols such as a money icon for cash withdrawal and spoken prompts led to a substantial reduction in user errors (Rahman, 2019).

Finally, error recovery mechanisms are essential in interfaces for people without literacy skills. When a user accidentally selects an incorrect icon, they may not be able to read a textual “undo” option. To address this, some systems incorporate safety net features, such as visual highlights, auditory feedback, or confirmation sounds upon selection. Others require a double-tap action for irreversible operations. While few systems fully implement such safeguards, the literature identifies these as best practices and notes their absence in many existing solutions.

User-centred design methods

User-centred design methodologies have been consistently shown to enhance the effectiveness of interface development for low-literacy populations. Iterative prototyping and participatory design practices allow designers to identify and resolve usability issues at early stages (Gulliksen et al., 2003). A substantial body of literature underscores the critical importance of participatory design involving target users, particularly when designing interfaces for people without literacy skills or semi-literate populations. Given the diversity among such populations in terms of culture, age, and prior experience with technology, designers are advised to begin by conducting field studies or interviews to elicit user needs and preferences.

For instance, Islam, Ahmed & Islam (2020) conducted focus group discussions with 40 people without literacy skills and semi-literate participants in Bangladesh to understand their requirements and usability expectations. From these insights, the researchers identified a set of core design principles such as the use of native language interfaces, voice prompts, symbolic icons, and minimal textual content. Similarly, Iqbal (2021) engaged SMS users without literacy skills in comparative evaluations of text-based vs icon- and voice-enhanced user interfaces. The findings revealed a strong preference for the icon-driven interface, further reinforcing the value of multimodal, non-textual designs. Other projects have employed ethnographic methods, such as observing how villagers recognize local landmarks to inform the design of culturally relevant icons (Rahman & Fukuda, 2015). Similarly, Somnath & Sunil (2022) studied farmers’ terminology for agricultural commodity prices to shape the structure of a voice-based dialogue system. These examples illustrate that direct user involvement often leads to valuable insights regarding visual legibility, auditory clarity, and the feasibility of required training.

In addition to qualitative feedback, researchers frequently employ standardized usability metrics to measure effectiveness. These metrics include task success rate, error frequency, and task completion time. For example, Rahman & Fukuda (2015) while evaluating an e-learning application found that 13 out of 15 participants without literacy skills successfully completed a given task using an improved user interface, compared to only eight out of 14 using the baseline version. This improvement translated into a 26% reduction in average task completion time, demonstrating the tangible benefits of tailored, user-centred design. Such empirical evaluations, often based on direct observation of people without literacy skills interacting with prototypes, are essential for validating design decisions and ensuring real-world applicability.

State of the art

Despite multitude of research, a handful of systems attempted to present market price information via voice or image processing. Some examples include a MIS (Mantena et al., 2011), Fruit/Vegetable Recognition (Muhtaseb, Sarahneh & Tamimi, 2012), and Vegeshop (Sakai et al., 2016). Nevertheless, linguistic limitations (e.g., MIS’s use of Telugu), poor background handling in image recognition, and the absence of non-textual feedback often hinder them.

None of these applications address the complete e-governance interaction loop, such as filing complaints, tracking responses, or ensuring accountability. This omission is particularly problematic, as information access without enforcement mechanisms perpetuates power asymmetries rather than resolving them. There is also limited discussion on scalability, particularly how these systems can be adapted to diverse regions with limited infrastructure and device variability. An overview of the limitations of the existing work can be viewed in Table 1 below.

Study procedure for icon assessment

Icons for the FVMHA application were selected through document analysis, drawing from various internet sources and open-source software documentation. This process identified 56 icons representing actions such as face recognition, identification number entry, and audio assistance. These icons and their corresponding action descriptions are detailed in Appendix A.

In addition to the 56 action-specific icons, a cross icon was included to indicate when none of the four primary icons adequately represented an action. The assessment of these icons in the FVMHA application proceeded through interviews conducted in two phases:

• 1.

  Participant selection and profile collection.

• 2.

  Data collection and study procedure.

These phases were executed sequentially to ensure a comprehensive evaluation and refinement of the icon selection process for the FVMHA application, as elaborated below.

Observational study

The selected icons were used to develop a prototype of the FVMHA application to evaluate the following usability aspects:

• 1.

  User preferences for two authentication methods: picture passwords vs text-based passwords.

• 2.

  Differences in usage with and without audio instructions.

• 3.

  Differences in usage with and without image icons.

• 4.

  Ability to identify currency notes.

• 5.

  Natural behaviour when interacting with the built-in camera without instructions.

• 6.

  Natural behaviour when interacting with the numeric input device without instructions.

The design of FVMHA

The FVMHA application is a multimodal solution comprising five modules, with the first four designed to achieve the study’s three objectives and a fifth module serving as an admin panel. The modules and their descriptions are as follows:

Study procedure and data collection

The usability evaluation was conducted using an OPPO F17 smartphone with non-literate participants. The following steps were implemented for each participant:

• 1.

  Participants were informed about the research purpose and their role in the study. Specifically, they were told that the study aims to evaluate the software’s ease of use, not their performance, to ensure they feel comfortable, behave naturally, and provide honest feedback on the FVMHA application’s functionality.

• 2.

  Demographic information was collected from each participant.

• 3.

  Five tasks were designed to assess the functionality of the four main modules, and each participant performed these tasks individually.

Steps 2 and 3 are detailed in the following subsections.

Participants selection

A total of 40 non-literate participants were selected for the usability evaluation, guided by methodological best practices and contextual relevance. Nielsen (2012) suggests that testing with five users can identify approximately 80% of usability issues due to the principle of diminishing returns. However, for complex or diverse user groups, such as non-literate consumers in rural Pakistan, a larger sample is necessary to capture variability in behaviours and needs.

The selection of 40 participants aimed to achieve saturation in usability feedback while ensuring diversity in gender, age, and mobile phone experience within the non-literate population. Participants were purposively sampled from the Mansehra district in Khyber Pakhtunkhwa (KPK), a region with a significant non-literate demographic. The sample included both basic and smartphone users, male and female participants, and individuals with varying levels of numeracy and Urdu language comprehension.

Verbal consent was obtained from each participant and recorded via thumb impressions on individualized consent forms. Participants were informed that they could request data deletion within 1 year of collection. Although this sample size does not ensure statistical generalizability across the entire non-literate population, it represents the study’s target group: non-literate and semi-literate adults in rural Pakistan who face systemic exclusion from digital services. The demographic diversity of the sample informs interface design recommendations by reflecting a realistic spectrum of consumer needs and experiences. A detailed overview of the participants’ demographics is provided in Table 3.

Task selection

All available functionalities of the FVMHA application were presented to participants as individual or combined tasks for execution. These tasks were mapped to the study’s objectives, with Table 4 illustrating the alignment between each task, the corresponding objective, and the rationale for their association.

Parameter selection

The usability of the FVMHA prototype was evaluated based on ISO Standard 9241-11 parameters: effectiveness, efficiency, and user satisfaction. Each session was screen-recorded for subsequent analysis. To establish a baseline for comparison, the effectiveness and efficiency of non-literate participants were compared with those of a literate user. It was hypothesized that a literate user would navigate the simplified interface with perfect effectiveness and greater efficiency. The first author tested this hypothesis by completing all tasks with maximum effectiveness, and their task completion time was recorded as the optimal efficiency benchmark, assuming literate users would perform tasks faster than non-literate or semi-literate participants.

Data collection

The usability of the FVMHA prototype was evaluated using ISO Standard 9241-11 parameters: effectiveness, efficiency, and user satisfaction. Effectiveness was assessed by measuring whether participants completed tasks on their first attempt, with any additional attempts recorded as errors. The following actions were classified as errors for each task:

Task 1 (Login Module):

• Rewriting the National Identification Card (NIC) number.

• Clicking on the admin panel.

Task 2 (Fruit and Vegetable Rate List Access Module):

• Taking an incorrect image (e.g., blurred, unrecognizable by the system), requiring participants to retake the image, is counted as an error.

Task 3 (Gallery Selection):

• Clicking on an incorrect icon instead of the gallery.

Task 4 (Complaint Registration):

• Tapping an incorrect icon instead of the complaint registration icon.

• Recording audio or video more than once.

• Exiting the screen without registering a complaint.

Task 5 (Complaint Tracking):

• Tapping an incorrect icon instead of the complaint tracking icon.

• Failing to recognize their registered complaint or its status.

Efficiency was measured using three parameters: the number of times participants listened to audio instructions, the total time taken to complete each task, and the frequency of requests for assistance. User satisfaction was evaluated using the System Usability Scale (SUS) questionnaire, adapted for non-literate participants through verbal administration in the local language, Hindko.

Participants were assigned a task and instructed to complete it. Upon completion, instructions for the next task were provided verbally in Hindko. A dedicated data recording module was developed to log user interactions with the FVMHA application automatically. This module generated a separate log file for each screen, capturing the following details:

• Current screen (e.g., login interface).

• Type of event (e.g., tap).

• Control interacted with (e.g., button).

• Name of the control (e.g., “Sign In”).

The log recorded the start time when a participant entered an interface and the stop time when they completed the task by interacting with a completion control (e.g., tapping the login button). Unsuccessful attempts, such as failed logins, were logged as error entries for the respective interface. Manual observations by the first author complemented the automated data collection, with details such as the number of button presses recorded in a logbook alongside the corresponding participant and screen.

Results

The results involved calculating the interfaces’ effectiveness, efficiency, and consumer satisfaction with the final prototype. The details of the results are as follows:

Efficiency

Task completion time:

Task completion time analysis, as presented in Table 6, provides insights into the efficiency of non-literate participants’ interactions with the FVMHA prototype. Key findings include:

• Efficiency Gap Across All Tasks: all tasks exceeded their optimal completion times, indicating a learning curve or usability challenges for non-literate participants. Nevertheless, all participants completed every task, reinforcing the prototype’s effectiveness.

• Most Efficient Task–Task 3 (Gallery Selection): Task 3 exhibited the least deviation from the optimal completion time (average: 30 s vs. optimal: 17 s), supporting the “simple is better” principle. This suggests that icon recognition and direct visual access are well-suited for non-literate users.

• Most Challenging Task–Task 4 (Complaint Registration): Task 4 had the highest average completion time (3 min 14 s vs. optimal: 2 min), indicating greater cognitive load or navigation complexity in the complaint registration module. However, the least efficient participant required only 74 s more than the literate benchmark, suggesting that the interface remains accessible, with additional time reflecting cautious, learning-based interactions rather than fundamental usability flaws.

• Task 1 (Login) and Task 2 (Fruit and Vegetable Rate List Access): both tasks showed significant deviations from optimal times, indicating a need for further simplification or enhanced audio assistance at these critical entry points.

• Task 5 (Complaint Tracking): Task 5 showed minimal deviation from the optimal time, suggesting strong usability for information retrieval tasks.

Time-Based Efficiency (TBE) was calculated using the following equation.

$${\displaystyle \frac{{\sum }_{j=1}^R{\sum }_{i=1}^N{\displaystyle \frac{n_{ij}}{t_{ij}}}}{NR}.}$$

Here,

N = total number of tasks

R = total number of users

n_ij = whether user j successfully completed task i (1 = success, 0 = failure)

t_ij = time taken by user j to complete task i.

Efficiency results per task, therefore, can be calculated as follows:

Task 1 = 0.37/40 = 0.009297

Task 2 = 0.48/40 = 0.01219005

Task 3 = 1.43/40 = 0.035795

Task 4 = 0.24/40 = 0.00609

Task 5 = 0.94/40 = 0.02355425.

Overall Efficiency:

By summing the efficiencies for all tasks:

Total Efficiency = 0.0093 + 0.0122 + 0.0358 + 0.0061 + 0.0236 = 0.087 tasks/sec.

This means users completed approximately one task every 11.5 s on average.

Overall Relative Efficiency (ORE):

Total relative efficiency was determined by comparing the task completion times of participants who completed each task with the aggregate completion times of all participants. This metric is expressed through the equation provided below.

$$\mathrm{O}\mathrm{R}\mathrm{E}={\displaystyle \frac{{\sum }_{\mathit{j}=\mathbf{1}}^R{\sum }_{\mathit{i}=\mathbf{1}}^N{\mathit{n}}_{\mathit{i}\mathit{j}}{\mathit{t}}_{\mathit{i}\mathit{j}}}{{\sum }_{\mathit{j}=\mathbf{1}}^R{\sum }_{\mathit{i}=\mathbf{1}}^N{\mathit{t}}_{\mathit{i}\mathit{j}}}\times 100\%.}$$

The total time taken by the 40 non-literate participants to complete all five tasks was 17,352 s. As all tasks were completed, regardless of the number of attempts, the denominator for the relative efficiency calculation was 17,352 s, resulting in a 100% relative efficiency. Figure 2 illustrates that Task 3 (Gallery Selection) had the shortest completion time, while Task 4 (Complaint Registration) had the longest, with both tasks showing comparable variability in completion times.

Number of times a consumer asks for help

An efficient prototype enables participants to complete tasks without assistance. In this study, a “best case” value of 1 was defined, indicating no assistance required. However, non-literate participants requested assistance ranging from 0 to 9 times per task. The average number of assistance requests per task was as follows:

• Task 1 (Login): 1.6 times

• Task 2 (Fruit and Vegetable Rate List Access): 0.5 times

• Task 3 (Gallery Selection): 0.25 times

• Task 4 (Complaint Registration): 2.1 times

• Task 5 (Complaint Tracking): 0.6 times.

On average, participants requested assistance less than once for three tasks (Tasks 2, 3, and 5) and approximately twice for the remaining two (Tasks 1 and 4). These results suggest that the FVMHA interface was clear, intuitive, and accessible for non-literate users, requiring minimal external support.

Satisfaction

User satisfaction, defined as users’ comfort and positive attitudes toward the application (Bose & Dipin, 2012), was assessed using the System Usability Scale (SUS). The SUS comprises 10 questions, with responses ranging from “strongly agree” to “strongly disagree” on a five-point scale. To accommodate non-literate participants, the questions were translated into Urdu and administered verbally (see Appendix C). An overview of the satisfaction evaluation results is presented in Table 6.

Formula for calculating satisfaction from the SUS scale:

Answers to odd number questions = X

Answers to even-numbered questions = Y.

Satisfaction = ((X - 5) + (25 – Y)) * 2.5.

The overall satisfaction score was calculated to be 68.06 with a confidence interval range of 64–72, as shown in Fig. 3. This indicates a threshold of above-average usability as indicated by Brooke (1996).

Effectiveness

All participants succeeded in every task, yielding a 100% overall completion rate. In practice, users seldom needed more than two tries per task, indicating that the interface was learnable and intuitive. This high success aligns with prior work showing that well-designed graphical, audio-enhanced interfaces enable low-literacy users to achieve their goals (Medhi et al., 2011). For example, Medhi et al. (2011) found that a graphical (text-free) mobile UI gave the highest task-completion rates for novice, low-literacy users. Our findings echo this: the simplest, icon-driven tasks (Tasks 3 and 5) were completed nearly optimally, whereas the most complex task (Task 4, complaint entry) required the most attempts and taps. Such patterns are expected given other studies that highlight the challenge of multi-step data entry for non-literate users (Islam et al., 2023). In summary, the interface’s success at getting every user to goal completion, with only a few attempts, indicates high effectiveness, consistent with design guidelines that recommend linear, consistent navigation flows for low-literacy audiences (Parikh, Ghosh & Chavan, 2002; Medhi et al., 2011). Overall, the 100% completion rate and low number of extra attempts indicate that the design effectively supports non-literate users in accomplishing their goals, in line with earlier findings on similar systems (Medhi et al., 2011).

Efficiency (Time-on-task and help)

Every task in the study took longer than its “optimal” benchmark, reflecting the expected learning curve for first-time users. This is consistent with prior research: textual or multi-step tasks typically slow novice users even in well-designed graphical systems (Ahmed et al., 2015). In our data, Task 3 (Gallery Access) was most efficient, averaging ~30 s vs an optimal 17 s. This small gap suggests that simple icon recognition is very effective–a result aligned with the principle that “simple is better” for low-literacy interfaces (Islam et al., 2023). Conversely, Task 4 (Complaint Registration) averaged ~3 min 14 s (optimal 2 min), the longest of any task. This substantial delay likely reflects the cognitive load of entering information; indeed, other studies note that form-filling is the most time-consuming step for non-literate users (Khan et al., 2016). Tasks 1 (Login) and 2 (Price Inquiry) also exceeded optimal times by a significant margin, suggesting that even these basic tasks could benefit from further simplification or audio guidance. By contrast, Task 5 (Status Check) was only slightly above optimal, indicating that straightforward retrieval tasks are quite accessible.

Finally, we observed how often users asked for help. Encouragingly, assistance requests were low: on average <1 help request per user for three of the five tasks, and about two requests for the two most difficult tasks (Login and Complaint). In practical terms, most users managed almost all tasks without needing intervention. This minimal need for assistance indicates that the interface cues (icons, audio in Urdu, clear layout) were generally sufficient. This outcome supports the design philosophy that a properly crafted, text-minimal UI can eliminate the need for external help or training (Islam et al., 2023). Indeed, Medhi et al. (2011) have similarly emphasized providing help (or voice guidance) on-screen, and our results suggest that this strategy was largely effective.

Satisfaction

User satisfaction was good. The mean System Usability Scale (SUS) score was 68.06 (95% confidence interval (CI) [64–72]). According to standard SUS benchmarks, a score around 68 is roughly average, with scores above 68 considered above average (Soegaard, 2024). In other words, our users found the system “ok” to “good” in usability. This matches qualitative feedback: participants reported feeling comfortable with the app and confident in using it, as reflected by the SUS score and comments. For comparison, Kalsoom et al. (2019) found that non-literate participants were “satisfied and motivated” to reuse a similarly designed job-search website. Thus, the satisfaction level we observed is in line with prior studies where thoughtful audio/visual interfaces led to positive reactions from low-literacy users.

Importantly, reaching a SUS score of ~68, just above the “average” cut-off, indicates usability above the threshold of mere acceptability. It suggests that non-literate users felt the system was helpful and not frustrating. Given the novelty of the technology and lack of formal training, this is a strong result. It also leaves room for improvement (e.g., pushing the score into the “good” range >80), perhaps by further refining the hardest tasks. Overall, however, the satisfaction findings reinforce that our core design choices (icons, native-language audio, minimal text) resonate well with users.

Limitations and future work

Some findings from this study may be generalizable to populations with diverse cultural and technological backgrounds. For instance, the use of meaningful, customized icons for buttons and interface elements could be effective across different regions. Similarly, audio and video instructions in local languages may have broad applicability. However, caution is warranted when generalizing these results to broader populations without further validation. Cultural differences, varying levels of technology exposure, and diverse iconographic interpretations may influence usability outcomes in other contexts. Future studies should replicate and extend these assessments across varied geographic, cultural, and technological contexts to enhance the external validity of the findings.

This study has several limitations. First, the fruit and vegetable recognition module included only 15 recognizable items. Expanding the module to include all available fruits and vegetables could enhance functionality but may increase interface complexity, potentially requiring a new study to evaluate its impact on non-literate users (Kalsoom et al., 2019). Additionally, the usability evaluation was limited to 40 participants from a single region (Mansehra, Khyber Pakhtunkhwa). Future research should involve larger and more regionally diverse samples to improve generalizability.