Felix Messo
.png)
Enhancing Conversational Agent For Dyslectic Users
MY ROLE
Usability Testing
UX/UI Design
THE WHAT
THE WHY
THE HOW
DELIVERABLES
Feature Recommendations
Usability Testing
Research
YEAR
2021
We designed 4 new features to be proposed for Hubert's NLP Chat bot, that would increase the accessiblity for dyslectic users and increase the social sustainability within the service. The goal was to increase the user experience for a broader target group.
In recruitment, there are challenges in conducting objective recruitment. That is to say that the recruiter treats all candidates objectively. In recruitment, interview chatbots are starting to be developed to support objective recruitment. After discussing with our client Hubert, we pin pointed that social sustainability and accessibility are beginning to become more important in the digitized world, we therefore chose to evaluate their service from a perspective of social sustainability and overall accessibility.
Since the service we wanted to evaluate was already in production, we started with researching about how dyslectic users are affected by design & information hierarchy. By doing so we could prepare a initial evaluation of the service with a dyslectic user group to see how they experience the current design. With the insights from the initial evaluation we then ideated through sketches and a prototype to use for validation of those ideas.
Research
In our early stages of research, we conducted task based usability tests on a demo version of Hubert's service. We combined the task based tests with the think-out loud method to understand the users experience. 2 tests were conducted with users that have dyslexia, each test was approximatley 45 minutes.
Respondant A
“You don't have time to start reading because things kept coming up. It's too much at once”
Respondant B
“I have to do my best to write grammatically correct, otherwise I feel the chatbot can screw up”
Initial Task Based Tests
Findings
Feedback: The respondents experienced that it was difficult to know if they gave the right answer. This contributed to them feeling stressed and unsure.
Clarity: Occasionally, the chatbot would also ask the same question again to indicate that it did not understand the given answer. However, the chatbot did not elaborate, which created confusion and stress.
Speed: A lot of text was produced during a sequence, which meant that the respondent would start reading but then get interrupted and have to scroll up. This was perceived as an annoyance because the user does not want to miss any information.
Ideation
Sketches
To start our ideation process, we sketched ideas of features that would try to solve the users goals:
-
Speed (1 & 2): For this pain point, we sketched an idea to give freedom to change the typing speed of the chat bot. We also sketched a new layout on the multiple choices
-
Feedback & Clarity (3 & 4): To try support the clarity in the conversation, we sketched a feature to let the user re-formulate the question. We also sketched an idea of a feature to let the user regret their answer, in case they felt like they gave a wrong answer.
User Flow
To help design the prototype and the flow of the interaction, I created user flows to better understand which steps the user need to take to reach their goals and how to improve that experience.

Prototype
To validate the ideas with the users I designed the features into a prototype in Figma.
Validation
Evaluation Objectives
The primary objectives of the evaluation were to:
-
Determine if the features adequately addressed the needs of dyslexic users.
-
Measure the usability of the interface.
-
Gather qualitative feedback on user experience and interaction with the conversational agent.
Methodology
We conducted 4 task-based evaluations with 4 dyslexic users. Participants were asked to complete specific tasks using the conversational agent while utilizing the designed features. Observations were made on their interactions, and participants were encouraged to verbalize their thoughts throughout the process.
Findings
-
Speed Slider
-
User Feedback: Participants reported that the default text speed was overwhelming. The ability to slow down the text was appreciated and significantly improved their comprehension.
-
Usability: Most users found the slider intuitive and easy to use. Some suggested clearer labeling of speed levels for better understanding.
-
-
Multi-Column Layout
-
User Feedback: The two-column layout was well-received, as it reduced the need to scroll and helped users make quicker decisions. Participants noted that it was easier to compare options side by side.
-
Usability: Users appreciated the reduced cognitive load, though a few mentioned a preference for slightly larger buttons to enhance touch accuracy.
-
-
Re-Formulate Button
-
User Feedback: Participants found the re-formulate button to be a critical feature, as it allowed them to clarify confusing questions. Many expressed a sense of empowerment knowing they could seek clarification.
-
Usability: The button was easily accessible, but some users suggested incorporating examples of what type of clarification could be requested.
-
-
Revert Answer Button
-
User Feedback: Users valued the ability to change their answers, feeling more in control of the interaction. This feature reduced anxiety about making mistakes.
-
Usability: While most participants found it straightforward, a few suggested that a confirmation pop-up might prevent accidental reversals.
-
Usability Test Conclusion
The task-based evaluation demonstrated that our features positively impacted dyslexic users’ interactions with the conversational agent. The feedback collected guides further refinements to enhance usability and ensure a supportive experience for all users. Moving forward, continuous user testing and feedback collection will be essential in iterating on these features.
How does the design support the user?
In summary, the features we have designed are strategically aligned with the goals and needs of our users. By addressing their concerns around certainty, clarity, and text management, we are creating a more inclusive and supportive environment for dyslexic users interacting with the conversational agent. These enhancements not only improve usability but also empower users to engage confidently and effectively in their interactions.
Reflection
For this project, we had a tight timeline of just one month to identify a client, conduct research, ideate, prototype, and validate our design. I believe that with more time for iteration, we could have achieved even better results. For instance, we discovered post-evaluation that dyslexic users struggle with text presented in all caps. Unfortunately, this insight was not addressed in our design solution featuring a two-column layout for multiple-choice options.
Additionally, the limited timeframe prevented us from fully validating some suggestions that emerged during our usability tests. While we established strong communication with our client at the beginning of the project, we experienced a lack of responsiveness towards the end. This may have been due to our failure to maintain regular updates on our progress, which ultimately hindered our ability to gather the client's feedback on our design solutions. Such feedback would have been invaluable in assessing the feasibility of our designs and ensuring they met the client's needs effectively.