Accessible Design is Good Design

RTEs, sign language, and learnings from a design hackathon

This past weekend, I took part in my first hackathon, the remote, 24-hour DubsTech Protothon, with over 300 designers from all over the world. I’m proud of what my team accomplished, humbled by the fantastic work of other designers, and inspired by the challenges we faced.

The problem of missing voices

Imagine not knowing who’s speaking in a video call. Not being able to follow the conversation while taking notes. Not being able to speak up and participate when playing an online game with friends. Life has moved to video chat due to the Covid-19 pandemic, but that shift left some behind. Nearly 6% of the world’s population has disabling hearing loss.

Tasked with leveraging existing (RTE) technology in a new enterprise space, my team and I drew on Agora’s Sign Language Recognition technology, and applied it to remote learning.

Listening to what the user needs

From my competitive analysis, I found that few services have provided the quality of resources needed for students facing hearing challenges to become equal participants in their online classrooms. User interviews emphasized the importance of strong visuals and body language, and bemoaned UI such as Facetime’s moving video bubbles, which make it impossible to focus on someone who is signing in a group conversation. The pain-points and needs of the user soon became clear: a service that provided high-quality captions, emphasized text, provided users with a voice so they could participate in real-time conversation, and didn’t come with a paywall.

Time crunch: keep iterating or move on?

Following our research and user interviews, our synthesis led us through the complexities of the problem space and took us back to the drawing board a few times. Of course, one of the biggest challenges in a hackathon is the time crunch. We time-boxed each part of the design process for ourselves in the morning. But as the hours ticked by, that timeline would inevitably change as we encountered many more questions than viable solutions.

Mapping our research into a user journey helped us to define our focus in this complex space, and we moved forward to build an all-in-one remote learning platform for universities that put the needs of students with hearing loss front and center.

Accessible design is good design, for all

Our solution included features such as:

• sign-language recognition technology knows when a student is signing, and highlights their screen, transitioning them to “speaker” mode
• real-time, color-coded transcription during a video call, available for download after class
• ability to highlight lines of transcript and save notes
• high-quality, real-time captions with AI recognized word emphasis

With these features, students with disabling hearing loss can participate in large lectures, small-group discussions, and online games with the confidence that they can keep up with their peers and be heard in the room.

These features not only improve usability for students with disabled hearing loss, but also help all students raise a hand and enter the conversation, keep organized notes, limit distractions, or better follow along in class with text and captions.

Learnings

I was lucky enough to work with two great designers from my bootcamp with whom I’ve had experience building products. We had great, open communication and a foundational understanding of how we work together through the design process. This was key!

Yet it was still tough to know how much time to spend in the research and ideation phases. I learned the importance of challenging assumptions, thinking outside the box, and still leaving enough time (and energy!) to create compelling UI and storytelling in your presentation.

How have you balanced the pieces of the design-thinking process in a hackathon? What were some of the lessons you came away with?

Thanks to the DubsTech team, Adobe, Agora, and the fantastic panel of judges!