AI-Powered Two Way Arabic Sign Language Translation for Inclusive Public Services

Many deaf citizens still struggle to complete essential government services because they cannot easily communicate with staff. Our innovation is an AI system that instantly translates between spoken Arabic and Arabic Sign Language, using a signing avatar and camera. It lets deaf citizens deal with Civil Affairs independently, privately, and with dignity.

Innovation Summary

Innovation Overview

The Civil Affairs Agency launched this innovation to remove a long-standing barrier faced by deaf and hard-of-hearing citizens when accessing essential government services such as identity registration, family records, and civil documentation. In service environments that rely on spoken communication, many sign language users have struggled to clearly explain their needs or understand procedures, often depending on relatives or limited interpreter support. This affected their independence, privacy, dignity, and trust in public institutions.

This challenge is particularly important in the Civil Affairs context because some core services cannot be completed fully online. Due to legal, regulatory, and identity-verification requirements, a number of transactions still require citizens to attend in person at service centers. As a result, accessibility must exist not only in digital channels, but also at the physical point of service. This makes the innovation especially important, as it enables effective and independent communication in settings where the citizen’s physical presence is mandatory.

The innovation is an AI-powered, two-way Arabic Sign Language translation solution that enables natural, real-time communication between deaf citizens and service staff. Using a camera, speech recognition, computer vision, natural language processing, and a 3D signing avatar, the system converts sign language into readable text or speech for employees and spoken Arabic into Arabic Sign Language for citizens, using standard portable devices at front-line service points. Its core objectives are to provide accurate, low-latency translation, reduce reliance on human interpreters, protect user privacy, and embed inclusive communication directly into everyday public services.

Beyond improving service delivery, the innovation reflects a broader commitment to inclusive and equitable public governance. It supports the principle that access to essential government services should not depend on hearing ability or the availability of a third party. By enabling direct communication between citizens and staff, the solution strengthens independence, promotes inclusion, and respects diversity in communication needs. In this way, it aligns with the values of the UN Sustainable Development Goals, especially reduced inequalities, inclusive institutions, and equal participation in public life.

The primary beneficiaries are deaf and hard-of-hearing citizens, who can now complete transactions more independently, quickly, and confidently, without needing companions to speak on their behalf. This improves access while strengthening confidentiality, dignity, and personal agency in services involving sensitive civil information. Frontline staff benefit from clearer interactions, fewer misunderstandings, and shorter service times, while the institution gains a more efficient, cost-effective, and inclusive service model aligned with Saudi Vision 2030.

Innovation Description

What Makes Your Project Innovative?

This innovation is the first AI-powered system in the region to provide real-time, two-way translation between Arabic Sign Language and spoken Arabic for government services, replacing the need for on-site human interpreters. It combines computer vision, speech recognition, and natural-language processing in one integrated engine that simultaneously converts sign language into structured text and spoken language into a lifelike 3D signing avatar, enabling natural, context-aware conversations at civil affairs counters.

Unlike previous ad‑hoc accessibility measures or one-way digital tools, it works on standard portable devices inside service centers, delivers responses within 2–3 seconds, and is tailored specifically to Arabic Sign Language and Saudi public-service processes, giving deaf citizens independent, private and dignified access to critical government services at scale.

What is the current status of your innovation?

The AI-powered two-way Arabic Sign Language system is currently in its implementation phase. Work is focused on integrating the modular AI components (speech recognition, computer vision, NLP, and 3D avatar) with front-line service processes, validating performance, and refining the solution through feedback from deaf users, service agents, and official sign language interpreters. This phase is intended to ensure the solution is fully ready for wider rollout.

Innovation Development

Collaborations & Partnerships

The innovation was led by the Civil Affairs Agency in collaboration with representatives of the deaf community, including the Authority for the Care of Persons with Disabilities and relevant community groups. As the first solution of its kind in this context, it also drew on research and language resources from universities and institutes. The Agency defined the need and requirements, while private local technology companies supported development and refinement.

Users, Stakeholders & Beneficiaries

Primary beneficiaries are deaf and hard-of-hearing citizens, who gain more independent, private, and clearer access to Civil Affairs services. Front-line staff benefit from smoother communication and fewer misunderstandings. The Civil Affairs Agency benefits through more inclusive and consistent service delivery. Disability-focused civil society groups gain a practical accessibility model, while local technology companies gain experience in inclusive Arabic AI solutions for public services.

Innovation Reflections

Results, Outcomes & Impacts

The innovation aims to remove physical and communication barriers so vulnerable and deaf citizens can independently access civil affairs services with dignity, speed and privacy. Through the Taqdeer mobile fleet and AI sign language system, Civil Affairs has already executed hundreds of thousands of decentralized identity operations, including over 203,000 in Riyadh and more than 110,000 in Makkah. The AI engine enables real‑time conversations for the people with disabilities with 2–3 second latency and maintains user satisfaction of around 4 out of 5, while reducing reliance on human interpreters and unnecessary travel.

Challenges and Failures

The most significant challenge was the structural scarcity of standardized Arabic Sign Language datasets, compounded by regional signing variation across the Arab world. Early prototypes also showed reduced accuracy under live service-center conditions due to lighting, background movement, and device constraints, creating trust risks in legally significant transactions. A key institutional lesson was that technical performance alone was insufficient: community trust had to be earned. The response combined purpose-built dataset creation, deaf community co-validation, hybrid word-level and fingerspelling fallbacks, edge optimization for 1–3 second latency, and transparent privacy, retention, and redress governance.

Conditions for Success

Success depended on a strong combination of institutional and technical enablers. Sustained leadership sponsorship from the Civil Affairs Agency ensured accessibility, privacy, and transparency remained core policy priorities. Strong alignment with data protection, confidentiality, and inclusive service obligations provided the governance legitimacy required for legally sensitive civil affairs transactions. Equally, advanced technical enablement—including specialist AI expertise, edge optimization, and secure integration—was indispensable. Most critically, collaboration with a research institute enabled dataset development, experimentation with new linguistic methods, and the scientific rigor necessary to make the solution viable.

Lessons Learned

A key lesson was the importance of experimentation when addressing a challenge with no established precedent. Progress depended on testing multiple approaches, learning quickly from failure, and using persistent trial and error to improve accuracy, usability, privacy, and inclusion outcomes. Leadership commitment was equally essential, particularly the willingness to prioritize dignity, confidentiality, and inclusive service delivery in legally sensitive civil affairs transactions. We also learned that strong technical expertise alone is not enough without a scientific approach. Collaboration with a research institute brought the rigor, linguistic insight, and structured validation needed to turn experimentation into a reliable solution.

Anything Else?

This innovation represents a shift from assistive solutions to inclusive-by-design public service systems. By embedding AI-powered sign language translation directly into service delivery, it redefines accessibility as a standard rather than an exception.
The model has strong potential for replication across Saudi government entities and countries facing similar accessibility challenges, particularly in regions where Arabic Sign Language is underserved. It also contributes to advancing global discussions on digital inclusion and ethical AI by demonstrating how emerging technologies can be responsibly applied to empower underserved communities.