Worldcoin Foundation open sources core components of the Orb’s software

The Orb was built to give individuals a way to prove their humanness online with maximum privacy and security. Now, the core components of the Orb’s software are publicly available on GitHub under an MIT/Apache 2.0 dual license. For more context on why open sourcing the Orb matters, see the hardware section of the Worldcoin Tech Tree.

These new open components complement the previously-released hardware and iris recognition repositories, and their public availability marks significant progress in making the Orb’s image processing transparent and its privacy claims verifiable. Today’s release includes all code on the Orb that is essential for capturing images and securely transferring them to World App^[1].

This post reviews the design requirements of the Orb’s software, highlights key implementation details, and looks ahead into its future.

If you’re interested in working on the Orb’s software, see open roles at project contributor Tools for Humanity here.

The Orb provides the highest verification level for World ID to distinguish humans from AI^[2]. It does this by first checking that the World ID holder is human using eye and face images. If these humanness checks are passed, the Orb then performs an iris verification process and submits the resulting iris code to a service that verifies the person’s uniqueness. The process is completed in a few seconds. For more information about how Orb-verified World IDs can be used, see here. 

The Orb’s sole use case for verifying humanness leads to the following top-level requirements, unprecedented for a biometric device:

1. Privacy: During the World ID verification, the Orb must process images locally on-device and then securely transfer them to the user’s custody. This eliminates any requirement for storing images on the Orb or uploading them to a central backend for processing^[3].
2. Security: The Worldcoin protocol expects Orbs to only verify World IDs of genuine humans. This means the Orb must be highly resistant to spoofing and tampering, even in adversarial environments^[4]. Modified or counterfeit devices must be rejected by the protocol. 
3. Transparency & Verifiability: The Orb must be designed to require minimal trust in its manufacturers and operators. There must be a way for the public to audit Orbs, including checking the state of the software and cryptographic keys. See the relevant whitepaper section for more details.

These requirements drive the design of the Orb and impact virtually all aspects of its software.

Tools for Humanity (TFH), a contributor to the Worldcoin project, has spent years building a secure and powerful computing environment for the Orb. The design centers around the NVIDIA Jetson, which locally processes the Orb’s five camera streams to verify a World ID in seconds. Each Orb additionally has two Arm Cortex M4 microcontrollers (running Zephyr RTOS) to handle things like power management, real-time controls, and tamper detection. Finally, each Orb has a dedicated secure element which, like a hardware wallet, stores a private signing key in a tamper-resistant enclosure.

The Orb runs several Rust applications on the Jetson, with a heavy use of async Rust. These applications handle core functions like processing sensor data, interfacing with neural networks, and applying system updates. For neural network inference, the Orb uses NVIDIA’s TensorRT engine to fully leverage the Jetson’s 384-core GPU. 

Tying this all together is Orb OS^[5]: a custom GNU/Linux distribution based on Debian and Ubuntu, optimized for security. Orb OS includes full-system integrity checks, a security-hardened Linux configuration, a robust over-the-air update system and support for reproducible builds. For critical cryptographic operations (e.g. authenticating to the backend), Orb OS communicates with a trusted execution environment (TEE) on the Jetson as well as the dedicated secure element. 

Among the key components of the Orb’s software that were released today is the do_signup function in orb-core, which encodes the Orb’s high-level flow for each World ID verification. This function calls the image capture phase and biometric processing phase, then submits signed iris codes to the uniqueness service. Additionally, do_signup calls upload_custody_images to securely send the user’s images to their World App.

In a matter of seconds, the Orb’s advanced optical system can reliably create high-quality eye and face images. This core functionality is enabled by custom software and neural networks that work together to implement face detection, eye tracking, autofocus and auto exposure.

The autofocus was one of the most difficult parts of the imaging system to implement. It controls the purpose-built liquid lens in order to quickly find a sharp image of an individual’s eyes. The Rust implementation contains a feedback controller, with sharpness estimates provided by a real-time neural network. The control output is sent from the Jetson to a microcontroller, which implements an inner control loop for driving the electrical current to the lens^[6].

Once the Orb has created eye and face images of sufficient quality, they are passed to the biometric pipeline. This pipeline has two goals: check that the images are authentic^[7] and generate iris codes. Both are handled by Python interfaces to custom neural networks, as well as a Gabor filter implementation.

The high-quality eye images from the previous phase are passed to the Iris Recognition Inference System (IRIS). This state-of-the-art implementation was built by TFH and subsequently open sourced by the Worldcoin Foundation. The Rust code is optimized to always keep IRIS and other large models in memory, avoiding the overhead of regularly re-loading these large components. This is accomplished with a design inspired by Android’s Zygote process, which avoids loading redundant copies of shared dependencies (like CUDA).

Privacy and security have always been top priorities when building the Orb. The Orb’s privacy claims^[8] were the focus of a white-box audit by Trail of Bits, which was recently published. Further audits are underway and, combined with open sourcing, reflect the Foundation’s aim to maintain a continuously high level of transparency around privacy and security.

One privacy-critical step is sending individual data from the Orb to World App for a feature called “Personal Custody”. Individuals can use this self-custodied data, for example, for Face Authentication in World App. Looking ahead, this feature may also reduce the frequency with which users would need to return to an Orb to re-verify their World ID.

The implementation on the Orb ensures individuals always remain in control of their data. Each data package is signed with the Orb’s private key, then encrypted with a user-provided public key^[9][10] before being transferred to an individual’s mobile phone. Signing is performed by a dedicated secure element, which is cryptographically paired to the Jetson during manufacturing^[11]. The encryption implementation uses a sealed box to ensure only the individual can decrypt this biometric data^[12]. Once the encrypted data is sent from the Orb to the individual’s World App, no unencrypted copies of this data exist anywhere. 

This initial release of the Orb’s software is an important step towards the goals^[13] of improving the transparency and verifiability of the Orb. For next steps, the Worldcoin Foundation and the broader Worldcoin community intend to focus on the following areas related to Orb software:

1. Maximally open development. Most components released today already have open development in their public repositories. However, some components (notably orb-core and orb-secure-element) are released as forks of the internal versions, with some sensitive^[14] code manually removed. In keeping with the protocol’s mission, the community’s goal would be to eliminate this divergence and move all non-sensitive code to open development.
2. Untrusted closed-source components. To minimize the required trust in closed-source components, they are run in a “sandbox” to restrict their system access. The current sandbox configuration is actively being improved to prevent all ML models from accessing the network and persistent storage. Critically, the sandbox implementation is itself open source.
3. Support decentralized audits.  The Worldcoin Tech Tree includes support for attesting properties of specific Orbs, like the exact software they are running. Anyone should be able to perform these attestations and publicize the results. This infrastructure is necessary for reducing the required trust in Orb suppliers. Several engineering projects are underway to support such attestations, including secure version reporting and public reproducible builds.

Finally, these components aim to help bring security, privacy, and transparency to many other projects, especially those running machine learning on edge devices. For example, AI-enabled learning environments (like this one) come with data privacy challenges similar to the Orb. It is the Worldcoin Foundation’s hope that the community will, to help make an AI-powered world safe, continuously improve the components open sourced today to maximize their utility for other projects.