openxrlab / xrapi

OpenXRLab XRAPI is an open source implementation of the “Information technology - Application interfaces of augmented reality system for mobile devices”.

Introduction

OpenXRLab XRAPI is an open source implementation of the "Information technology - Application interfaces of augmented reality system for mobile devices"(《信息技术 移动设备增强现实系统应用接口》). This standard specifies the data types and interface definitions for mobile device augmented reality system application development, and be used to guide the design, development and implementation of mobile device augmented reality systems. XIAPI is part of the OpenXRLab project. We hope that XIAPI can connect industry and academia, give full play to their respective advantages, and form a closed loop to promote ecological development.

demo_video1.mp4

demo_video2.mp4

demo_video3.mp4

Major Features

The augmented reality system on mobile devices consists of modules such as real-time tracking and positioning, scale estimation, landmark recognition and tracking, 3D reconstruction, illumination estimation, face alignment, gesture recognition etc.

XRAPI includes the following interface modules:

• Real-time 6DoF pose tracking: Use input data from mobile devices to calculate 6DoF pose.
• Illumination estimation: Use video streams to calculate illumination information.
• Scale estimation: Use input data to calculate scale information about the physical world.
• Marker recognition and tracking: Marker recognition and tracking based on video streams.
• 3D reconstruction: Use 6DoF pose, scale information and depth data to reconstruct the 3D model of the scene.
• Face alignment: Predicting a set of facial keypoints from a video stream.
• Gesture recognition: Detect hand bounding boxes and classify gestures based on video streams.

Introduction to standards

Information technology - Application interfaces of augmented reality system for mobile devices

• Scope of application: The standard specifies the data types and interfaces for mobile device augmented reality system application development.

• Main issues to be solved: The application interface of the mobile device augmented reality system is a bridge that realizes the interaction between the device operating system, the augmented reality engine and the application program. The application interface is an indispensable basic link to realize the augmented reality function on the mobile device.In order to further improve the compatibility of mobile device augmented reality systems and guide the design, development and application of mobile device augmented reality systems, this standard specifies the data types and interface definitions for mobile device augmented reality system application development.

Installation

We provide detailed installation tutorial for XRAPI.

Getting Started

Please refer to quick start for the basic usage of XRAPI.

License

The license of our codebase is Apache-2.0. Note that this license only applies to code in our library, the dependencies of which are separate and individually licensed. We would like to pay tribute to open-source implementations to which we rely on. Please be aware that using the content of dependencies may affect the license of our codebase. Some supported methods may carry additional licenses.

Citation

If you use this toolbox or benchmark in your research, please cite this project.

@misc{xrapi,
    title={OpenXRLab application interfaces of augmented reality system for mobile devices},
    author={XRAPI Contributors},
    howpublished = {\url{https://github.com/openxrlab/xrapi}},
    year={2024}
}

Contributing

We appreciate all contributions to improve XRAPI. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

XRAPI is an open source project that is contributed by researchers and engineers from both the academia and the industry. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We hope that XIAPI can connect industry and academia, give full play to their respective advantages, and form a closed loop to promote ecological development.

Projects in OpenXRLab

• XRAPI: OpenXRLab application interfaces of augmented reality system for mobile devices.
• XRPrimer: OpenXRLab foundational library for XR-related algorithms.
• XRSLAM: OpenXRLab Visual-inertial SLAM Toolbox and Benchmark.
• XRSfM: OpenXRLab Structure-from-Motion Toolbox and Benchmark.
• XRLocalization: OpenXRLab Visual Localization Toolbox and Server.
• XRMoCap: OpenXRLab Multi-view Motion Capture Toolbox and Benchmark.
• XRMoGen: OpenXRLab Human Motion Generation Toolbox and Benchmark.
• XRNeRF: OpenXRLab Neural Radiance Field (NeRF) Toolbox and Benchmark.