ROAR VR Project Page
ROAR VR
By Sam Moturi, Jiajian Lu, Hankai Zhao, Xinyun Cao
Motivation: 
1. Reduce cost of racing sport -- our car is 1:10 size to real car.
2. Build a platform for testing autonomous vehicles.
3. Facilitate research in transportation field.


Localization Development Process:
Step 1: Testing singular AR marker
Step 2: Arranging markers on Richmond Field Station Track
Step 3: Record on car-mounted camera and output location information

Streaming Development Process:
Step 1. Connect the jetson car and unity server to the same network.
Step 2. Set up the GStreamer pipeline to stream RGB images.
Step 3. Capture depth image and encode it as RGB image.
Step 4. Decode the depth image in Unity and display.

Final Prototype Structure:
1. We achieve Localization with the AR marker and 3D reconstruction with keyframe matching. 
2. To achieve the robustness of the system, we manually set the positions of AR marker in the real world track and do rounds of testing. 3. We also do the 3D Streaming, using intel realsense SDK to provide API to access images and set up the GStreamer Pipeline to stream the images through the network.
4. In our design, we'll make a designed VR city rendered to VR headset to provide extremely attractive VR car racing experience. However we don't have enough time during this semester to achieve that.

Localization Team Final Prototype:
1. When AR markers are detected:
- Calculate current position using marker positions recorded in json file.
- Record the current configuration and frame in variables.



2. When AR markers are not detected:
- Key Frame Matching is called.
- Feature Extraction Using FAST and BRIEF.
- Compute point clouds.
- Estimate rigid transformation.

Streaming Final Prototype:
1. Stream the RGB and depth images of the realsense camera and  RGB image of CSI camera from the racing car to Unity through gstreamer.
2. The RGB and depth images from realsense camera are displayed in the front window and the image from CSI camera is displayed on the rear view mirror.

Check Out Our Project Video!

Thank you for Watching!
ROAR VR Project Page
2
109
0
Published:
Multiple Owners
Xinyun Cao