Ph.D. (Engg): Navigation of Autonomous Vehicles using Event Cameras and Modified RRT Methods

Autonomous vehicles, such as unmanned aerial vehicles (UAVs) and autonomous mobile robots (AMRs), are at the forefront of technological innovation and are widely used across various applications. As these vehicles become more agile and operate primarily in unstructured environments, the components of the navigation pipeline must function in real time while optimizing limited onboard computing and memory resources. The challenges faced by a fast-moving vehicle in indoor environments differ significantly from those encountered by outdoor systems. This thesis focuses on autonomous vehicles operating in indoor, GPS-denied, and unstructured environments. The algorithms presented address these specific challenges and contribute to the growing body of research on real-time navigation solutions for such scenarios. In this thesis, we have investigated and addressed various aspects of the autonomous vehicle navigation pipeline. A key focus throughout the work is ensuring real-time performance on edge computing systems. Inspired by the emergence of bio-inspired event cameras, which offer potential solutions to the limitations of current state-of-the-art algorithms, the first part of the thesis explores the use of these sensors for perception tasks such as localization and obstacle avoidance. Event cameras provide several advantages, including motion blur-free data output, a high dynamic range, and enhanced low-light sensitivity. These features make them particularly suitable for improving Visual-Inertial Odometry (VIO) systems over traditional frame-based cameras. However, the sparse and asynchronous nature of event data poses challenges for conventional computer vision algorithms. Existing approaches often convert event streams into image-like representations, limiting the full potential of event cameras. To overcome these challenges, asynchronous (data-driven) methods are essential for event-camera-based VIO solutions. The work here introduces an end-to-end data-driven event camera-based Visual-Inertial Odometry (AeVIO) algorithm that updates the system state based on camera velocity. The algorithm performs event feature detection and tracking asynchronously from the event stream and integrates these measurements with IMU data using a structureless Extended Kalman Filter (EKF) to refine state estimates. Given that the data rate of event cameras depends on the scene texture and the relative motion between the object and the camera, we also explore their application for high-speed obstacle avoidance. Time-to-contact (TTC) is a critical measure estimating the time before collision if the current motion remains unchanged. While event cameras excel at capturing small, rapid changes, they lack the detailed scene information that depth cameras provide. We present a novel approach to fuse the low temporal resolution data from a depth camera with the high-speed output of an event camera to compute TTC with obstacles. The proposed algorithm is integrated into the AirSim simulator and evaluated across various dynamic obstacle scenarios, demonstrating its effectiveness in collision avoidance. The second part of this thesis focuses on the path planning component of the autonomous navigation pipeline. Effective navigation for AMRs and UAVs requires advanced path planning that accounts for kinematic constraints and enables smooth trajectory execution in complex, cluttered environments. We investigate a probabilistic framework based on the Rapidly Exploring Random Tree (RRT) algorithm, which incorporates vehicle kinematics to identify the most likely direction for the next node generation. This approach utilizes Gaussian Mixture Models (GMMs) to improve node generation efficiency while addressing optimization challenges in both 2D and 3D spaces. This acts as dynamic bias in the algorithm. Additionally, we introduce a next-node selection heuristic that directs the search tree expansion toward the goal while avoiding obstacles. To enhance convergence, we explore methods to discretize both the action and search spaces. Initially, the method is applied to AMRs and is subsequently extended to the more complex task of 3D path planning for UAVs. In summary, this thesis contributes to the navigation pipeline by developing simple, computationally efficient algorithms that leverage event sensors and probabilistic methods. These algorithms are designed to operate in real-time on modern UAVs and AMRs while preserving their agility, enabling operation in indoor GPS-denied environments, and accommodating limited onboard computing resources.

Speaker: Ankit Gupta

Research Supervisor: Debasish Ghose