Visual object detection and pose estimation of texture-less objects in unstructured environments

Aggarwal, Ayush (2025). Visual object detection and pose estimation of texture-less objects in unstructured environments. University of Birmingham. Ph.D.

Preview

Aggarwal2025PhD.pdf
Text - Accepted Version
Available under License All rights reserved.
Download (53MB) | Preview

Abstract

Advancements in industries such as manufacturing, logistics, healthcare, and energy increasingly rely on autonomous robots to enhance reliability, efficiency, and safety. A critical factor in these advancements is the ability of these robots to accurately perceive and understand their environments, enabling effective interaction and decision-making. Visual data, including images and three-dimensional (3D) point clouds, serve as reliable sources of information for these systems. However, vision-based approaches encounter numerous challenges when faced with texture less objects, articulated shapes, and unstructured environments with varying lighting conditions and occlusions, which often degrade performance.

To address these challenges, this thesis develops novel methods for visual object detection and pose estimation, specifically tailored for texture less objects in unstructured environments. Firstly, a 3D edge detection method has been proposed to extract 3D edge features from a scene's depth map. The 3D edges, representing the geometrical boundaries of an object, are independent of the object's texture, making them well-suited for challenging scenarios.

The extracted 3D edges are subsequently utilised to estimate six-degree-of-freedom (6-DoF) pose of objects in unstructured environments. This task is approached as an alignment problem, where the object's model cloud is aligned with the observed 3D edge point cloud. The sparse nature of the 3D edges allows the network to effectively manage limited data, making it robust in handling occlusions and unstructured scenes where dense object information is often unavailable.

Furthermore, this thesis proposes a method for classifying articulated objects. Objects' articulation introduces additional complexities, which significantly reduce the performance of standard methods for pose estimation, grasping, and manipulation. To address this, a classification approach is developed to determine whether an object is rigid or articulated by analyzing a video sequence that captures object manipulations. By identifying the object's properties within the scene, this approach enables the deployment of property-specific methods tailored to perform the corresponding tasks effectively.

Finally, a learning-based visual servoing method utilising the 3D edge point cloud is proposed to evaluate the effectiveness of 3D edges in practical scenarios. A deep learning-based network is implemented to learn latent features from 3D edges, which, together with the centroid of the 3D edge cloud, are utilised as visual cues to guide the servoing task.

The efficacy of the proposed vision-based methods has been validated through various performance metrics, including rotation error, translation error, F-measure, precision, recall, and average model distance (ADD) error. The performance of these methods has been analysed across multiple benchmark datasets and compared with state-of-the-art (SOTA) methods available in the literature. Results demonstrate competitive or superior performance compared to state-of-the-art methods, confirming the effectiveness of the proposed approaches for texture-less objects and unstructured environments.

Type of Work:

Thesis (Doctorates > Ph.D.)

Award Type:

Doctorates > Ph.D.

Supervisor(s):