Robotic grasping strategies in unstructured environments with vision and tactile sensing

Miranda de Farias, Cristiana ORCID: 0000-0002-8356-608X (2025). Robotic grasping strategies in unstructured environments with vision and tactile sensing. University of Birmingham. Ph.D.

MirandadeFarias2025PhD.pdf
Text - Accepted Version
Available under License Creative Commons Attribution.
Download (83MB)

Abstract

Robotics has increasingly transitioned from controlled laboratory and industrial settings, where environments and tasks are highly predictable and repetitive, to unstructured environments characterised by variable object shapes, movements, and incomplete sensory information. In such contexts, it is important for robots to adapt and generalise their skills to manage a diverse array of objects, which may be deformed or in motion. This thesis investigates the generalisation of grasping capabilities across various scenarios, including when objects have suffered deformations, when sensory information is missing or in dynamic scenes.

Firstly, this thesis proposes methodologies for extending grasping tasks to different object instances, including those subjected to significant isometric deformations. Initially, grasp transfer algorithms are developed to enable robots to transfer grasps to deformed object instances. Building upon this, the approach is expanded to achieve category-level generalisation, allowing not only the transfer of grasps but also the execution of entire tasks—such as mixing bottle contents, applying glue to surfaces, or pressing buttons—across different object categories. Furthermore, the method is further expanded to allow task-aware grasp functions in partially observed object instances by integrating shape estimation techniques with shape uncertainty measurements to ensure grasp stability.

However, given that shape estimation methods can miss surface features when prior information is limited, this thesis proposes to enhance and refine grasp planning across an initially estimated shape through multimodal sensor feedback. For that a novel tactile exploration technique is introduced to jointly enhance shape estimation and grasp stability using a probabilistic metric for force closure.

Finally, to address the dynamic aspects of real-world environments, this work develops grasping capabilities for moving objects. A visual servoing method is introduced, which accounts for challenges such as objects approaching the edges of the workspace and encountering representation singularities, thus enabling effective deployment in unstructured environments.

Type of Work:

Thesis (Doctorates > Ph.D.)

Award Type:

Doctorates > Ph.D.

Supervisor(s):