Human-Centric Assistive Robotic Systems

Introduction

Human-Centric Assistive Robotic Systems (ARS) investigate autonomous mobile robots that operate safely and effectively in environments shared with people. Research focuses on developing generalized methodologies that enable robots to perceive, interpret, and respond to human behaviour while performing assistive tasks in dynamic and uncertain environments. Unlike traditional autonomous systems that rely solely on environmental sensing, assistive robots must also reason about human intent, preferences, activities, and contextual information to support robust decision-making.

Research at CIMLab integrates multimodal perception, person detection and tracking, human–robot interaction, semantic mapping, navigation, and adaptive decision-making into autonomous robotic architectures capable of assisting users in hospitals, long-term care facilities, offices, homes, and other human-centered environments. Particular emphasis is placed on uncertainty-aware perception, contextual reasoning, and autonomous search strategies that enable robots to locate, identify, and assist people despite incomplete information, changing environments, and unpredictable human behaviour.

Methodologies are validated through simulation and experiments using autonomous mobile robotic platforms operating in realistic human-centered environments. The resulting research contributes to the development of generalized decision-making methodologies applicable across a broad range of assistive and service robotic applications.

Representative Research: Development of Mobile Robot Architectures for Person Detection, Tracking, and Search – Angus Fung (PhD, 2025)

This research developed autonomous mobile robot architectures that enable robots to detect, track, and locate individuals in complex human-centered environments, including hospitals, offices, university campuses, and search-and-rescue settings. The work addressed three tightly coupled challenges: robust person detection under occlusions and changing illumination, adaptive long-term tracking despite appearance variations, and autonomous person search without complete knowledge of user schedules or locations.

Human-centric assistive robotic system

The proposed architectures integrated multimodal perception, temporal reasoning, semantic mapping, and large language model–based search planning to enable robots to interpret natural language instructions, reason about spatial context, and efficiently locate individuals in real time. Extensive simulation and experimental validation demonstrated significant improvements in person detection accuracy, tracking robustness, and search efficiency compared with existing robotic approaches, advancing autonomous mobile robots capable of operating safely and effectively in dynamic human-centered environments.

Selected Publications:

Fung, A., Benhabib, B., & Nejat, G. (2025). LDTrack: Dynamic People Tracking by Service Robots Using Diffusion Models. International Journal of Computer Vision, 133(6), 3392–3412.
DOI: 10.1007/s11263-024-02336-9
Publisher: https://doi.org/10.1007/s11263-024-02336-9

Hong, A., Lunscher, N., Hu, T., Tsuboi, Y., Zhang, X., dos Reis Alves, S. F., Nejat, G., and Benhabib, B. (2021). A Multimodal Emotional Human–Robot Interaction Architecture for Social Robots Engaged in Bidirectional Communication. IEEE Transactions on Cybernetics, 51(12), 5954–5968.
DOI: 10.1109/TCYB.2020.3047206
Publisher: https://doi.org/10.1109/TCYB.2020.3047206

Nuger, E., & Benhabib, B. (2018). A Methodology for Multi-Camera Surface-Shape Estimation of Deformable Unknown Objects. Robotics, 7(4), 69.
DOI: 10.3390/robotics7040069
Publisher: https://doi.org/10.3390/robotics7040069