Real-time energy management of the hybrid off-highway vehicle using reinforcement learning

Download Statistics

Downloads

Downloads per month over past year

Shuai, Bin (2022). Real-time energy management of the hybrid off-highway vehicle using reinforcement learning. University of Birmingham. Ph.D.

Preview

Shuai2022PhD.pdf
Text - Accepted Version
Available under License All rights reserved.
Download (5MB) | Preview

Abstract

Rapid developments in transportation and mobility systems including off-highway vehicles have a significant impact on the environment and energy. Electrification of off-highway vehicles makes a significant contribution to decarbonisation with advancements in vehicle technologies. This Ph.D. research focuses on the development of an advanced energy management strategy (EMS) for the hybrid off-highway aircraft-towing tractor, harnessing reinforcement learning (RL), one of the most promising artificial intelligence technologies. Based on real-time modelling of the vehicle system using MATLAB/Simulink, the dedicated RL algorithms are developed to enhance the exploration capability and reduce the overestimation of the reward-action pairs so that they can improve the energy efficiency of the hybrid vehicle in learning cycles and real-world driving. The real-time control functionalities of the proposed energy system are tested and verified on the hardware-in-the-loop platform.
By introducing three decay functions to the epsilon-greedy process of the RL-based EMS, the impacts of the exploration-to-exploitation ratio (E2E) on improving vehicle energy efficiency are investigated. A step-based decay function is proposed as a unified decay function incorporated with the vehicle EMS. Its contribution to the improvement of trustworthiness is demonstrated by a comparison with standard Q learning. An improvement rate of up to 6.21% higher than the standard Q-learning is achieved.
Based on the performance analysis of the RL-based EMS with different decay functions, information-fused and synchronised ensemble learning frameworks are developed to enhance the optimality and robustness of the RL-based EMS. The random-based, maximum-based and weighted-based methods are studied and compared under the ensemble learning frameworks. The study suggests that the weighted-based method is the best of the three ensemble methods. This research leads to a new synchronised ensemble learning framework founded on the weighted-based method, with up to 1.06% and 5.06% higher vehicle energy efficiency than the maximum-based method and random-based method, respectively. Additionally, the weighted-based method achieves 1.15% higher vehicle energy efficiency than the standard Q-learning.
Finally, by observing overestimation phenomena in the RL-based EMS under disturbed learning and undisturbed learning scenarios, a predictive double Q-learning framework with backup models and random action execution policy, respectively is developed and used to increase the efficiency of the learning experience and determine the final action to improve the robustness of the RL-based EMS for the hybrid off-highway vehicles.

Type of Work:

Thesis (Doctorates > Ph.D.)

Award Type:

Doctorates > Ph.D.

Supervisor(s):