Machine Learning and Control Systems Laboratory
Yonsei team won the first place at the 2nd Autonomous Driving Robot Race, 2022
Yonsei team with undergraduate students from mechanical engineering and graduate students from MLCS won the first place at the 2nd Autonomous Driving Robot Race, held in Sun Moon University, Asan, Chungcheongnam-do, Republic of Korea, November 12th 2022.
Yonsei team won the first place at the 2nd Autonomous Driving Robot Race, 2022
Yonsei team with undergraduate students from mechanical engineering and graduate students from MLCS won the first place at the 2nd Autonomous Driving Robot Race, held in Sun Moon University, Asan, Chungcheongnam-do, Republic of Korea, November 12th 2022.
Yonsei team won the first place at the 2nd Autonomous Driving Robot Race, 2022
Yonsei team with undergraduate students from mechanical engineering and graduate students from MLCS won the first place at the 2nd Autonomous Driving Robot Race, held in Sun Moon University, Asan, Chungcheongnam-do, Republic of Korea, November 12th 2022.
Policy Design for an Ankle-Foot Orthosis Using Simulated Physical Human-Robot Interaction via Deep Reinforcement Learning, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022
In this paper, we propose a two-stage policy training framework based on deep reinforcement learning (deep RL) to design a robot controller using human-robot dynamic simulation. In Stage 1, the optimal policy of generating human gaits is obtained from deep RL-based imitation learning on a healthy subject model using the musculoskeletal simulation in OpenSim-RL. In Stage 2, human models in which the right soleus muscle is weakened to a certain severity are created by modifying the human model obtained from Stage 1. Comparative analysis of kinematic and kinetic simulation results with the experimental data shows that the derived human muscu- loskeletal model imitates a human walking.
Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchies, IEEE Robotics and Automation Letters (RA-L), IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022
In this study, we incorporate simultaneous activation of the skills and structure them into multiple hierarchies in a recursive fashion in order to orchestrate the skills with different action spaces via multiplicative Gaussian distributions. Exploiting the modularity, interpretability can also be achieved by observing the modules that are used in the new task if each of the skills is known. We demonstrate how the proposed scheme can be employed in practice by solving a pick and place task with a 6 DoF manipulator and examine the effects of each property from ablation studies.
Nonlinear Model Predictive Control with Cost Function Scheduling for a Wheeled Mobile Robot in the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 23–27, 2022 Kyoto, Japan.
This paper proposes a novel MPC approach with a scheduled quadratic cost function that approximates the true objective in order to optimally control a nonlinear system with a sparse/binary objective. The cost function parameter is optimally scheduled by a parameter scheduling policy obtained by solving a Markov decision process (MDP) constructed from sampled trajectories from any nonlinear MPC solver. The simulation and experimental results successfully demonstrate the effectiveness of our MPC approach in cases of the point stabilization problem of a differential drive WMR.
Regularized Nonlinear Regression for Simultaneously Selecting and Estimating Key Model Parameters: Application to Head-Neck Position Tracking, Engineering Applications of Artificial Intelligence, 2022
In this paper, we propose a new method to simultaneously select and estimate sensitive parameters as key model parameters while fixing the remaining parameters to a set of typical values. The problem is formulated as a nonlinear least-squares estimator with L1-regularization on the deviation of parameters from a set of typical values. In addition, a modified optimization approach is introduced to find the solution to the formulated problem. As a result, we provided consistency and oracle properties of the proposed estimator as a theoretical foundation. To show the effectiveness of the proposed method, we conducted simulation and experimental studies.
A warning system for urolithiasis via retrograde intrarenal surgery using machine learning: an experimental study, BMC Urology, 2022
We developed an acceleration measurement system that recorded the power signal data for a number of hours, yielding distinguishable characteristics among three different states (idle state, stones, and tissue–laser interface) by conducting fast Fourier transform (FFT) analysis. A discrete wavelet transform (DWT) was used for feature extraction, and a random forest classification algorithm was applied to classify the current state of the laser-tissue interface. Our monitoring system receives the shockwave signals generated from the RIRS urolithiasis treatment procedure and generates the laser irradiance status by rapidly recognizing (in 0.5 s) the current laser exposure state with high accuracy (95%). We postulate that this can significantly minimize surgeon error during RIRS.
Behavior Tree-Based Task Planning for Multiple Mobile Robots using a Data Distribution Service, 2022 IEEE/ASME International Conferene on Advanced Intelligent Mechatronics, Royton Sapporo, Sapporo, Hokkaido, Japan, July 11th-15th, 2022
In this study, we propose a task planning framework for multiple robots that builds on a behavior tree (BT). BTs communicate with a data distribution service (DDS) to send and receive data. Since the standard BT derived from one root node with a single tick is unsuitable for multiple robots, a novel type of BT action and improved nodes are proposed to control multiple robots through a DDS asynchronously. To show the feasibility of our framework in a real-world application, three mobile robots were experimentally coordinated for them to travel alternately to four goal positions by the proposed single task planning unit via a DDS.
Suspension Control Strategies Using Switched Soft Actor-Critic Models for Real Roads, IEEE Transactions on Industrial Electronics, 2022
In this paper, we propose learning and control strategies for a semi-active suspension system in a full car using soft actor-critic (SAC) models on real roads, where many road profiles with various power of disturbance exist (e.g., speed bumps and general roads). Our proposed switching learning system continuously identifies two different road disturbance profiles in real-time such that the appropriately designed SAC model can be learned and applied accordingly. Our switching SAC approach outperforms those of advanced and conventional benchmark suspension systems. Finally, we also presented our successfully implemented SAC training system in a real car on real roads. The trained SAC model outperforms conventional controllers reducing the z-directional acceleration and pitch, similar to the simulation results, which is highly related to the riding comfort and vehicle maneuverability.
Safety-critical Control with Nonaffine Control Inputs via a Relaxed Control Barrier Function for an Autonomous Vehicle, IEEE Robotics and Automation Letters, IEEE International Conference on Robotics and Automation (ICRA), 2022
This letter proposes a novel control design for an autonomous vehicle system with nonaffine control inputs that can track the desired trajectories while considering the safety constraint. The safety and the stability of the closed-loop system are analyzed using a singular perturbation method. The proposed method is validated using the numerical simulation and the high-fidelity car simulator under realistic driving scenarios.
Output feedback control design using Extended High-Gain Observers and dynamic inversion with projection for a small scaled helicopter, Automatica 2021
This paper presents the output feedback control design for a tracking problem of a small-scale helicopter in the presence of uncertainties. A time-scale approach is suggested to cope with underactuated mechanical systems to overcome lack of the number of inputs. The singular perturbation method is used to analyze the stability of the closed-loop system in a multi-time scale structure. Based on the stability analysis, the design procedure for the proposed algorithm is presented for practical implementation. The effectiveness of the proposed control algorithm is shown via numerical simulations as well as experimental tests with a small-scale helicopter in an outdoor environment.
State reconstruction in a nonlinear vehicle suspension system using deep neural networks, Nonlinear Dynamics, 2021
In this work, we proposed and designed a long short-term memory (LSTM)-based neural network to estimate the velocity and position states of a full car’s suspension system using only online data streams from cheap inertial sensory measurements. Our approach shows that with only noisy accelerometer and gyroscope sensors, we can successfully reconstruct nearly perfect states in a full car’s nonlinear dynamic system using a well-trained LSTM-based neural network combined with the extended Kalman filter.
Virtual racing driver coaching project funded by Hyundai Motor Company
We aim to develop a virtual racing driver coach in order to coach human drivers to improve driving skills. The first goal is to develop a classification model which is able to evaluate the driving levels of racing drivers. The second objective is to develop a virtual racing driver that can be a reference to each level of the drivers and visually guide optimal vehicle control to the driver.
Nonaffine Helicopter Control Design and Implementation based on a Robust Explicit Nonlinear Model Predictive Control in IEEE Transactions on Control Systems Technology, 2021
To solve a control problem of the helicopter under model uncertainties and disturbance present environments, an explicit nonlinear model predictive control (ENMPC), a dynamic inversion, and an extended high-gain observer (EHGO) are combined in a multi-time-scale fashion. The multi-time scaled structure and the ENMPC provide the framework of the control design, the dynamic inversion deals with non-affine control inputs, and the EHGO estimates the unmeasured system states and uncertainties. The successful outdoor experiments with the proposed control implemented on autopilot hardware demonstrate the validity of our approach in the presence of model uncertainties and wind disturbances.
Multi-output Infinite Horizon Gaussian Processes in the Proceedings of 2021 International Conference on Robotics and Automation (ICRA 2021)
In this paper, we propose a multi-output infinite-horizon Gaussian process (MOIHGP) that generalizes the single-output IHGP to deal with multiple outputs for better prediction. Our approach allows us to consider correlations between multiple outputs for better prediction, even with occlusions in a Bayesian way. Finally, we successfully demonstrate the effectiveness of our approach by benchmark and experimental results. For simulated benchmark experiments with high noise levels, our approach reduced 16.6% of the averaged RMSE value achieved by the single-output IHGP.
Vision-based Uncertainty-aware Lane Keeping Strategy using Deep Reinforcement Learning in the Journal of Dynamic Systems, Measurement, and Control, 2021
In this work, for the uncertaintyaware lane keeping, we first propose a convolutional mixture density network (CMDN) model that estimates the lateral position error, the yaw angle error, and their corresponding uncertainties from the camera vision. We then establish a vision-based uncertainty-aware lane keeping strategy in which a high-level reinforcement learning policy hierarchically modulates the reference longitudinal speed as well as the low-level lateral control.
Automated cephalometric landmark detection with confidence regions using Bayesian convolutional neural networks, BMC Oral Health, 2020
In this paper, we propose the novel framework for locating cephalometric landmarks with confidence regions based on uncertainties using Bayesian Convolutional Neural Networks (BCNN). With Bayesian inference over iterative CNN model calculations, we can derive the confidence region (95%) of an identified landmark considering model uncertainty, and significantly higher the in-region accuracy. Given the uncertainty and confidence areas of the estimated location, clinicians are expected to determine whether the results of the framework are reliable and to make a more accurate diagnosis.
Prediction of Reward Functions for Deep Reinforcement Learning via Gaussian Process Regression in IEEE/ASME Transactions on Mechatronics and IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2020
This new article proposes an efficient way to solve the inverse reinforcement learning problem based on the sparse Gaussian process (GP) prediction with l1-regularization only using a highly limited number of expert demonstrations. A GP model is proposed to be trained to predict a reward function using trajectory-reward pair data generated by deep reinforcement learning (RL) with different reward functions. The experimental results clearly show that the robots can clone the experts’ optimality in navigation trajectories avoiding obstacles using only with a very small number of expert demonstration data sets.
Efficient Sampling for Rapid Estimation of 3D Stiffness Distribution via Active Tactile Exploration in IEEE/ASME Transactions on Mechatronics and IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2020
This paper proposes a novel efficient sampling strategy to rapidly estimate the distribution of stiffness over an inhomogeneous object with a highly limited number of sample points taken from the object surface. The main objective of this work is to improve the efficiency of the estimation process while producing an accurate estimate for both the overall distribution and some particular areas (i.e., high/low stiff areas). Physical experiments on a variety of inhomogeneous objects demonstrate the advantage of the proposed algorithm in comparison to a popular existing algorithm in terms of accuracy and estimation speed.
Customer-Sepcific Robotic Attendant for VR Simulators in IEEE Transactions on Automation Science and Engineering
This article proposes a new type of robotic attendant for VR simulators, which provides personalized high-quality services and automatically operates the VR simulators in VR theme parks. In addition, the robotic attendant improves customer satisfaction by providing personalized services based on the estimation of the customer’s age, gender, and game progress. Finally, we experimentally show that the personalized services provided by the robotic attendant improve customer satisfaction. The results of the satisfaction questionnaire and independent- samples t-test validate our proposed scheme.
Automated skeletal classification using lateral cephalogram based on AI in Journal of Dental Research:
This study aims to provide an accurate and robust skeletal diagnostic system by incorporating a convolutional neural network (CNN) into a one-step, end-to-end diagnostic system using lateral cephalograms. A multimodal CNN model was constructed based on 5,890 lateral cephalograms and demographic data as an input. The proposed system exhibited greater than 90% sensitivity, specificity, and accuracy for vertical and sagittal skeletal diagnosis. The proposed CNN incorporated system showed potential for skeletal orthodontic diagnosis without the need for intermediary landmarking procedures.Autonomous driving via marriage between deep learning and control theory: Unexpected Collision Avoidance Driving Strategy Using Deep Reinforcement Learning in IEEE Access
We are working on a three year project entitled by “Autonomous driving control systems utilizing deep-learning based situation awareness and its uncertainty”, funded by National Research Foundation of Korea (NRF) (PI Choi). In this project, we study a novel deep learning model that estimates various parameters required for the autonomous driving control and their uncertainties as well in real time. Finally, we develop autonomous driving control strategies that can exploit estimated parameters and their uncertainties in order to guarantee the safety and performance of the autonomous vehicle.Inverse optimal control (IOC) and/or inverse reinforcment learning (IRL):
Experts' demonstration data can be used to learn reward functions in order to efficiently build artificial intelligence via inverse optimal control or inverse reinforcment. We are currently applying IRL to build AI for dual armed mobile robots to interact with human workers in dynamically uncertain environments.
Inverse optimal control (IOC) and/or inverse reinforcment learning (IRL):
Experts' demonstration data can be used to learn reward functions in order to efficiently build artificial intelligence via inverse optimal control or inverse reinforcment. We are currently applying IRL to build AI for dual armed mobile robots to interact with human workers in dynamically uncertain environments.
Patient-Specific Prediction of Abdominal Aortic Aneurysm Expansion using Bayesian Calibration in IEEE journal of biomedical and health informatics
The goal of this project is to make prediction and gauge its uncertainty in the computer models for a patient (such as a growth and remodeling model) from observations and various uncertainties such as measurement noise, estimation errors, and biological variability. This research has been supported by an NIH R01 grant with a project “Growth and Remodeling Model of Abdominal Aortic Aneurysm: Toward Clinical Applications.”
A MLCS team won the first place in BMW/Yonsei Student Research Competition 2018
Sponsored by College of Engineering, Yonsei University and by BMW Korea, the award ceremony was held on April 16, 2018 at the BMW Driving Center in Songdo, Incheon. Six teams from Yonsei University selected through the first round screening in early January announced their research results for about 3 months. Among them, "Autonomous ability to cope with risky situation via the awareness and reinforcement learning of vehicle surroundings" A research team of the school of mechanical engineering was awarded the First Prize. This MLCS team, composed of undergraduate students Sung-Won Lee, Jung-Hoon Lee, Sung-Ha Woo and graduate students Myunghoe Kim and Jaehyun Lim, researched under the guidance of Professor Jongeun Choi and won 5 million won. The last three months of the tournament ended at the end of the awards ceremony, but it will be further expanded in a variety of ways as a model of a research contest where colleges and industry labs work together in the future.AI service robots for VR theme parks:
The goal of the project is to develop an artificial intelligence based service robot for VR theme parks. The service robot can give cheering, encouragement, explanation, guidance, etc. according to the situations to make the customer happy. The service robot determines the current situation and recognizes features of the customer to provide customer-specific services based on vision, game status data, etc. using various machine learning techniques.
Prediction and adaptive sampling for mobile sensor networks:
We have developed spatio-temporal prediction and adaptive sampling algorithms for mobile sensor networks. Mobile sensing agents need to locally learn the uncertain environment collaboratively with neighboring agents to achieve a global goal such as exploration and environmental monitoring. We are developing environmental adaptive sampling algorithms for mobile sensor networks to predict scalar fields of interest using nonparametric approaches raning from Gaussian processes, Gaussian Markov random fields, and kernel regression. This project has been funded by an NSF CAREER Award. A collection of successful outcomes will be disseminated as a SpringerBrief: “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks,” by Xu, Choi, Dass, and Maiti.
Prediction and adaptive sampling for mobile sensor networks:
We have developed spatio-temporal prediction and adaptive sampling algorithms for mobile sensor networks. Mobile sensing agents need to locally learn the uncertain environment collaboratively with neighboring agents to achieve a global goal such as exploration and environmental monitoring. We are developing environmental adaptive sampling algorithms for mobile sensor networks to predict scalar fields of interest using nonparametric approaches raning from Gaussian processes, Gaussian Markov random fields, and kernel regression. This project has been funded by an NSF CAREER Award. A collection of successful outcomes will be disseminated as a SpringerBrief: “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks,” by Xu, Choi, Dass, and Maiti.
Prediction and adaptive sampling for mobile sensor networks:
We have developed spatio-temporal prediction and adaptive sampling algorithms for mobile sensor networks. Mobile sensing agents need to locally learn the uncertain environment collaboratively with neighboring agents to achieve a global goal such as exploration and environmental monitoring. We are developing environmental adaptive sampling algorithms for mobile sensor networks to predict scalar fields of interest using nonparametric approaches raning from Gaussian processes, Gaussian Markov random fields, and kernel regression. This project has been funded by an NSF CAREER Award. A collection of successful outcomes will be disseminated as a SpringerBrief: “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks,” by Xu, Choi, Dass, and Maiti.
Prediction and adaptive sampling for mobile sensor networks:
We have developed spatio-temporal prediction and adaptive sampling algorithms for mobile sensor networks. Mobile sensing agents need to locally learn the uncertain environment collaboratively with neighboring agents to achieve a global goal such as exploration and environmental monitoring. We are developing environmental adaptive sampling algorithms for mobile sensor networks to predict scalar fields of interest using nonparametric approaches raning from Gaussian processes, Gaussian Markov random fields, and kernel regression. This project has been funded by an NSF CAREER Award. A collection of successful outcomes will be disseminated as a SpringerBrief: “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks,” by Xu, Choi, Dass, and Maiti.
Prediction and adaptive sampling for mobile sensor networks:
We have developed spatio-temporal prediction and adaptive sampling algorithms for mobile sensor networks. Mobile sensing agents need to locally learn the uncertain environment collaboratively with neighboring agents to achieve a global goal such as exploration and environmental monitoring. We are developing environmental adaptive sampling algorithms for mobile sensor networks to predict scalar fields of interest using nonparametric approaches raning from Gaussian processes, Gaussian Markov random fields, and kernel regression. This project has been funded by an NSF CAREER Award. A collection of successful outcomes will be disseminated as a SpringerBrief: “Bayesian Prediction and Adaptive Sampling Algorithms for Mobile Sensor Networks,” by Xu, Choi, Dass, and Maiti.
Modeling, assessment, and rehabilitation of human motor control systems:
The goal of the project is to apply concepts from systems and control theory to model and analyze the human motor control systems in order to generate useful information for treatments and interventions. Human-robot physical interaction is used to assess the motor control of the subject. The intent of the estimated motor control is analyzed by solving the formulated inverse optimal control problem. This project has been funded by an NIH center grant, “Systems Science Center for Musculoskeletal CAM Therapies.” See more information at Center for Orthopedic Research.
Modeling, assessment, and rehabilitation of human motor control systems:
The goal of the project is to apply concepts from systems and control theory to model and analyze the human motor control systems in order to generate useful information for treatments and interventions. Human-robot physical interaction is used to assess the motor control of the subject. The intent of the estimated motor control is analyzed by solving the formulated inverse optimal control problem. This project has been funded by an NIH center grant, “Systems Science Center for Musculoskeletal CAM Therapies.” See more information at Center for Orthopedic Research.
Linear parameter varying (LPV) modeling and control:
Gain-scheduling controllers for various LPV systems with hard constraints have been designed based on the numerically efficient convex optimization or linear matrix inequality (LMI) technique. The simulation and experimental results demonstrate the effectiveness of the proposed approaches to energy-efficient engine systems. A collection of the works is published as a SpringerBrief: “Linear parameter-varying control for engineering applications,” by White, Zhu, and Choi.Vision
We strive to solve challenging societal problems by exploiting powerful modern computational and mathematical techniques from machine learning, systems and control, deep reinforcement learning, and Bayesian statistics. Our laboratory has been conducting, and continuously seeks for intriguing and multidisciplinary projects with scientists, medical clinicians, and engineers from robotics and automotive companies, including building artificial intelligence for inverse reinforcement learnig of a driving style for autonomous driving; self-driving and manipulating mobile robots interacting with people in dynamical and uncertain environments; optimal path-planning for racing or electric vehicles; medical machine vision and data-driven, patient-specific calibration of computational models; and intent-analysis and physical human-robot interaction (pHRI) for gauging and rehabilitating human motor control systems. As results, our solutions are creative and successful with intellectual merits, and happen to synergistically combine recent advances in machine learning techniques such as deep learning, (deep) reinforcement learning, inverse reinforcement learning, and Gaussian process regression with engineering, mathematical, and/or statistical techniques such as convex optimization-based control synthesis, system identification, and Bayesian inferential methods.
News
- Professor Choi served
- as a Guest Editor with a two-year term (2021 and 2022) for IEEE/ASME TMECH/AIM Focused Section on Emerging Topics.
- IEEE Robotics and Automation Letters. Review timeline is given here. RAS Conference/Journal Management System.
- 2021 IEEE International Conference on Robotics and Automation.
- ASME Journal of Dynamic Systems, Measurement and Control. ASME Journal Management System.
- International Journal of Precision Engineering and Manufacturing. IJPEM Journal Management System.
School of Mechanical Engineering, Yonsei University 50 Yonsei Ro, Seodaemun Gu, Seoul 03722, Republic of Korea