Human Activity Recognition (HAR) based on IMU sensors is a crucial area in ubiquitous computing. Because of the trend of deploying AI on IoT devices or smartphones, more researchers are designing different HAR models for embedded devices. Deployment of models in embedded devices can help enhance the efficiency of HAR. We propose a multi-level HAR modeling pipeline called Stage-Logits-Memory Distillation (SMLDist) for constructing deep convolutional HAR models with embedded hardware support. SMLDist includes stage distillation, memory distillation, and logits distillation. Stage distillation constrains the learning direction of the intermediate features. The teacher model teaches the student models how to explain and store the inner relationship among high-dimensional features based on Hopfield networks in memory distillation. Logits distillation builds logits distilled by a smoothed conditional rule to preserve the probability distribution and enhance the softer target accuracy. We compare the accuracy, F1 macro score, and energy cost on embedded platforms of a MobileNet V3 model built by SMLDist with various state-of-the-art HAR frameworks. The product model has a good balance with robustness and efficiency. SMLDist can also compress models with a minor performance loss at an equal compression ratio to other advanced knowledge distillation methods on seven public datasets.
State estimation with sensors is essential for mobile robots. Due to different performance of sensors in different environments, how to fuse measurements of various sensors is a problem. In this paper, we propose a tightly coupled multi-sensor fusion framework, Lvio-Fusion, which fuses stereo camera, Lidar, IMU, and GPS based on the graph optimization. Especially for urban traffic scenes, we introduce a segmented global pose graph optimization with GPS and loop-closure, which can eliminate accumulated drifts. Additionally, we creatively use a actor-critic method in reinforcement learning to adaptively adjust sensors weight. After training, actor-critic agent can provide the system better and dynamic sensors weight. We evaluate the performance of our system on public datasets and compare it with other state-of-the-art methods, which shows that the proposed method achieves high estimation accuracy and robustness to various environments. And our implementations are open source and highly scalable.
