Multi-modal Human Haptic Sensing Systems Development

Abstract:

Human balance ability is human basic physiological ability. It has significant potential application in medical diagnosis and law-enforcement action. But current measurement methods for human balance ability all have some deficiencies. Human skin is a very important organ of our human body, closely related to our health and beauty. Skin health itself is also more and more important. But currently there lacks of a portable skin testing device that can realize multi-modal sensing. Now haptic sensing technology is currently a cutting-edge research area and has lots of application in human-robot interaction and VR. Haptic signal contains extremely abundant information which has not been fully exploited. So we are motivated to exploit the haptic principle called Frustrated Total Internal Reflection (FTIR) to develop new balance sensor and skin sensor in order to solve current problems.

The data representation form of balance sensor and subsequent corresponding analysis methods are established. For implementation, the hardware and software for balance sensor both have been setup. It can effectively detect the tread force distribution information under human feet with very high resolution. The principle and structure of balance sensor are described in detail. We have used this balance sensor to capture tread force distribution variation videos for 6 categories of different human motions. A novel 3D convolutional neural network is developed to classify these 6 actions, which achieves high accuracy.

Similarly, the data representation form of skin sensor and subsequent corresponding analysis methods are established. The hardware and software for skin sensor both have been setup, too. Its multiple operation modes are illustrated, which can be used to measure different skin properties. The principles of its contact force measurement and skin deformation measurement realization are both explained. Experiments have been done to verify its contact force measurement ability and skin deformation measurement ability. Experiment results indicate the measurement with high accuracy.

At last, we proposed our work plan on experimental studies, theoretical studies and human experiment. The problems we are about to solve and the contributions we are going to make are emphasized. The potential applications of our developed balance sensor and skin sensor are suggested. Future work timetable has also been scheduled.