The Effect of Imperfect Labelling on an LSTM Deep Learning Intent Classifier for Assistive Technology

Intent sensing, the estimation of what it is that a human wants to make happen, is a useful input for collaborative robots, allowing them to intuitively detect and anticipate user needs. It is easier to detect an action after it has completed than it is to predict an action in advance. Therefore, it is proposed that a pre-trained after-the-fact classifier could be used to label unsupervised training data for a predictive classifier, and thus allow a self-learning system with a constantly expanding data set, tailored to the specific user. However, the after-the-fact classifier will not be 100% accurate, so this labelling will be imperfect. To determine whether such a system could be viable, it is important to investigate the effect of imperfect labelling on a deep learning intent sensing system. This is tested on time-series data from an existing data set of Parkinson’s disease patients and controls, classified with a Long Short-Term Memory recurrent neural network to estimate intent over time. The results show that while large numbers of label errors cause a negative impact on the classifier accuracy, the classifier is robust to small amounts of label noise, with a label error proportion of 1/12 having only a very small impact (~0.6%) on the accuracy of the classifier. This supports the viability of a self-learning intent sensing system.