As a result of terminal arthritis in the knee, many patients are implanted with an artificial knee joint. Several rehabilitation measures must be taken to helpso the patients may recover their full range of motion in conclusion to the operation. In the first few weeks after the operation, patients are instructed and supported by physiotherapists while performing various exercises in a rehabilitation center. In order to achieve and maintain the full range of motion in the long term, patients have to repeat certain exercises at home. It is important that these exercises are carried out correctly and continuously. If the exercises are not continuously trained, rehabilitation of the pain will fail. Likewise, continuously incorrectly executed exercise movements do not contribute to the rehabilitation success and can potentially cause injuries. In order to address these difficulties, the Therapy Assist Project aims to develop a rehabilitation system which motivates the user to exercise frequently and gives feedback on her/his performance.

To achieve these goals, an interactive platform is being developed in collaboration with GeBioM, Cesys, LAVAlabs, University of Duisburg and ZaR Münster. Within this platform, several exercises are implemented, each as an individual mini-game (gamification-approach) with independent user statistics. Due to this gamification of the exercises in so- called exergames, the patient should be motivated to improve her/his performance inof the game and thus her/his physical condition. In order to track the user’s performance while playing the exergame, the application recognizes her/his exercise execution and calculates the exercise execution accuracy. Precisely, the HSD develops tracking technologies, exercise execution monitoring algorithms and user- centered feedback that helps to correct her/his pose.

Since we have to measure movements in order to rate a patient’s exercise execution, several inertial measurement units (IMUs) are combined with the measurements of a Microsoft Kinect, i.e. the raw sensor data (accelerometer, gyroscope, and magnetometer) are used to calculate the orientation of each IMU and these orientations are fused with the orientation data of the Kinect via filtering algorithms (e.g. Kalman Filter). In order to monitor the execution of an exercise via Programming by Demonstration (PbD), the exercise is demonstrated by a professional physiotherapist, recorded and processed. If a practitioner plays the exergame, the practitioner’s joint-orientations (e.g. knee, upper and/or lower leg) are compared to the preprocessed PbD-recording. In case the practitioner’s execution differs from the correct exercise execution, a dynamic feedback is given which shows that her/his pose is incorrect and gives an indication on how she or he can obtain the correct pose. This way the user knows exactly how to move the joints to obtain the correct pose. By applying this monitoring-feedback-approach, the user may learn efficiently how to execute the exercises in a correct and healthy manner. Since the user spends little time in improper poses, damage potential of the wrong exercise execution can be reduced and the effectiveness of the training can be maximized, such that possible follow-up operations will be prevented.

Further information

University Duisburg: 
ZaR Münster: