GameRAT

​In GameRAT, we are continuing the development of a remote rehabilitation solution. In this project we are creating minigame modules for virtual and mixed reality application, which will be used for rehabilitation purposes by patients at home. As a second part, we are creating a web solution for professionals, where they are able to define new exercises, create complete exercise routines and send them to the patients remotely. This first part of the web solution will be released as open source during May as the first deliverable from the project. We have also done a large amount of motion capture for different exercises, which will form a basic library for professionals to use.

On the patient side, after receiving new training regime from the professionals, the application will automatically compare the selected exercises with the movement ranges in different minigames, and select the most suitable for the user. After this, user can complete the game, which motivates them to do the exercises without even noticing it. To make sure they are staying within a safe movement range, we will implement AI-based guardians, which will notify the user if their movements are outside of the safe area. On top of this, we will have a pain estimation scale in the game, where users can report how they felt during the game.

At the moment we have developed 4 different mixed reality games for different kind of arm movements. Mixed reality was selected, to allow easier use of different training equipment like weights or flexbands. In the first one, the user is a wizard making potions, and they have to pick correct ingredients from the shelf, mix them and cast the required spells. In the second one, the user will hold a pet frog with both of their hands, and direct the frog to eat insects that are flying around the environment. In the third one, the user is a DJ with a mixer and record players, on which they have to follow directions to perform their music correctly. And in the last one, the user will sit down in a boat, and row around a nice friendly river in their living room.

All the time, during the game, the app records data on how the user is performing. All of this feedback will be sent to the professionals, along with the full movement data from the user. This data will be analyzed and based on the results, the professional can alter the exercises as needed. To make the system as easy as possible for the patients, we are looking for different tracking solutions and making comparisons between them. The goal is to find the lightest solution with the least external equipment, while offering accurate enough data for the professionals to follow the progress of the patient.

The project will continue until the end of March next year. During next winter, we will conduct studies with professionals, to validate the usefulness of the application and make sure the user experience for the patients is as good as possible. Thank you to the whole EMIL team for enabling us to do this research!