Abstract

Motion capture (Mocap) is applied to motor rehabilitation of patients recovering from a trauma, a surgery, or other impairing conditions. Some rehabilitation exercises are easily tracked with low-cost technologies and a simple Mocap setup, while some others are extremely hard to track because they imply small movements and require high accuracy. In these last cases, the obvious solution is to use high performing motion tracking systems, but these devices are generally too expensive in the rehabilitation context. The aim of this paper is to provide a Mocap solution suitable for any kind of exercise but still based on low-cost sensors. This result can be reached embedding some artificial intelligence (AI), in particular a convolutional neural network (CNN), to gather a better outcome from the optical acquisition. The paper provides a methodology including the way to perform patient's tracking and to elaborate the data from infra-red sensors and from the red, green, blue (RGB) cameras in order to create a user-friendly application for physiotherapists. The approach has been tested with a known complex case concerning the rehabilitation of shoulders. The proposed solution succeeded in detecting small movements and incorrect patient behavior, as for instance, a compensatory elevation of the scapula during the lateral abduction of the arm. The approach evaluated by medical personnel provided good results and encouraged its application in different kinds of rehabilitation practices as well as in different fields where low-cost Mocap could be introduced.

References

1.
Rossoni
,
M.
,
Fumagalli
,
A.
, and
Colombo
,
G.
,
2019
, “
Machine Health State Recognition Through Images Classification With Neural Network for Condition-Based Maintenance
,”
International Conference on Design, Simulation, Manufacturing: The Innovation Exchange
,
Cham
,
June 9–12
, pp.
432
443
.
2.
Colyer
,
S. L.
,
Evans
,
M.
,
Cosker
,
D. P.
, and
Salo
,
A. I.
,
2018
, “
A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System
,”
Sports Medicine
,
4
(
1
), pp.
24
39
. 10.1186/s40798-018-0139-y
3.
Da Gama
,
A.
,
Fallavollita
,
P.
,
Teichrieb
,
V.
, and
Navab
,
N.
,
2015
, “
Motor Rehabilitation Using Kinect: A Systematic Review
,”
Games Health J.
,
4
(
2
), pp.
123
135
. 10.1089/g4h.2014.0047
4.
Mao
,
C. Y.
,
Jaw
,
W. C.
, and
Cheng
,
H. C.
,
1997
, “
Frozen Shoulder: Correlation Between the Response to Physical Therapy and Follow-up Shoulder Arthrography
,”
Arch. Phys. Med. Rehabil.
,
78
(
8
), pp.
857
859
. 10.1016/S0003-9993(97)90200-8
5.
Lin
,
H. C.
,
Chiang
,
S. Y.
,
Lee
,
K.
, and
Kan
,
Y. C.
,
2015
, “
An Activity Recognition Model Using Inertial Sensor Nodes in a Wireless Sensor Network for Frozen Shoulder Rehabilitation Exercises
,”
Sensors
,
15
(
1
), pp.
2181
2204
. 10.3390/s150102181
6.
Popescu
,
D.
,
Copilusi
,
C. P.
,
Roibu
,
H.
,
Rusu
,
L.
, and
Marin
,
M. I.
,
2017
, “
Human Upper Limb Motions Video Analysis Used for Rehabilitation Robotics
,”
Proceedings of the International Conference on Applied Physics, System Science and Computers
,
Cham
,
Sept. 27–29
, pp.
264
273
.
7.
Rastegarpanah
,
A.
,
Saadat
,
M.
, and
Borboni
,
A.
,
2016
, “
Parallel Robot for Lower Limb Rehabilitation Exercises
,”
Appl. Bionics Biomech.
,
2016
(
8584735
), pp.
1
10
. 10.1155/2016/8584735
8.
Sousa
,
M.
,
Vieira
,
J.
,
Medeiros
,
D.
,
Arsenio
,
A.
, and
Jorge
,
J.
,
2016
, “
SleeveAR: Augmented Reality for Rehabilitation Using Realtime Feedback
,”
Proceedings of the 21st International Conference on Intelligent User Interfaces
,
Sonoma, CA
,
March
, pp.
175
185
.
9.
Yeung
,
K. Y.
,
Kwok
,
T. H.
, and
Wang
,
C. C. L.
,
2013
, “
Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications
,”
ASME J. Comput. Inf. Sci. Eng.
,
13
(
4
), p.
4
. 10.1115/1.4025404
10.
Kwok
,
T. H.
,
Yeung
,
K. Y.
, and
Wang
,
C. C. L.
,
2014
, “
Volumetric Template Fitting for Human Body Reconstruction From Incomplete Data
,”
J. Manuf. Syst.
,
33
(
4
), pp.
678
689
. 10.1016/j.jmsy.2014.05.009
11.
Domingues
,
A.
,
Jorge
,
J.
, and
Lopes
,
D. S.
,
2018
, “
Kinect-Based Biofeedback Interfaces to Improve Upper Limb Rehabilitation
,”
Ann. Med.
,
50
(
2
), pp.
S110
S111
. 10.1080/07853890.2017.1407035
12.
Gal
,
N.
,
Andrei
,
D.
,
Nemeş
,
D. I.
,
Nădăşan
,
E.
, and
Stoicu-Tivadar
,
V.
,
2014
, “
A Kinect Based Intelligent e-Rehabilitation System in Physical Therapy
,”
Stud. Health Technol. Inform.
,
210
(
2015
), pp.
489
493
. 10.3233/978-1-61499-512-8-489
13.
Zhao
,
W.
,
Feng
,
H.
,
Lun
,
R.
,
Espy
,
D. D.
, and
Reinthal
,
M. A.
,
2014
, “
A Kinect-Based Rehabilitation Exercise Monitoring and Guidance System
,”
Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science (ICSESS)
,
Beijing,China
,
June 27–29
, pp.
762
765
.
14.
Otte
,
K.
,
Kayser
,
B.
,
Mansow-Model
,
S.
,
Verrel
,
J.
,
Friedemann
,
P.
,
Brandt
,
A. U.
, and
Schmitz-Hübsch
,
T.
,
2016
, “
Accuracy and Reliability of the Kinect Version 2 for Clinical Measurement of Motor Function
,”
PLoS One
,
11
(
11
), p.
e0166532
. 10.1371/journal.pone.0166532
15.
Wilson
,
J. D.
,
Khan-Perez
,
J.
,
Marley
,
D.
,
Buttress
,
S.
,
Walton
,
M.
,
Li
,
B.
, and
Roy
,
B.
,
2017
, “
Can Shoulder Range of Movement be Measured Accurately Using the Microsoft Kinect Sensor Plus Medical Interactive Recovery Assistant (MIRA) Software?
,”
J. Shoulder Elb. Surg.
,
26
(
12
), pp.
e382
e389
. 10.1016/j.jse.2017.06.004
16.
Kendall
,
A.
,
Grimes
,
M.
, and
Cipolla
,
R.
,
2015
, “
Posenet: A Convolutional Network for Real-Time 6-dof Camera Relocalization
,”
Proceedings of the IEEE International Conference on Computer Vision
,
Santiago, Chile
,
Dec. 7–13
, pp.
2938
2946
.
17.
Chen
,
W.
,
Jiang
,
Z.
,
Guo
,
H.
, and
Ni
,
X.
,
2020
, “
Fall Detection Based on Key Points of Human-Skeleton Using OpenPose
,”
Symmetry
,
12
(
5
), p.
744
. 10.3390/sym12050744
18.
Hagihara
,
H.
,
Ienaga
,
N.
,
Enomoto
,
D.
,
Takahata
,
S.
,
Ishihara
,
H.
,
Noda
,
H.
, and
Terayama
,
K.
,
2020
, “
Computer Vision-Based Approach for Quantifying Occupational Therapists’ Qualitative Evaluations of Postural Control
,”
Occup. Ther. Int.
,
2020
(
8542191
), pp.
1
9
. 10.1155/2020/8542191
19.
Regazzoni
,
D.
,
Vitali
,
A.
,
Rizzi
,
C.
, and
Colombo
,
G.
,
2018
, “
A Method to Analyse Generic Human Motion With Low-Cost Mocap Technologies
,”
Proceedings of the ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
.
20.
Vitali
,
A.
,
Regazzoni
,
D.
, and
Rizzi
,
C.
,
2019
, “
Digital Motion Acquisition to Assess Spinal Cord Injured (SCI) Patients
,”
Comput. Aided Des. Appl.
,
16
(
5
), pp.
962
971
. 10.14733/cadaps.2019.962-971
21.
Zennaro
,
S.
,
Munaro
,
M.
,
Milani
,
S.
,
Zanuttigh
,
P.
,
Bernardi
,
A.
,
Ghidoni
,
S.
, and
Menegatti
,
E.
,
2015
, “
Performance Evaluation of the 1st and 2nd Generation Kinect for Multimedia Applications
,”
Proceedings of IEEE International Conference on Multimedia and Expo (ICME)
,
Turin, Italy
,
June 29–July 3
, pp.
1
6
.
22.
Wei
,
S.
,
Ramakrishna
,
V.
,
Kanade
,
T.
, and
Sheikh
,
Y.
,
2016
, “
Convolutional Pose Machines
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 27–30
, pp.
4724
4732
. 10.1109/CVPR.2016.511OpenPose
23.
Riley
,
K. F.
,
2006
,
Mathematical Methods for Physics and Engineering
,
Cambridge University Press
,
Cambridge
, pp.
931
942
.
You do not currently have access to this content.