Multimodal Tracking of Therapy Activity and Effort During Rehabilitation

Rehabilitation therapy documentation is predominantly manual and therapist-dependent, with variation in structure, timing, and level of detail across practitioners. Existing records support clinical communication but do not systematically capture the temporal structure or quantitative characteristics of therapy sessions, such as movement intensity, physiological effort, and activity type. This feasibility study evaluates whether a multimodal logging approach can capture this information reliably during routine neurorehabilitation without disrupting clinical workflows. The approach combines three synchronized data streams. Wrist-worn, non-invasive CE-marked sensors continuously record physiological and movement signals (including photoplethysmography-derived heart rate and respiration, skin temperature, electrodermal activity, and acceleration). A single, fixed-position camera records observable movement during the session, from which automated computer-vision methods derive motion representations such as joint coordinates and abstract skeletal or mesh-based models; these derived representations contain biomechanical coordinate data only and do not retain identifying visual features. A therapist-controlled audio recorder captures spoken instructions and task descriptions to provide contextual annotation of session content. Each participant completes three routine therapy sessions within an approximately two-week period at a single rehabilitation center. Each session follows an identical structure: a five-minute baseline resting measurement, the routine therapy session with continuous sensor, video, and voice recording, brief self-reported effort ratings after each activity, and a five-minute recovery resting measurement. Therapy is delivered according to standard clinical practice; the recording systems are used only for data collection and provide no real-time output, feedback, or therapeutic guidance to the therapist or participant. Because the study is exploratory and observational, no hypothesis testing is planned. Analyses are descriptive and focus on data completeness, signal quality, and the practical integration of the recording systems into routine care, together with descriptive associations between therapist-reported effort, patient-reported effort, and sensor- and video-derived measures of activity and physiological load. Recordings are captured electronically, coded under a study ID, and stored on secure access-restricted institutional infrastructure.