AR Walking Assist
A walking-assist system that shows a person, through AR glasses, a video of “another self” walking, and gradually enlarges that gait to raise the person’s stride and walking speed.
HoloLens / AR
Walk-Mate model
Mirror neuron system
3D avatar
Gait rehabilitation
Context
Master’s research, Tokyo Institute of Technology (Miyake Lab)
Platform
Microsoft HoloLens, fully wireless
Participants
11 healthy young men (mean age 23.8 ± 2.25)
Method
Wilcoxon signed-rank test on stride length, speed and cadence
Research background and question
Aging, stroke and Parkinson’s disease all change how people walk, and that decline in gait is a major driver of lower quality of life. Rehabilitation methods that use sound or force cues to entrain walking rhythm — such as the Walk-Mate model developed in the Miyake Lab — help with the “timing” side of gait, but struggle to directly guide its “spatial” side: stride length, how high the foot lifts.
Visual information, on the other hand, is good at conveying spatial change, and motor-observation therapy — which uses the mirror neuron system, where simply watching someone else’s movement pulls your own movement toward it — has been reported to improve gait in stroke patients.
Question. If someone walking is shown, in real time, a 3D avatar resembling themselves whose gait is gradually changed, can that also change their own gait — without them noticing?
Related work and novelty
Conventional motor-observation therapy gets its effect from watching a movement and then performing it; whether observing it while performing it changes anything was not established.
Sound- and force-based walking assist (the Walk-Mate model and similar) is strong at entraining rhythm, but cannot directly guide spatial elements such as stride length.
Prior work (Meerhoff, Shan and others) looked at keeping distance from a walking avatar or synchronising with an auditory cue, but changing the avatar’s own gait in real time, resembling the user, to aim at an unconscious change in their gait, was this study’s own position.
Using an AR head-mounted display (HoloLens) meant the video could be shown safely, fully wirelessly, while walking and seeing the real environment — unlike prior work using projectors or floor projection.
Method and approach
A three-module system built around HoloLens:
Motion sensing
Detects vertical head movement from HoloLens’s own environment-tracking camera; each cycle’s minimum (foot contact) gives the walking rhythm.
Motion control
Uses the Walk-Mate model developed in the Miyake Lab — a phase-oscillator mutual-entrainment model — to generate, in real time, an avatar’s walking rhythm synchronised to the user’s.
Motion-video synthesis
Walks the avatar at the synchronised rhythm and projects it life-size, 5 m ahead of the user. On an external trigger (a HoloLens clicker), the avatar’s gait — arm swing, foot lift — is gradually changed over 20 seconds while staying rhythm-synchronised.
Building the avatar: about 60 photos of each participant’s own body were taken, turned into a 3D model with Autodesk ReMake/ReCap, and rigged in Mixamo — one personalised avatar per participant.
Experiment and evaluation design
Participants
11 healthy young men, mean age 23.8 ± 2.25.
Setting
A roughly 55 m straight corridor, no obstacles or slope.
No-change condition
The avatar’s gait is never changed.
Motion-change condition
From the 10 m mark, the avatar’s arm-swing and foot-lift angles are gradually increased (knee rotation ×1.42, elbow rotation ×2.17 on average).
Each condition run 5 times in random order, 10 trials in total.
Wearable sensors (3-axis accelerometer and gyroscope) on both ankles measured walking speed, stride length and gait cycle.
After the experiment, a questionnaire checked whether each participant had noticed the avatar’s gait changing, so the “noticed” and “didn’t notice” groups could also be analysed separately.
Statistical analysis used the Wilcoxon signed-rank test.
Results
Stride length
Significantly greater under motion-change than no-change (p < 0.01).
Walking speed
A trend toward significantly higher under motion-change (p < 0.1).
Gait cycle
No significant difference between conditions — the rhythm itself didn’t change; only stride and speed did.
Even limited to the 5 participants who did not notice the avatar’s gait changing, stride length still showed a trend toward significant increase (p < 0.1). The same trend held for the 6 who did notice, with no significant difference between the two groups.
Discussion
Stride length and speed increased even though the gait cycle (rhythm) itself did not change, suggesting the visual stimulus affected the user’s gait itself rather than their walking rhythm.
Because the gait change also occurred in participants who never noticed the avatar changing, this supports the idea that it arose from unconscious mirroring rather than deliberate imitation.
One interpretation: integrating the user’s own sense of movement with the visual stimulus of a self-resembling avatar evokes a “sense of agency” — the feeling that it is oneself moving that avatar — which may have contributed to pulling the gait along.
Compared with motor-imagery or motor-observation therapy, which depend on a patient’s own memory and imagination, this system may be able to change gait without relying on either.
Limitations and future directions
Tested only on healthy young adults; the effect on patients with gait disorders is still unverified.
Arm swing and foot lift were changed together, so how much each one contributed to the stride-length change can’t be separated.
Only the immediate effect was observed; longer-term effects from sustained training remain future work.
HoloLens’s mechanical latency and differences in individual body sense are not fully reflected in the accuracy of rhythm entrainment.
A more direct check of the sense of agency, for example through subjective questionnaires, is still needed.
Future work: combine auditory cues (the Walk-Mate model) with the visual cue, toward a multimodal assist system that can guide both the temporal and spatial sides of gait more freely.
This research was later extended by a junior PhD student, Dane A.L. Miller and colleagues, into a method that controls distance (a spatial cue) and phase difference (a temporal cue) at the same time, published in IEEE Access in 2022.
Read the IEEE Access Paper