RealityShiftAR
A homemade see-through AR headset that transforms the face of the person in front of you in real time with generative AI, stripping away the “noise” of appearance to make face-to-face conversation easier.
Raspberry Pi
Stable Diffusion
Mediapipe / OpenCV
3D-printed enclosure
Solo project
Maker Faire Tokyo 2023
Platform
A standalone headset running on Raspberry Pi alone, no Unity
Pipeline
Mediapipe + OpenCV face capture, Stable Diffusion img2img generation over Wi-Fi
My role
Solo project: planning, software, hardware and the live demo
Shown at
Maker Faire Tokyo 2023, as ailab*, alongside XSA and SlipperX
Background and technical challenge
In May 2023, I showed a piece at Kyoto Maker Faire that used Python to shift a speaker’s voice — through formant conversion — into a girl, a “handsome guy” or an anime villain. A visitor’s offhand comment there — “wouldn’t it be funny if the person’s face changed too, not just their voice?” — became the seed for this project.
That grew into a bigger idea: a person’s appearance or voice can get in the way of face-to-face conversation. You freeze up talking to someone cute, or hold back your honest opinion around an intimidating older person — appearance becomes “noise” that gets in the way of what’s actually being said, even though the content is what should matter. Since you can’t change the other person, why not remake the world you see instead? Drawing on Osamu Tezuka’s Phoenix: Resurrection — where everyone looks like a monster except, to the android protagonist, other humans — I set out to build AR glasses that hack your vision.
The technical challenge: build this as a standalone AR device that runs on a Raspberry Pi alone, without a heavyweight engine like Unity, detecting faces in real time and layering a generative-AI-transformed image over them without it looking obviously wrong.
Challenge. A standalone AR headset, Raspberry Pi only, that swaps a real face for a generated one in real time — convincingly.
Role and scope
A personal project. I handled the whole thing myself — planning and requirements, software (face detection and the image-generation pipeline), hardware (CAD design and 3D printing of the enclosure), and running the live demo.
Technology and architecture
Requirements: a standalone AR headset that runs on Raspberry Pi alone, no Unity, and that transforms a face with generative AI — keeping something of the person’s own likeness — rather than just swapping in a stock illustration.
1. Capture
The Raspberry Pi’s camera films the scene in front of the wearer.
2. Face detection & crop
Mediapipe detects the face; OpenCV crops just that region.
3. Send & generate
The cropped face is sent over Wi-Fi to a gaming laptop. Stable Diffusion WebUI, running there, generates the transformed image via its img2img Web API and saves it back to the Pi.
4. Composite & display
The generated image is composited back over the original face position and shown on a 7-inch LCD as the AR view — a look-through-the-display design, with no optical lens system.
Hardware:
CAD design in Fusion 360; the part touching the face was solidified with sculpt modelling.
The glasses body and cover were printed as two parts on a stereolithography 3D printer (over a day plus 6 hours of printing, 500 g of filament).
What took some care
Generation lag was freezing the camera feed. Stable Diffusion takes 2–3 seconds per image, and waiting on that synchronously froze the whole camera view. Fixed by using Python’s threading to keep firing generation requests to the gaming PC on a separate thread, decoupled from the camera’s own draw loop.
3D-printing trial and error: unit 1 had the Raspberry Pi colliding with the lid; unit 2 fixed that but printed with a rough surface. Traced to degraded filament — swapping to fresh filament fixed it (and taught me filament needs care in storage, since it’s sensitive to humidity).
The Raspberry Pi overheated mid-demo. It never showed up during debugging at home, but across two straight days at Maker Faire Tokyo 2023 the Pi thermally throttled and dropped out repeatedly — a hard lesson in needing real heatsinks or a fan.
As a generative-AI quirk, images the creator never intended also came out — and I chose to treat that surprise as part of the experience, not a bug.
Results and impact
Working demo
On a gaming PC, confirmed a wide range of transformations just by changing the prompt — younger, more handsome, older man, older woman and more.
Anime-style boy
Anime-style girl
Aging
A finished headset
Completed the self-built see-through AR enclosure and shared it on X (Twitter).
Maker Faire Tokyo 2023
Shown as a hands-on exhibit under ailab*, alongside XSA and SlipperX. Groups of visiting students burst out laughing seeing each other’s transformed faces, and passers-by who laughed first often stopped to ask about it.
A Unity-free foundation
Building on Raspberry Pi + Python made it easy to fold in newer tech — generative AI, speech-to-text — without ever needing Unity.
Learnings and next steps
Building a self-made AR headset end to end: choosing a “look through the display” design over a full optical AR system with real lens design kept the difficulty and timeline in balance.
On the hardware side, I learned first-hand how much 3D-print quality depends on how the material has been stored.
Running two straight days at a show taught me how much robustness matters — heat-dissipation design, but also just building things so that unplugging and replugging the power fixes most problems.
I’d only planned for one face at home; at the show, “I want to try it with my friend” came up constantly, making simultaneous multi-face detection a real requirement, not a nice-to-have.
Next: build on this hardware foundation to add voice conversion and speech synthesis, extending the AR experience to hearing as well as sight.