Multi-Plane Display uses image processing and Pepper's Ghost illusions to enable any photo to be viewed with real depth, creating a "2 and a half D" Illusion.
-
Depth segmentation
A depth map is extracted from the image using Depth Anything v2 and used to segment the scene into foreground, middleground, and background layers. -
Viewer tracking
Monocular head tracking estimates the viewer’s position relative to the display. -
Shadow correction
Shadow maps are dynamically adjusted based on the viewer’s position to prevent off-axis ghosting artifacts. -
Optical display
The layered images are displayed through a parascope mirror array, producing the virtual display stacking effect.
 Input Image |
 Processed Image |
 Viewable Result |
- Automatic image processing with input from USB drive
- Compatibility with .jpg, .jpeg, .png, and .tiff
- Automated occlusion maps with real-time headtracking
- Automatic Display turn-off and idle mode when unobserved for 30 seconds
- Waveshare 10.1" Display https://a.co/d/bgIxvvW
- Raspberry Pi 5 https://a.co/d/f5K5WJR
- Cooling Fan https://a.co/d/cPdcFge
- Semi-reflective acrylic (162x110mm) (optimally use 66/33 and 50/50 teleprompter glass, but this stuff is cheap) https://a.co/d/37wRoJS
- acrylic mirror (162x110mm) https://a.co/d/5MYHUT7
- 8MP camera https://a.co/d/0b0CgG3
- RPi 5 PSU https://a.co/d/g17dBXe
- Micro SD card https://a.co/d/0lqvz4a
- USB Drive https://a.co/d/dX8yHnJ
- 250 mm CSI cable https://a.co/d/1fIMcX2
INSTALLATION
- Configure the Raspberry Pi camera and display
Edit the firmware configuration file:
sudo nano /boot/firmware/config.txt
Change this line:
camera_auto_detect=1
to:
camera_auto_detect=0
Add the following lines to the bottom of the file:
dtoverlay=vc4-kms-v3d
dtoverlay=vc4-kms-dsi-waveshare-panel-v2,10_1_inch_a
dtoverlay=imx219,cam0
Save the file and reboot the system.
- Install system dependencies
Update the system:
sudo apt update
sudo apt upgrade
Install required packages:
sudo apt install -y python3-libcamera
sudo apt install -y libcamera-apps libcamera-dev libcamera-tools
sudo apt install -y python3-picamera2
sudo apt install -y unclutter
sudo apt install -y libcap-dev
- Clone the repository
git clone https://github.com/aidanmontag13/MultiPlaneDisplay.git
cd MultiPlaneDisplay
- Create the Python environment
Create a virtual environment with access to system packages (required for libcamera):
python3 -m venv venv --system-site-packages
Activate the environment:
source venv/bin/activate
Install Python dependencies:
pip install -r requirements.txt
- Install the systemd service
Move the service file:
sudo mv ~/MultiPlaneDisplay/run_multiplane.service /etc/systemd/system/
Reload systemd:
sudo systemctl daemon-reload
Start the service:
sudo systemctl start run_multiplane.service
sudo systemctl enable run_multiplane.service