[CVPR 2026] 3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image

Project Page | Paper | Model | Dataset | Online Demo

3D-Fixer proposes a novel In-Place Completion paradigm to create high-fidelity 3D scene from a single image. Specifically, 3D-Fixer extends 3D object generative priors to generate complete 3D assets conditioning on the partially visible point cloud at the same location, which is cropped from the fragented geometry obtained from the geometry estimation methods. Unlike prior works that require explicit pose alignment, 3D-Fixer explicitly utilizes the fragmented geometry as the spatial anchor to preserve layout fidelity.

Features

• High Quality: It generates high quality 3D assets for diverse scenes.
• High Generalizability: It generalizes to real world scenes and complex scenes.
• Novel Paradigm: It shifts scene generation scheme to In-Place Completion paradigm, without time-consuming per-scene optimization.

Updates

• [2025-03] Release model weights, evaluation dataset, and inference scripts of 3D-Fixer.

Installation

Prerequisites

• System: The code is currently tested only on Linux.
• Hardware: An NVIDIA GPU with at least 24GB of memory is necessary. The code has been verified on NVIDIA RTX 4090, and NVIDIA RTX L20 GPUs.
• Software:
  • The CUDA Toolkit is needed to compile certain submodules. The code has been tested with CUDA versions 11.8 and 12.8.
  • Conda is recommended for managing dependencies.
  • Python version 3.8 or higher is required.

Installation Steps

1. Clone the repo:

   git clone --recurse-submodules https://github.com/HorizonRobotics/3D-Fixer
   cd 3D-Fixer

2. Install the dependencies (Following TRELLIS):

   Create a new conda environment named threeDFixer and install the dependencies:

   . ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast

   The detailed usage of setup.sh can be found by running . ./setup.sh --help.

   Usage: setup.sh [OPTIONS]
   Options:
       -h, --help              Display this help message
       --new-env               Create a new conda environment
       --basic                 Install basic dependencies
       --train                 Install training dependencies
       --xformers              Install xformers
       --flash-attn            Install flash-attn
       --diffoctreerast        Install diffoctreerast
       --spconv                Install spconv
       --mipgaussian           Install mip-splatting
       --kaolin                Install kaolin
       --nvdiffrast            Install nvdiffrast
       --demo                  Install all dependencies for demo

Pretrained Models

We host the pretrained model at huggingface.

The models are hosted on Hugging Face. You can directly load the models with their repository names in the code:

ThreeDFixerPipeline.from_pretrained("HorizonRobotics/3D-Fixer")

If you prefer loading the model from local, you can download the model files from the links above and load the model with the folder path (folder structure should be maintained), download the MoGe v2 ckpts, and modify the scene_cond_model in /path/to/3D-Fixer/pipeline.json to /path/to/MoGe v2 ckpts. Then use 3D-Fixer as follows:

ThreeDFixerPipeline.from_pretrained("/path/to/3D-Fixer")

Usage

Launch Demo

We actively fixing a small bug of input plugin. Stay tuned.

Evaluation

We provide the inference and evaluation code on our test set, Gen3DSR test set, and MIDI test set.

ARSG-110K test set

Please download the ARSG-110K-testset.zip and object_assets.zip from here, and unzip the files. ARSG-110K-testset.zip contains the scene data of our test set, and object_assets.zip contains our pre-processed object assets from Toys4K. Then you can run the following commands to perform inference:

python inference_ours_testset.py \
    --output_dir {PATH_TO_SAVE_RESULTS} \
    --testset_dir {PATH_TO_ARSG-110K-testset} \
    --model_dir {PATH_TO_LOAD_PRETRAINED_MODELS} \
    --rank 0 \
    --world_size 1

After running inference, you can use the following commands to get the evaluation metrics:

python eval_metrics_ours_testset.py \
    --output_dir {PATH_TO_SAVE_RESULTS} \ 
    --testset_dir {PATH_TO_ARSG-110K-testset} \
    --assets_dir {PATH_TO_object_assets}

Gen3DSR test set

Please follow the instruction from Gen3DSR to download the Gen3DSR test set. And download the pre-segmented masks from here, which we generate using the code from Gen3DSR. Put the pre-segmented masks in the Gen3DSR test set, and run the following code to perform inference:

python inference_gen3dsr_testset.py \
    --output_dir {PATH_TO_SAVE_RESULTS} \
    --testset_dir {PATH_TO_Gen3DSR_TESTSET} \
    --model_dir {PATH_TO_LOAD_PRETRAINED_MODELS} \
    --rank 0 \
    --world_size 1

After running inference, you can use the following commands to get the evaluation metrics:

python eval_metrics_gen3dsr_testset.py \
    --rec_path {PATH_TO_SAVE_RESULTS} \ 
    --data_root {PATH_TO_Gen3DSR_TESTSET}

MIDI test set

Please follow the instruction from MIDI to download the MIDI test set. Then run the following code to perform inference:

python inference_midi_testset_parallel.py \
    --output_dir {PATH_TO_SAVE_RESULTS} \
    --testset_dir {PATH_TO_MIDI_TESTSET} \
    --model_dir {PATH_TO_LOAD_PRETRAINED_MODELS} \
    --rank 0 \
    --world_size 1

After running inference, you can use the following commands to get the evaluation metrics:

python eval_metrics_midi_testset.py \
    --output_dir {PATH_TO_SAVE_RESULTS} \ 
    --testset_dir {PATH_TO_OURS_TESTSET}

Citation

@inproceedings{yin2026tdfixer,
  title={3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image},
  author={Yin, Ze-Xin and Liu, Liu and Wang, Xinjie and Sui, Wei and Su, Zhizhong and Yang, Jian and Xie, jin},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  year={2026}
}