Abstract: Reconstructing detailed three-dimensional (3D) models from multi-view images often involves a trade-off between efficiency and fidelity. Existing methods based on volumetric representations or dense meshes can be computationally expensive, while primitive-based methods struggle to capture fine geometric details. We propose a novel method that addresses this challenge by combining differentiable primitive abstraction with adaptive mesh refinement. Our method first abstracts the scene into a set of cuboid primitives represented by analytical signed distance functions (SDFs), enabling part separability. This stage leverages differentiable volume rendering to efficiently optimize the primitives’ poses and sizes. Subsequently, an automatic coarse-to-fine refinement procedure, guided by rendering loss, restores fine geometric details. Our method yields high-quality, part-separable meshes with low geometric complexity, suitable for applications requiring part manipulation and efficient rendering. We demonstrate the effectiveness of our method on the Technical University of Denmark Multi-View Stereo (DTU-MVS), BlendedMVS, and Tanks and Temples datasets, achieving a better balance between mesh complexity and reconstruction fidelity compared with existing techniques.