Meta's SAM 3D converts 2D images to detailed 3D models for e-commerce, therapy & robotics.
Facebook parent company Meta has introduced SAM 3D, a new technology designed to transform 2D images into highly detailed 3D representations of objects, humans, and entire scenes. By accurately capturing both shape and pose, SAM 3D enables interactive three-dimensional modeling suitable for applications ranging from e-commerce and physical therapy to robotics. The system reconstructs geometry and texture with precision, offering a practical approach to bringing digital objects and human movements into immersive 3D spaces.
Advanced 3D Modeling of Objects, Humans, and Scenes
SAM 3D functions through two specialized models: SAM 3D Objects and SAM 3D Body. SAM 3D Objects is designed to transform a single image into a detailed 3D reconstruction of masked objects. The model captures both geometry and texture, producing high-fidelity 3D representations that can be independently posed and manipulated for interaction. Its reconstructions are robust even when parts of the object are partially occluded in the input image, ensuring accuracy in real-world conditions. Additionally, SAM 3D Objects can integrate multiple items into a single scene, allowing seamless joint positioning alongside human reconstructions generated by SAM 3D Body.
SAM 3D Body focuses on accurately reconstructing human body shape and pose from a single image, even when parts of the body are partially hidden or in unique positions. The model creates manipulable 3D representations suitable for interaction and practical applications, such as motion analysis or virtual scene integration. SAM 3D Body also supports joint reconstructions, enabling multiple people to be positioned together in a shared 3D environment alongside objects generated by SAM 3D Objects.
Driving Research and Supporting Real-World Applications
In practical terms, SAM 3D has applications across multiple industries. In e-commerce, it allows platforms like Facebook Marketplace to place 3D augmented reality overlays of products, such as furniture, in users’ homes, helping visualize fit, scale, and style before purchase. In healthcare, SAM 3D Body uses AI-driven pose estimation to provide detailed 3D analysis of patient positions, enabling personalized rehabilitation programs and improved outcomes. In robotics, the system supplies accurate 3D shape and position information, supporting complex tasks such as object manipulation and retrieval, and enhancing efficiency and reliability in automated operations.
Meta reports that SAM 3D achieves performance exceeding current state-of-the-art benchmarks for both object and human reconstruction. To further research in this field, the company will soon unveil the SAM 3D Artist Object Dataset, a collection of diverse real-world images and 3D meshes. The dataset is designed to challenge existing reconstruction methods and encourage progress toward visually grounded 3D modeling, moving the focus from curated or synthetic data to real-world perception.
AI-Powered 3D Creation Across Industries
SAM 3D is part of a growing wave of AI technologies that transform 2D images into fully realized 3D models, bridging the gap between traditional imaging and interactive, manipulable 3D content.
Earlier this year, NVIDIA introduced PartPacker, an AI-powered system capable of generating editable 3D models from a single image. Unlike traditional approaches that produce a single, unified mesh, PartPacker generates part-based models, allowing designers to edit or animate individual components independently. This flexibility makes it a valuable tool for applications ranging from 3D printing and animation to gaming and research, providing a faster and more customizable path from concept to finished 3D asset.
Similarly, Chinese tech company Tencent has launched its Hunyuan 3D creation engine, an AI-powered platform that quickly generates detailed 3D assets from multimodal inputs such as text prompts, reference images, or sketches. By accelerating processes that usually take days or weeks, the engine simplifies 3D content creation for a wide range of applications. To support enterprise adoption, Tencent Cloud has introduced the Hunyuan 3D Model API, allowing companies to integrate the technology into workflows for gaming, online retail, VFX, advertising, social media, 3D printing, and other digital production.
