This project presents an innovative model that converts textual prompts into 3D meshes. Utilizing advanced Neural Radiance Field (NeRF) techniques combined with a new corner view image generation model, our project transforms verbal descriptions into detailed 3D meshes. By simplifying the modeling process, this makes it accessible to all, including those with disabilities. The core of the innovation is in the model’s use of speech input in combination with the newly developed text-to-3D model. This creates a seamless transition from text to 3D mesh. This leads to the opportunity for immediate application into educational settings. This is where it can provide visually impaired students with a way to physically understand and interact with topics through 3D printing these created models. Farther reaching than just education, this technology can impact virtual reality and different types of media design where we see 3D modeling becoming more and more prevalent. In addition to advancing 3D modeling, this tool includes a broader group in 3D design. Please find the code to our project at:  https://github.com/JHand11/Speech-Driven-3D-Modeling

• This report represents the work of one or more WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its website without editorial or peer review.

Creator

• Hand, Jacob
• Antes, Ryan

Publisher

• Worcester Polytechnic Institute

Identifier

• E-project-042524-121336
• 121692

Keyword

• Artificial Intelligence
• Machine Learning
• 3D Modeling

Advisor

• Zhang, Ziming

Year

• 2024

Date created

• 2024-04-25

Resource type

• Major Qualifying Project

Major

• Electrical & Computer Engineering

Source

• E-project-042524-121336

Rights statement

• In Copyright