It runs on Windows. Full details can be found here. #8
Comments
|
Thanks for confirming! I've already updated the README to reflect Windows support. |
|
I tested images(object) in different styles, and I found that Trellis can't correctly construct even the basic shapes, while Spar3D at least gets the large shapes right, though the details are still lacking. try this: |
|
If I had to guess, the inability to construct the basic shapes is due to the source image being 2D. Trellis seems to assume that you're giving it a real-world photo of an object (or at least a 3D render of one). Anime images are 2D and the shading is insufficient to adequately define the forms which are needed for reconstruction. To test this assumption, I found two images of comparable anime figures, but rather than use the 2D cel shading, I found 3D printed objects. I ran these through both Trellis and Spar3D. In the first test, I found a high resolution image (over 1024x10124) and used that. In order to ensure that the results weren't skewed due to improper background removal, I removed the background in Photoshop and sent the transparent PNG directly to both Trellis and Spar3D. Here are the results.
In the second test, I used a low res image (400x400) and fed it though both generators. Again, the background was removed in Photoshop first so as to not skew the results.
Neither generator produced perfect results, although it's clear that Trellis did a better job at capturing the broad forms. Compare the legs on the models from the high resolution image:
Compare the way that Trellis synthesized the back of the hair compared to Spar3D:
I don't think either of these models are usable for production, but there's no doubt that the Trellis model is better than the Spar3D model and would provide a stronger base to begin hand modeling from. Additionally, given the complex nature of these figures (body, hair, clothing, accessories), It's a lot to ask for any 3D reconstruction. If I were to do this for production, I'd try to generate the body separately from the clothes, and the accessories separate from those also. Then I'd assemble everything in my back end 3D app. |
|
In 2023, I tried generate dynamic models in pipeline as you said: hair, face, head, body, hand, garment, ... But this workload is overwhelmingly heavy, especially since I am conducting research and development on my own. An old sample: It looks the 'Structured Laten' in Trellis is very effitive. :-) |
|
Generating a mesh that can be animated from an image to 3D tool is going to be very difficult. The topology of models coming from these tools simply doesn't meet the requirements for animation. I looked at the sample you provided. I didn't see anything intrinsically unique about that character that would require image to 3D tools. It seems like you could use one of the many free character tools out there to get something very close to what you showed. Blender has several good ones. MakeHuman is also good if you just need a simple clothed character that you can go on to customize. Perhaps I'm missing something. Feel free to correct me if I'm misunderstanding your goals. |
|
My developing vesion and ulimate goal is: My old sample, the input is the face image and the action-text, the outputs include different mesh: head-with-face, hair, body, garment, and more. |
|
I also found Spar3D in general being coarser than Trellis. Spar3D is much faster and the point editing feature is useful on the other hand though. I feel for fast prototyping this might be the better choice, but for detailed modeling bigger models like Trellis could be better. Wouldn't say either is the gold standard |
This may be achievable by AI in the future, but it's a ways off. If you haven't already looked into it, Reallusion's Character Creator 4 (CC4) does this already. If you use the Headshot plugin, it will allow you to create a full 3D character from 1 or more photos. https://www.reallusion.com/character-creator/headshot/ The base meshes have sliders that you can use to set body type. Clothing can be purchased or created relatively easily using tools in other 3D apps. High quality hair cards can be added in CC4 or using the free tools in Blender. The CC4 character export to Blender includes a pipeline tool that will apply Rigify. That rig can then be connected to your webcam to transfer real-time animation back to your model. CC4 isn't open source. It's not expensive though for what you're looking to do. Having worked as a Character Artist in AAA games, I can tell you that our internal pipline was extremely complicated. As an independent, I'd strongly recommend using CC4 as my main character tool. The wealth of support, tutorials, etc. are worth it. |
|
Thanks @realstevewarner, you are so kind. |
I can confirm that the install works on Windows 10. Generation time on a 4090 is about 3 to 5 seconds.
I created a conda environment and set it to use Python 3.10. I cloned the repo and followed the instructions. I never got an error. Unlike Trellis, which took about 12 hours of back-and-forth to get installed, this went over without a hitch.
Results are acceptable for very distant objects or as a very rough starting point for building out a custom mesh from scratch, but they still fall short of Trellis, which I consider to be the gold standard as of January 9th, 2025.
Here you can see the results of Trellis compared with Spar3D.
I created a lengthy post on X about my experience with Spar3D (S3D), including an exploration of settings, remesh results, etc. You can find that here if you're interested:
https://x.com/SteveWarnerFL/status/1877422143673082151
The text was updated successfully, but these errors were encountered: