AI Pose Creator for DAZ Genesis 8/9

[Otto Von Herunterhängen]

I haven't bought or used it yet, I am going to, will give you a review. It's had a couple of reviews already. Now THIS is what DAZ needs to get on top of.

It needs to be a picture of a single person, whole body. The program translates it into a DAZ pose. Stand alone program, you do need a video card that supports AI, but it's an executable, you don't need to install a runtime environment. No Internet connection needed for the program to run.

Image to Genesis 8/9 Pose Generator AI V1



Edit: Just bought it. 3.6 GB program, haven't tried it out yet.

 

[Otto Von Herunterhängen]

It doesn't do expressions, just bodies, and it doesn't generate a thumbnail, you'll need to do that yourself. And you sometimes need to change the hip translation. But um, wow, pretty good.

 

[Otto Von Herunterhängen]

Hmmm, you can also generate the original pose with AI

One of Nyghtfalls...

 

[MCDesigns]

Looks like the pose creators just lost their jobs to AI

 

[Otto Von Herunterhängen]

Looks like the pose creators just lost their jobs to AI

More the entire motion capture industry I would think, but the important thing is what @TailsWin was saying, this isn't something I needed to go to a data center for, it's a little 4 GB stand alone program running on my PC that only requires 4 GB of RAM on the video card and can translate any photo of a single human posing into a 3D pose file... in less than 10 seconds.

It is special purpose to DAZ3D because that's how the programmer is marketing it, but I assume it can be written for any 3D engine.

Here's one picture I really liked because of the pose and expression from @JohnM PKF Tour Group Execution (fully clothed, so it can go on the general side) But it is really poor resolution, 124 KB, and not even full body, so the program had to do a bit of guessing. And of course I had to recreate the expression. 10 seconds of program time, a couple of minutes of adjustment in DAZ

 

[MCDesigns]

Just goes to show how funky the rigging on DAZ figure are compared to real life .

 

[TailsWin]

Interesting. I'd be curious to know what the architecture of the model is. Probably a transformer of some sort, but funnily enough, this is probably something a convolutional neural network (an older method that was all the rage just before transformers) could do too, at least for stills. I guess it's just that nobody thought about it before this AI boom, or at least I've not heard about it.

Just yesterday I was watching something about mocap for a videogame, and it actually surprised me they still do that. A full room of rigs, a scaffolding the actor has to wear in front of their face, and days to transfer the scene into data. You'd think this would be one of the first things to use AI for, since it's so expensive and mocap studios have such high-quality data to train on. They could offer a better service for a tiny fraction of the cost. And you'd still need actors, animators and other people, so this could actually be a good thing for everybody, and a way to offer the movie/game studios an alternative before they decide to just AI-generate everything.

 

[Otto Von Herunterhängen]

Did a couple more, generated the poses with AI then used the program to translate them into 3D poses. There is some minor tweeking that needs to be done to all of them, but I am really pleased with the results. "Dead" poses are hard to find, and I can quickly generate more than enough this way.

These don't match up exactly with the originals because the camera angle is different, I have it set to do the thumbnails. I did do #5 in as close to the original camera angle as possible.

The AI Poses followed by the 3D ones

 

[MCDesigns]

Good examples. Will have to pick this up if i ever get back into DS.

 

[Otto Von Herunterhängen]

Interesting. I'd be curious to know what the architecture of the model is. Probably a transformer of some sort, but funnily enough, this is probably something a convolutional neural network (an older method that was all the rage just before transformers) could do too, at least for stills. I guess it's just that nobody thought about it before this AI boom, or at least I've not heard about it.

Just yesterday I was watching something about mocap for a videogame, and it actually surprised me they still do that. A full room of rigs, a scaffolding the actor has to wear in front of their face, and days to transfer the scene into data. You'd think this would be one of the first things to use AI for, since it's so expensive and mocap studios have such high-quality data to train on. They could offer a better service for a tiny fraction of the cost. And you'd still need actors, animators and other people, so this could actually be a good thing for everybody, and a way to offer the movie/game studios an alternative before they decide to just AI-generate everything.

Question for you, I'm in a discussion over on Renderhub where they are accusing this guy of violating IP rights by training AI on vendor pose files. I'm taking the position he probably generated 25,000 pose files via script to get rotations to match his output from the picture file. I don't see how the part that writes the duf could be trained from random vendor pose files.

How would you go about writing that part?

 

[Nyghtfall3D]

I'm taking the position he probably generated 25,000 pose files via script

The vendor replied to the thread themselves and acknowledged they wrote a script.

 

[Otto Von Herunterhängen]

The vendor replied to the thread themselves and acknowledged they wrote a script.

Yes I saw, they pretty much did exactly what I thought they had for exactly the reasons I thought they did it that way. They also thanked me for taking up for them in an email

 

[TailsWin]

Interesting. Frankly I wouldn't know either way. Why the need to train a model if you can use a script then?

If anything, I would've thought he had photorealistic pics AI-generated out of existing 3D poses/renders. That would be a super straightforward way to get data pairs to train on.

 

[Otto Von Herunterhängen]

Interesting. Frankly I wouldn't know either way. Why the need to train a model if you can use a script then?

If anything, I would've thought he had photorealistic pics AI-generated out of existing 3D poses/renders. That would be a super straightforward way to get data pairs to train on.

He used the script to generate the references to map the incoming pose to the 3D rigging of the character. For that you need both the picture of the pose and the resulting pose file. Apparently he did use AI training on both ends, to break down the incoming picture into a pose, then to map that pose onto 3D rigging.

The basic question was, if he used 25,000 pictures to do this, where did he get those pictures from. It turns out he generated them from the pose files he created, which as far as I was concerned was the only sensible way to do it. The other method would have been to render vendor poses then use those pose files to do the mapping, which would have been far less efficient.

And of course he got dog piled with "He's using stolen IP for training!" No, he didn't.

 

[TailsWin]

Considering the sheer output some of you comic artists have, having own 25k poses to make a training dataset isn't out of the question Which is why I said this would've been easy for any mocap studio to do, since they have the source images, the resulting poses in 3D space, as well as metadata to link them together.

Btw I wonder what kind of IP are we talking about anyway. Is moving a 3D skeleton into a particular pose even copyrightable in the first place? I'm not sure whether poses alone would qualify as IP regardless, but I don't know if that's ever been tested.