A new tool - artificial intelligence to standardize actor images

[chef 3746]

Hi,

I have finally been successful in creating a tool using AI to help standardize Actor Images for Emby.

This was no easy task. 

I thought I'd post and see if there was interest in a utility like this.

Here are some examples of the subtle changes the utility will make to the person images to make them uniform in the UI.

 

Before Images:

         

 

After Images:

          

 

The AI targets the background of the image, and then we create a  shadow behind the actor.

Of course the background image is completely customizable. You can make the background the actor is standing in front of anything you want. These images just use a white background to show the shadow a little more.

 

I think this creates subtle changes that may have a nice impact on the actors row in emby clients.

 

 

Edited September 4, 2022 by chef

[neik 837]

YOU are the man.

Very nice little plugin that unifies the look of the actor images.
Would love to give it a try if you are willing to share.

[chef 3746]

6 minutes ago, neik said:

YOU are the man.

Very nice little plugin that unifies the look of the actor images.
Would love to give it a try if you are willing to share.

I will absolutely share with you @neik !!

It isn't a plugin... Or maybe it should be... That part I'm not entirely sure about.

Right now it is an executable which runs, however... It could be a plugin. 

It's written in python for the AI, and in c# for rendering the actual finished product.

 

 

 

Edited September 4, 2022 by chef

[Spaceboy 2494]

looks great!

[Oracle 74]

I'm interested!

[chef 3746]

This currently involves installing the python interpreter on a computer on  your network (it doesn't have to be your emby server).

are people still interested in that?

Edited September 5, 2022 by chef

[chef 3746]

Oh yeah! I just uninstalled everything, and reinstalled it all again... just to see how hard it would be.

It is super easy.

And I created more pics in seconds!

  

 

 

Edited September 5, 2022 by chef

[neik 837]

chef, just wondering how good it does work with anime characters?

[chef 3746]

13 minutes ago, neik said:

chef, just wondering how good it does work with anime characters?

Should work great!

[slevin7 54]

That is absolutely impressive what you are creating. 
I am of course very interested 

[fbrassin 35]

Wow, you really made i great job.

You did something i would never thought it should have been possible. 

[chef 3746]

3 hours ago, slevin7 said:

That is absolutely impressive what you are creating. 
I am of course very interested 

 

I looked everywhere online for a way to remove backgrounds, but all there was was paid APIs.

Then I decided that I would learn how python worked since python seems to have the best AI possibilities.

Luckily my friend @samuelqwe is a secret master at python and he answered a lot of questions.

@rbjtech and @Cheesegeezer helped me keep the new images looking classy.

 

I trained pytorch with a person detection model. Then it trains a background model.

Once the image is segmented, I use c# to overlay the Mat image with the nice and clean backdrop.

I think this only runs on windows at the moment though. But I'm learning how to build it cross platform today.

All the components are there, I just need to make sure that the AI builds cross platform. 

The first time you run the app, it will train the AI. It takes about 10 minutes. 

After that the models are ready, and images are created in seconds. 

Edited September 5, 2022 by chef

[Cheesegeezer 3087]

Looking hot Chef Benny!!!

[CBers 6771]

Looks good Chef

Would be nice to have uniform images of actors.

I assume it just takes your current image and creates the same but with the background you choose?
 

[chef 3746]

4 minutes ago, CBers said:

Looks good Chef

Would be nice to have uniform images of actors.

I assume it just takes your current image and creates the same but with the background you choose?
 

Yes. You can create a background in Photoshop, and then copy it into the execution directory. That image will be used.

For instance, my first attempt I created these using Emby green.

But after staring at them long enough, Rich Dave and I realized that they are not as nice looking as the white with backdrop shadow.

they kinda just look cutout. 

the white backdrop with shadow looks a lot more classy.

[ebr 14918]

IF (and that is a big if) this performs adequately (< 1/10 sec per image) you could implement this as an imageProcessor plug-in but be sure it only targets people images.

In reality, though, it is better as some sort of process that alters the actual images as that will be much more efficient (one and done whereas an image processor will recreate these every 30 days).

[chef 3746]

22 minutes ago, ebr said:

IF (and that is a big if) this performs adequately (< 1/10 sec per image) you could implement this as an imageProcessor plug-in but be sure it only targets people images.

In reality, though, it is better as some sort of process that alters the actual images as that will be much more efficient (one and done whereas an image processor will recreate these every 30 days).

I would love to see this as a plugin. 

Definitely only targets people. The url is non negotiable. 

...and only works with emby.

Large person images (the really big ones)  take about 3-4 seconds. 

Mid size images (most of the people images are this size) take less then half a second.

Currently, Python interpreter has to be installed.

I tired to build a standalone executable, but I failed..  still have some python stuff to learn.

It's still kind of infant, but it is consistent in it's production.

I've had one or two images that the AI had issues with, but I've process thousands of perfect ones.

Considering that online apis charge upwards of 0.10 cents per images. I think this application is worth a dollar or two donation I may ask for it.... When it is ready. 

[chef 3746]

This fast enough?

I was thinking if the process was threaded you could do more then one at a time.

So, two or three in that amount of time (about 2-3 seconds)

1 second per image.

And I'll fix it to process everything in memory which will also make it much faster. Right now it is writing files. 

It doesn't have to do that.

 

 

Edited September 5, 2022 by chef

[ebr 14918]

The more I think about it the less I think this should be an ImageProcessor because these only need to be done once for the entire life of the image.  So some sort of process that runs as a scheduled task would make more sense probably.

The tricky part is that people images are not downloaded until requested by an app and you don't want this process to cause the download of ALL people images.

[chef 3746]

7 minutes ago, ebr said:

The more I think about it the less I think this should be an ImageProcessor because these only need to be done once for the entire life of the image.  So some sort of process that runs as a scheduled task would make more sense probably.

The tricky part is that people images are not downloaded until requested by an app and you don't want this process to cause the download of ALL people images.

I was thinking maybe  a scheduled task that checks the creation date of a person image (dealing with the actual file system to check dates).

If the creation date is greater then that of the last run time of the task, we can run the image processor on the actor image.

Maybe...

If the file exists then process it, otherwise find the next existing primary image in the metadata folder.

 

Edited September 5, 2022 by chef

[chef 3746]

Hold the phone!!

I just realized that you can run this with cuda cores on a GPU.

This now looks for cuda and if it finds it, you can use it.

If we thread the process then technically the AI runs using the gpu, and the processor just mattes the image. 

We could thread this thing 20 times over... Well... maybe not 20... But one heck of a lot faster rendering on the GPU if there is one. 

 

I officially might love python. ..

[Spaceboy 2494]

4 hours ago, ebr said:

you don't want this process to cause the download of ALL people images.

you might...

over a period of time obviously

[chef 3746]

BOOM! It's a web service!!

 

Now we can use the Javascript ApiClient to talk to Emby.

Edited September 6, 2022 by chef

[chef 3746]

20 hours ago, ebr said:

The more I think about it the less I think this should be an ImageProcessor because these only need to be done once for the entire life of the image.  So some sort of process that runs as a scheduled task would make more sense probably.

The tricky part is that people images are not downloaded until requested by an app and you don't want this process to cause the download of ALL people images.

I have 10 000 items in my people folder.

It's a big task for Emby, maybe it is best that this runs as an external app.

I was able to build a web service into it,  that hosts some html, and js.

It might be a good idea to have this service accessible on the network to process the image for Emby.

A local provider of sorts (that might be kind of cool).

The project still needs the python interpreter installed, and might only work on windows (still untested cross platform).

Maybe having this available to emby on the network, but not necessarily on the same machine might also be a benefit. 

 

Edited September 6, 2022 by chef

[slevin7 54]

if you like, i could test this on ubuntu and macos.