Emby Blog

2 minutes ago, ember1205 said:

Oh... On mine, is specifically tells me the directory and file name that was written. What OS are you running?

 

running Debian on a raspi. I used winscp to search where it said the txt doc was and I couldn't find it. /config/passwordreset.txt

2 minutes ago, AP123 said:

running Debian on a raspi. I used winscp to search where it said the txt doc was and I couldn't find it. /config/passwordreset.txt

Try /var/lib/emby/config/passwordreset.txt?

1 minute ago, ember1205 said:

Try /var/lib/emby/config/passwordreset.txt?

sorry dont' have that directory. after everything I've done the last 24 hours, this last step is going to kill me

1 minute ago, AP123 said:

sorry dont' have that directory. after everything I've done the last 24 hours, this last step is going to kill me

Does WinSCP have a search function ? Move to "/" and search for passwordreset.txt

If not, you need to log in via SSH (PuTTY) and execute "find / -name passwordreset.txt" to locate that file

3 minutes ago, ember1205 said:

Does WinSCP have a search function ? Move to "/" and search for passwordreset.txt

If not, you need to log in via SSH (PuTTY) and execute "find / -name passwordreset.txt" to locate that file

found the text doc by searching just for passwordreset.txt but the link doesn't work. fun...

2 minutes ago, AP123 said:

found the text doc by searching just for passwordreset.txt but the link doesn't work. fun...

Hi, what exactly do you mean by "doesn't work"?

4 minutes ago, Luke said:

Hi, what exactly do you mean by "doesn't work"?

I figured it out. the IP address it was directing me to was incorrect. I changed the IP in the link provided to my emby server IP and it worked. literally the link "was broken" it said "link does not work, it took too long to load. the "in home LAN access" address is not correct and that's what the text file was directing me to. don't know why it isn't the IP of my server

Edited May 26, 2023 by AP123

3 minutes ago, AP123 said:

found the text doc by searching just for passwordreset.txt but the link doesn't work. fun...

There's a time limit on it... maybe you will need to click the Reset button again for a new one?

6 minutes ago, AP123 said:

I figured it out. the IP address it was directing me to was incorrect. I changed the IP in the link provided to my emby server IP and it worked. literally the link "was broken" it said "link does not work, it took too long to load. the "in home LAN access" address is not correct and that's what the text file was directing me to. don't know why it isn't the IP of my server

Have you customized any emby server network settings?

Just my personal opinion. This whole episode of server compromising was a cluster f*ck that no one wanted and desired it. At this point the Devs don't need more should have could have. For me since I have been using their content organization solution since MediaBrowser, they earned a OK, we screwed up. I consider this as them just paying a boatload of tuition at the school of hard knocks.

Note to self, dial down the rhetoric and up the helpful suggestions.

Edited May 26, 2023 by One2Go

3 minutes ago, Luke said:

Have you customized any emby server network settings?

no clue what that means. so I'll just go ahead and say no. 

All I need to do now is update my emby from portainer. Again, someone did this for me so I have no idea how to do it. Any CLEAR STEP BY STEP instructions would be amazing. This is not meant to demean those trying to help, but giving half answers and assuming everyone here is a pro could lead to personal frustration for some.

7 hours ago, Q-Droid said:

The instructions could be reordered for clarity. Until they are approach the action plan like a punch list. Complete the items you can and come back to the ones you can't due to dependencies like a running server. And for everyone the very first step should be to disable router port forwarding, firewall rules, gateway rules that allow the remote access.

That's a nice idea, but then we have people like myself who run Emby (and many other apps) in Docker, which causes other issues. The lack of information about the DLL file that was injected was also frustrating my work, because due to Docker, I had no way to bring up the server without network access.

While I appreciate the work that they did to shut down the botnet, the 'full story to come' with next to no 'partial story to start with' is incredibly frustrating. It gave me no useful information to help understand what kind of threat my server was under. A few other things I'm now also noting:

The infection for me (and others I've seen here) started on May 11th, 2023, using an exploit first laid out on February 2nd, 2020. Not only is there the question of why an exploit allowing for high level penetration was allowed to languish until after it was actually weaponized, but I remember that same day I got a ton of issues with other sites.

My Emby does have a FQDN for outside home-access. The day the attack happened all of my other subdomains suddenly got 'malicious attack' warnings from Google and Firefox, and a YouTube video that had a link on one of the other subdomains got a community strike for 'scam activity'.

I appealed the community strike because, based on the lack of any other information, I saw nothing suggesting it wasn't anything more than a technical hiccup. Maybe an IP change at Cloudflare, or a momentary cert error with LetsEncrypt, or something. I asked YouTube for more information, and while the strike was removed, I never got any technical feedback.

This is an issue with the larger Internet, where small time domain holders may not have the resources to find out when they're threatened.

6 hours ago, rbjtech said:

A RP will re-write it's own headers - so it's highly unlikely that vulnerability passed through your RP.  

Which means that when I was compromised it was IP based and not FQDN based?

As I've said before, my install was dockerized, and I had an apache config for the subdomain name, which to connect the two would have required a reverse proxy.

6 minutes ago, AP123 said:

All I need to do now is update my emby from portainer. Again, someone did this for me so I have no idea how to do it. Any CLEAR STEP BY STEP instructions would be amazing. This is not meant to demean those trying to help, but giving half answers and assuming everyone here is a pro could lead to personal frustration for some.

I don't have a good answer for you on how to upgrade via Portainer. From the command-line of the Docker host itself, you -should- be able to issue this command: 

docker pull emby/embyserver:latest

2 hours ago, ember1205 said:

Other comments indicating that extensive investigation was done, so it isn't unreasonable to believe that at least SOME examples of data exfiltration are know and could be shared as things others may have reported.

I do understand that the likelihood of being able to nail it down to an explicit list is almost impossible, but it would be useful to at least share some of the things that are known to have occurred.

I guess I am not following what you are asking for.  You would like lists of the exact data that was stolen or???

Sounds to me like this was mainly be used as a botnet vector using automated tools and less the type of attack where individual users are analyzing every computer to inflict maximum damage or exfiltrate data to hold for ransom.  The biggest concern I would have as an end user that was compromised would be specifically related to other credentials stored on the compromised server as there are many automated scripts which can scan systems to harvest credentials that can be fed back to teams of people who can leverage those to cause further damage.  Additionally, if anyone was storing crypto keys for cryptocurrency on these systems there are many automated programs to target those as well.

2 hours ago, ember1205 said:

Other comments indicating that extensive investigation was done, so it isn't unreasonable to believe that at least SOME examples of data exfiltration are know and could be shared as things others may have reported.

I do understand that the likelihood of being able to nail it down to an explicit list is almost impossible, but it would be useful to at least share some of the things that are known to have occurred.

I guess I am not following what you are asking for.  You would like lists of the exact data that was stolen or what???  It's not like Emby is a security system that, once installed, can know everything else that is happening on the computer.

Sounds to me like this was mainly be used as a botnet vector using automated tools and less the type of attack where individual users are analyzing every computer to inflict maximum damage or exfiltrate data to hold for ransom.  The biggest concern I would have as an end user that was compromised would be specifically related to other credentials stored on the compromised server as there are many automated scripts which can scan systems to harvest credentials that can be fed back to teams of people who can leverage those to cause further damage.  Additionally, if anyone was storing crypto keys for cryptocurrency on these systems there are many automated programs to target those as well.

1 hour ago, Scott D said:

Not sure what flavor of windows you are running, but mine is shut down on an almost monthly basis.  All in the name of installing security updates.

I've never had that happen. Like I said if an update fixes it I'm fine with that. So far it hasn't and I have to change system files on my own to fix it. If you are referring to shutting down and restarting to install update, that's fine. This is not the same. My Emby is shut down and will not restart even with an update.

Howdy everyone,

I'm late to post here, but have been keeping an eye on the subject-matter since the 11th . We had someone attempt to gain access on the 11th as well. MOST users had a local password set already so we were okay but our logs did show someone attempted to access the server by spoofing the local IP 127.0.0.1.  The one user (with minimum access) was accessed due to no local password (a new account that was being setup the day prior so that slipped past my watch) so we deleted that user profile for safety measures and they had to use a new user name/password. I also no longer list names locally either. It doesn't look like they went through our domain so i got a new IP from my ISP as an extra precaution and im rerouting the IP associated with our domain name so users won't have to even notice the change.

Also with the attempt on the 11th, i setup fail2ban to work with emby and it also looks at local IPs and will ban them for invalid attempts as well as global IPs. I would highly recommend folks to set up fail2ban. I followed a few of the posts i found in this forum to set mine up and did a few duckduck searches 2 get the cloudflare synchronization with it working as well (in case they ever did go through the domain name in the future). I wish I could post the steps I took but I had help setting it up since I just recently jumped ship from windows a few months ago so I'm still learning.

Emby team, thanks so much for your quick response & patch. I just updated my server today when I woke up.

edit: ps. after very close careful examination of our log files, there was no plugins added and those .dll files were not installed thankfully.

Zander

Edited May 26, 2023 by Zander3768

12 minutes ago, moviefan said:

I guess I am not following what you are asking for.  You would like lists of the exact data that was stolen or what???  It's not like Emby is a security system that, once installed, can know everything else that is happening on the computer.

Sounds to me like this was mainly be used as a botnet vector using automated tools and less the type of attack where individual users are analyzing every computer to inflict maximum damage or exfiltrate data to hold for ransom.  The biggest concern I would have as an end user that was compromised would be specifically related to other credentials stored on the compromised server as there are many automated scripts which can scan systems to harvest credentials that can be fed back to teams of people who can leverage those to cause further damage.  Additionally, if anyone was storing crypto keys for cryptocurrency on these systems there are many automated programs to target those as well.

When vulnerabilities are divulged, details about the items that are/were at risk are disclosed with them. NONE of the actual risks have been provided. The only statement that has been made is "Analysis of the plug-in has revealed that it is forwarding the private Emby Server login credentials including the password for every successful login to an external server under control of the hackers." 

 

Why was Scripter-X / ScripterX not mentioned as the actual custom plugin that they loaded to create the backdoor? Why was 5/11/2023, approximately noon EDT / 9AM PDT not mentioned as when the hack initially occurred? If not data was "stolen" (aside from login credentials), why was there so much data leaving my network from my Emby server? Why was the vast majority of that data being sent to GitHub, Princeton University, and AWS? What would the so-called botnet possibly want with server login credentials outside of looking to pull content off of the servers?

 

After significant analysis and research Emby determined that this was a critical vulnerability that required shutting down my server but they don't know what data is/was actually at risk? If there's no understanding of what data is being pulled off of the machines, how was it determined that this was so critical? There's a LOT more to this story that isn't being shared with the community. And for those like myself that have invested into the community with extensive amount of time and have purchased the Premiere license, being treated this way is disgraceful.

17 minutes ago, hotwheels26180 said:

I've never had that happen. Like I said if an update fixes it I'm fine with that. So far it hasn't and I have to change system files on my own to fix it. If you are referring to shutting down and restarting to install update, that's fine. This is not the same. My Emby is shut down and will not restart even with an update.

If that's what you believe, you aren't paying attention. With Windows 10, MS started shoving restarts and patching down your throat. You aren't allowed to opt out. My hard drive is encrypted and requires a password to unlock - I'm always painfully aware of when they have stuck their collective nose where it absolutely does not belong.

1 hour ago, ember1205 said:

The only statement that has been made is "Analysis of the plug-in has revealed that it is forwarding the private Emby Server login credentials including the password for every successful login to an external server under control of the hackers." 

Hi.  That's the only thing we're sure of but there is really no way to know what else may have been done.  We haven't positively identified anything else at this point.  We have no evidence that anything outside of Emby credentials were taken/compromised.

1 hour ago, ember1205 said:

What would the so-called botnet possibly want with server login credentials outside of looking to pull content off of the servers?

Many people use the same credentials on multiple sites.  The supposition would be that the Emby credentials might be duplicated elsewhere where there would be more value to them.

 

3 hours ago, ember1205 said:

but they don't know what data is/was actually at risk?

They knew that potentially all of it was at risk; but they have found no evidence of any data actually being taken than they have already described, in the systems they have access to analyse.  In general, how far the danger extends depends on the user's setup. 

The vulnerability itself enables external access to be made to appear local, but this has no significant effect unless there are users who could log in without password on the local system - this still only gives access to looking at movies and so on unless there is an admin user which can be logged in locally without password, when it becomes possible for the hacker to install a plugin that happens to be able to download other code - but even this is still restricted to the Emby system itself, unless the Emby server is being run under an OS admin account, at which point the whole machine becomes vulnerable.

So no - they can't know just how much at risk your system was - that's for you to determine, as you know about those settings which Emby doesn't.

Paul

4 hours ago, AP123 said:

All I need to do now is update my emby from portainer. Again, someone did this for me so I have no idea how to do it. Any CLEAR STEP BY STEP instructions would be amazing. This is not meant to demean those trying to help, but giving half answers and assuming everyone here is a pro could lead to personal frustration for some.

Login to your Portainer WebUI and on the first screen, click on your local environment listed.

Then on the left hand side of the screen click on Containers.

From the list of containers, click on your Emby container.

On the next screen, click on the Recreate button. A pop-up will appear asking if you are sure. Click the toggle on the pop-up that says Re-pull image, then click the recreate button.

This will pull the latest image from docker hub based on the image tag being used for your container (usually latest), then recreate the container using the same configuration and settings.

1 hour ago, Lambtalk said:

Login to your Portainer WebUI and on the first screen, click on your local environment listed.

Then on the left hand side of the screen click on Containers.

From the list of containers, click on your Emby container.

On the next screen, click on the Recreate button. A pop-up will appear asking if you are sure. Click the toggle on the pop-up that says Re-pull image, then click the recreate button.

This will pull the latest image from docker hub based on the image tag being used for your container (usually latest), then recreate the container using the same configuration and settings.

Thank you. I was able to figure it out but I very much appreciate the reply.