My home media server project got a little out of hand…

[skidmarks 199]

I’ve been building a pretty ambitious home media/server monitoring and recovery system around Emby, Plex, Cloudflare, and the rest of my home infrastructure.

The goal is simple: if something important goes down while I’m away, the system should be able to recognize what actually failed, avoid making the problem worse, recover the right thing automatically when it’s safe to do so, and leave a clear record of what happened.

Cloudflare is also a significant part of the design. I know there has been a lot of discussion lately about whether Cloudflare is still a practical option for home media environments. Without getting into the nuts and bolts of my implementation, I can say that I’m actively using it as part of a secure remote-access architecture and have been deliberately testing not only normal operation, but failure and recovery behavior as well.

It’s grown into a lot more than a dashboard. I now have health monitoring, service checks, dependency awareness, controlled automatic recovery, startup protection, failure history, and safeguards to keep one bad recovery attempt from taking down everything else.

One of the more interesting recent tests started with Plex acting strangely after a reboot. Plex eventually recovered on its own, but the recovery system exposed a completely different hidden problem: one stuck recovery attempt could block recovery for another service. That led to redesigning the recovery layer so each service is handled independently, with its own limits and safety rules.

I’m also documenting the whole thing as I go so it can be rebuilt from scratch years from now instead of becoming one of those “only the guy who built it understands it” projects.

It’s still very much unfinished, but it’s now at the point where I can deliberately break parts of the system, watch it determine what actually failed, recover what it should, refuse what it shouldn’t, and prove afterward what happened.  I am also in the middle of building the API that will work hand and hand with my control panel to visually inspect and repair IF the automation should fail its checks and balances.

For a home media setup, it has gotten a little ridiculous.

And I’m having way too much fun with it.  

There are a few control panel elements, not the entire panel, that i am willing to share for now.  I now completely understand what the developers go thru to deliver an application with little to no bugs every update that is released.  Months and more months have been put into this and probably many more before everything i Invision is implemented.   

 

See guys i didn't go away, i took a break and have come back with a little project i've been working on.

Edited 8 hours ago by skidmarks

[ebr 16650]

Cool.  Hopefully, it doesn't end up with "Sorry, Dave..."