Jump to content

Linux commercial cutting and remuxing


Recommended Posts

afogleson
Posted

I do sort of have a couple questions... 

how many people are actually running emby on *nix and do commercial cutting/post processing of live tv recordings? 

and then the follow up to that... what are you using?

So i will run through my story/setup. 

I am running on *nix (mostly in a docker container - or 4 if you have seen my load balancing thread lol) 

I used to run a Windows box with MCE Buddy on it that then did all the post processing. 

then I moved that to a Virtual machine running in linux (in a docker container also.... I mean i have 96 cores I have not yet taxed the box at all) 

and i got tired of not having something that just runs on Linux (yes I have the cores but it would be nice to just run some command or open a gui app and do it all in linux. So of course, being an old software guy I wrote something (in java.... its portable after all) Right now i have it running pretty flawlessly as a command line only (granted i have not added the options like multiple comcutting and remuxing at once and such but so far it keeps up with my recordings. (I average probably 20 or 30 a day so  i am pretty much continuously commercial cutting or remuxing to HEVC) 

Now for some other questions... 

for the life of me i cannot get the post processing command to work. I point that at a bash script and i have stripped out everything except an echo that redirects to a log file... and i never see it come out. (i may have to set up another docker container that is standalone just to test this more) so for anyone post processing live tv on *nix using bash.... let me know. it has to be something silly/stupid but I have checked all the "obvious things" (yes it is executable, heck by ANYONE essentially chmod 777 on it. its obviously in a directory that emby can see.... etc etc. so im at a loss right now.) its ok since I run the command line program that has an endless loop of processing or checking every 15 minutes to see if something is ready to process and then it processes any that are, and then goes back to checking... so like i said it is keeping up, but with what amounts to (for the purposes of this) having almost unlimited cpu horsepower I could run up several programs at once and process 3 or 4 or 5 at a time without any real hit. (yeah i will get to that on the command line program) but id be interested in thoughts on what I may have missed on the post processor command 

and the final question, is anyone interested in a linux variant (really it will be java and likely free-ish  so  it could be windows, mac, etc, and maybe at some point it would be worth making some features payware, but basic stuff... probably always free. (ie... single threaded one recording at a time... it does not seem to make any sense to make people pay for that) It will eventually have a ui... im more concerned initially with getting all the basic features down in the actual processing. 

Posted

Running fine for me (bare metal).

Are you calling a non-emby version of ffmpeg? 

If so,  "'unset LD_LIBRARY_PATH" before calling ffmpeg (or any command that uses it, ie comskip) or preface the ffmpeg command like this "LD_LIBRARY_PATH=''' ffmpeg ..."

Also, you may want to look at the plugin ScripterX to run your java stuff in parallel. 

afogleson
Posted
1 hour ago, TMCsw said:

Running fine for me (bare metal).

Are you calling a non-emby version of ffmpeg? 

If so,  "'unset LD_LIBRARY_PATH" before calling ffmpeg (or any command that uses it, ie comskip) or preface the ffmpeg command like this "LD_LIBRARY_PATH=''' ffmpeg ..."

Also, you may want to look at the plugin ScripterX to run your java stuff in parallel. 

Im not calling FFMpeg at all. i call a bash script, that then calls my java program to do the commercial cutting and remuxing to HEVC. there is a little more to it but it is pretty simple. 

#!/bin/bash

log=/media/qnap/tmp1/processLiveTV.log
echo "" >> $log
echo "" >> $log
echo "" >> $log

path="$1"
echo "received $1 from emby as the ${path}" >> $log
echo "Calling MCEBuddy with path ${path}" >> $log

#jabba use amazon-corretto@1.17.0-0.35.1
/media/data/comSkipCut/MCEBuddy-Linux-1.0-SNAPSHOT/bin/MCEBuddy-Linux --mainPath='$path' --ageLimit=0 >> "${log}
 

if i got some kind of java can not be found or such... then it would at least make sense but this looks like it does not even get to the bash script. and yes it is executable and everything. in fact it has more permissions than i would ever consider (777 lol) 

i do not see any of that stuff in the log file. when i run this from a normal command prompt, it spits it all out fine and runs the MCEBuddy (really need a different name to avoid copyright i suppose. but thats for later) 

afogleson
Posted
Just now, afogleson said:

Im not calling FFMpeg at all. i call a bash script, that then calls my java program to do the commercial cutting and remuxing to HEVC. there is a little more to it but it is pretty simple. 

 

by the way i think i said in the main post... but i am getting around this right now in the actual java program in that I can run it and if you call it with a different parameter it runs forever on a loop and will check for new files to process every X seconds (another parameter) so basically it just comcuts and remuxes pretty much all day and night lol. 

Posted

Well then, let’s put A wrapper inside your BASH script:

#!/bin/bash -x ### <--

Stuff(){
	log=/media/qnap/tmp1/processLiveTV.log
	echo "" ###>> $log
	echo "" ###>> $log
	echo "" ###>> $log
	path="$1"
	echo "received $1 from emby as the ${path}" ###>> $log
	echo "Calling MCEBuddy with path ${path}" ###>> $log
	#jabba use amazon-corretto@1.17.0-0.35.1
	/media/data/comSkipCut/MCEBuddy-Linux-1.0-SNAPSHOT/bin/MCEBuddy-Linux --mainPath='$path' --ageLimit=0  ###>> "${log}
} # Stuff

Stuff "$@" 2>&1 >/tmp/stuff.txt
echo '*****  ENVIRONMENT *****' >>/tmp/stuff.txt
set >>/tmp/stuff.txt
echo '*****  done  *****' >>/tmp/stuff.txt

What's in /tmp/stuff.txt ?

Posted

Very first step - the Emby official container doesn't have bash, I think LSIO container might. So depending on which container you use you'll have to adjust between bash and sh.

 

afogleson
Posted
6 hours ago, Q-Droid said:

Very first step - the Emby official container doesn't have bash, I think LSIO container might. So depending on which container you use you'll have to adjust between bash and sh.

 

Well it must have gotten bash "recently" then. the container i am using has bash....

its easy enough to prove:

docker exec -it YourEmbyContainerName  /bin/bash     

or if you want to be sure.... 

docker exec -it YourEmbyContainerName sh

and once in "which bash"   i think it may come up as /usr/bin/bash 

I double checked all of this in my container (see attached pic) 

for reference my emby container is "lscr.io/linuxserver/emby:beta-version-4.9.1.2"

I have a preference for those containers... but I dont think it matters much. worst case scenario i would just make my own cotainer. but i really dont feel like doing that every time there is a new beta release :)  

WHAT is far more likely is that it is puking on the need for java..... (just occurred to me) so i might check that out... i think i got more confused by never seeing the echos out to my log files.... seeing those but not getting processed would have clued me in a lot faster :) 

 

bashInEmbyContainer.png

afogleson
Posted
15 hours ago, TMCsw said:

Well then, let’s put A wrapper inside your BASH script:

#!/bin/bash -x ### <--

Stuff(){
	log=/media/qnap/tmp1/processLiveTV.log
	echo "" ###>> $log
	echo "" ###>> $log
	echo "" ###>> $log
	path="$1"
	echo "received $1 from emby as the ${path}" ###>> $log
	echo "Calling MCEBuddy with path ${path}" ###>> $log
	#jabba use amazon-corretto@1.17.0-0.35.1
	/media/data/comSkipCut/MCEBuddy-Linux-1.0-SNAPSHOT/bin/MCEBuddy-Linux --mainPath='$path' --ageLimit=0  ###>> "${log}
} # Stuff

Stuff "$@" 2>&1 >/tmp/stuff.txt
echo '*****  ENVIRONMENT *****' >>/tmp/stuff.txt
set >>/tmp/stuff.txt
echo '*****  done  *****' >>/tmp/stuff.txt

What's in /tmp/stuff.txt ?

actually i am going to go on a hunch first. the container has everything i need except one thing... java.... I just went into the container and installed java... so now I'll see if the shell works. (you can even see where i commented out "jabba use...." to tell it to use a specific  java. but stupid me never said... do we have a java at all? so i am betting that is the problem... if so... im not likely to fix that. Id have to start building my own containers and that seems more problems than just having the thing run on bare metal and constantly process/check if there is anything to process. (thats working fine really)  I COULD do this all outside the container and map that back into the container i suppose and such... seems a lot of pain for little gain and not "portable" to other emby users easily 

 

Posted

You're using the LSIO container which matches what I stated in my post.

 

 

afogleson
Posted (edited)
51 minutes ago, Q-Droid said:

You're using the LSIO container which matches what I stated in my post.

 

 

Im confused what you stated in your post is "The emby container does NOT have bash" 

however in that container if you docker exec into a shell there... you can easily SEE there IS a bash executable. I dont mind being wrong, but maybe double check in a container and tell me nope I dont see it... I even included a JPG where i execed into the container and did a which bash and it showed a bash. which means... your statement there was no bash was incrorrect.  and that would lead me down a very deep hole that is not the solution to the issue. NOT having a java is a bigger issue but i can fix that

Edited by afogleson
Posted

Read it again - the "Emby official container". The image maintained and released by the Emby team. The LSIO container image, and others, are not maintained by Emby so if you run into problems in Docker the devs/support are likely to ask if you're using the official image and/or whether the problem can be reproduced in the official image.

 

 

afogleson
Posted
30 minutes ago, Q-Droid said:

Read it again - the "Emby official container". The image maintained and released by the Emby team. The LSIO container image, and others, are not maintained by Emby so if you run into problems in Docker the devs/support are likely to ask if you're using the official image and/or whether the problem can be reproduced in the official image.

 

 

ok.... I read it again... it says literally "You're using the LSIO container which matches what I stated in my post." what part of the LSIO container matches "the container does not have java?" I understand completely that some container maintained by the emby team might not have bash... but that lead me down a path of... sure it does I MADE sure it has bash. and now thanks to some "minor" (ok so it took about an hour but worth while) my emby container has java... and now with a new container (i need to test this all not on my production container of course) with it all set up I can test easily. I'll probably do a bit more mapping to make it a lot easier on me (i keep forgetting which directory things are stored in... much better to keep it all in the same dir (for me /media/data/emby) where all the mapped local directories are. 

afogleson
Posted
Just now, afogleson said:

ok.... I read it again... 

reproduction on bare metal is easy.... and should be the standard we would be judged by, not by some container. I HAPPEN to be using a container... luckily not one with alpine linux :) Alpine is small, and likely the reason for using it... more that than the security aspects of alpine

afogleson
Posted

UGH. so  i have it all working and decided... it is not worthwhile in an emby container. :) its not "overly" complex or such and i suppose i could script it all up to manage it every time a new emby release comes out. but essentially i had to 

  1. first install a JDK. (since i am using java) 
  2. perform an apt update (since my container being a linuxserver release runs on ubuntu)
  3. install comskip (easy enough with apt install comskip)
  4. copy the comcut script over to usr/local/bin
  5. install HandbrakeCli to the container
  6. do some mappings to make some things easier (and a lot of these i could do "permamently" but the harder stuff is getting the 2 things installed in the container every upgrade or heaven forbid every time an image gets "deleted"

then it all works... considering that it is much easier for me to just have my commercial cutting util monitor the recordings directory and let it just process as it sees them completed (no change in the file in 2 minuites is what i do) 



 

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...