Server Plugin: Transcoding Tests

[softworkz 4278]

Starting from today, you can find a new plugin in the catalog:

Transcoding Tests

 

The plugin allows to perform a wide range of tests using a defined set of source files. 
These tests will help to better understand user issues, compare the behavior on different systems, while it rules out many of those factors which are often hard to impossible to rule out. 

Another use case is benchmarking and/or comparing the effects of different settings on performance and quality.

Finally, this allows to recognize and analyze regressions and differences across versions more reliably. The results are stored outside the log folder and are not subject to regular cleanup. Each job execution is archived as a single file and those can be kept even when the actual output is cleared.

The initial version includes essential functionality. From here on, we will see what else might be needed or useful.

 

Updates

• Version 4.8.0.16
  • This version allows opening local output files or folders from Chrome (Windows) 
    (see end of second post)
  • The checkbox "Run this plan right now" is working now

Edited November 20, 2022 by softworkz

[softworkz 4278]

Instructions

Install the plugin from the catalog, then reboot Emby Server.

On the server dashboard, find the navigation menu entry "Transcoding Tests" somewhere among the last entries.

First of all, you need to 

Configure Settings

 

Please enter two different paths for test files and the generated output

The output can take quite an amount of storage, most of which is taken by the transcoded output files.

I would recommend the setting above, as it is sometimes useful to be able to inspect the results, but you can choose to "Keep output from most recent run" to save space.

Create Test Plan

A test plan defines a certain set of tests which can be executed as often as desired.

Click "Add Test Plan" to run the test plan wizard.

For starters, it makes the most sense to create a plan that is focused on a certain area, so choose "Start by Choosing Test Areas", which will show the following screen:

 

Choose an area of interest and click "Next".

Test File Selection

You will see a list of files which are applicable for the selected area(s).

Accept the default selections or make your own and click next

Detail Selections

In the following wizard pages, you will be presented a number of selections. Which ones you'll see depends on your previous selections (test files and areas). You will see only those selections that are applicably to at least a subset of the selections that are already made.

BE CAREFUL: All those selections are MULTIPLIERS, so you can quickly end up with a huge list of tests

Transcode Segment Selection

For initial experiments, I would recommend to choose either(!) "1 Minute from Start" or "3 Minutes from Start" - to make it less boring to watch the executions.

Wizard Finish

 

I would recommend to enter a descriptive name instead of the default test plan name.

The checkbox "Run this plan right now" is working in the latest plugin version now.

If you made it that far - great! You have created your first test plan.

 

Execute Test Plan

To execute your test plan, click the three-dot menu next to the test plan in the list and choose "Execute Test Plan".

A new wizard will open. You can choose to execute specific runs from the plans only, or simply click "Next".

Test File Download

In this step, the required test files will be downloaded. You can only wait until completion and click "Next".

Probing Test Files

'Now, the test files get probed quickly and you may choose whether to extract images from the output files. To minimize delay, choose "No image" or "Single Image".

Press "Finish" and the execution will start. 

It will look like this:

 

TIP
It's often interesting to open the "User Sessions" view in a separate tab or window to watch during execution.

 

Opening Output Files

The latest version of the plugin allows opening output files by clicking on them on the result pages.
Folders are shown in the file explorer, and files are opened directly in the registered application.

This works on Windows only with Chrome (probably Edge as well) and after installing the following Chrome extension:

https://chrome.google.com/webstore/detail/local-explorer-file-manag/eokekhgpaakbkfkmjjcbffibkencdfkl

 

 

 

Edited November 20, 2022 by softworkz

[neik 862]

@rbjtech, have you been able to play with this?
If so, how does it compare to your previous test results?

[SeekingWisdom 1]

i have installed the transcoding tests from the plugin library and it appears in my plugins.

Under manage emby server "Alerts" it says "Transcoding tests was installed"

The "Transcoding tests" is under advanced (last entry) in the dashboard.

When I go to the library and select transcoding tests under install it says "you currently have version 4.8.0.11 installed".

However, when you click on it nothing happens (i.e. the others in the menu you see a page open when you click on them but nothing happens  when you click on transcoding tests.

I have stopped and restarted the emby server a number of times (i.e stop/run)

Installed version of emby is 4.8.0.10-704080010

Not sure what to do.

Edited October 4, 2022 by SeekingWisdom

[softworkz 4278]

When you press F12 and look at the "Console" tab - are there any errors?

[Happy2Play 9140]

Tested this, there is no option to configure.

If you click Settings you get pop up

Then when you click Got it and navigate away and back to plugin you get a blank screen with this console error.

genericedit.js?v=4.8.0.10:1 
        
       Uncaught (in promise) TypeError: babelHelpers.toConsumableArray is not a function
    at EL (genericedit.js?v=4.8.0.10:1:25537)
    at INPUT (genericedit.js?v=4.8.0.10:1:26690)
    at genericedit.js?v=4.8.0.10:1:16190
    at EL (genericedit.js?v=4.8.0.10:1:25799)
    at DIV (genericedit.js?v=4.8.0.10:1:26223)
    at genericedit.js?v=4.8.0.10:1:16143
    at EL (genericedit.js?v=4.8.0.10:1:25799)
    at DIV (genericedit.js?v=4.8.0.10:1:26223)
    at genericedit.js?v=4.8.0.10:1:16070
    at EL (genericedit.js?v=4.8.0.10:1:25799)
EL @ genericedit.js?v=4.8.0.10:1
INPUT @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
EL @ genericedit.js?v=4.8.0.10:1
DIV @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
EL @ genericedit.js?v=4.8.0.10:1
DIV @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
EL @ genericedit.js?v=4.8.0.10:1
DIV @ genericedit.js?v=4.8.0.10:1
renderFilePicker @ genericedit.js?v=4.8.0.10:1
renderSingleItem @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
EL @ genericedit.js?v=4.8.0.10:1
DIV @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
EL @ genericedit.js?v=4.8.0.10:1
DIV @ genericedit.js?v=4.8.0.10:1
(anonymous) @ genericedit.js?v=4.8.0.10:1
Promise.then (async)
(anonymous) @ genericui.js?v=4.8.0.10:1
Promise.then (async)
(anonymous) @ genericui.js?v=4.8.0.10:1
showStageWizard @ genericui.js?v=4.8.0.10:1
showStage @ genericui.js?v=4.8.0.10:1
(anonymous) @ genericui.js?v=4.8.0.10:1
Promise.then (async)
View @ genericui.js?v=4.8.0.10:1
onBeforeChange @ viewmanager.js?v=4.8.0.10:1
(anonymous) @ viewmanager.js?v=4.8.0.10:1
Promise.then (async)
(anonymous) @ viewmanager.js?v=4.8.0.10:1
Promise.then (async)
ViewManager.loadView @ viewmanager.js?v=4.8.0.10:1
(anonymous) @ approuter.js?v=4.8.0.10:1
(anonymous) @ approuter.js?v=4.8.0.10:1
Promise.then (async)
loadContentUrl @ approuter.js?v=4.8.0.10:1
(anonymous) @ approuter.js?v=4.8.0.10:1
onInitComplete @ approuter.js?v=4.8.0.10:1
Promise.then (async)
initRoute @ approuter.js?v=4.8.0.10:1
handleRoute @ approuter.js?v=4.8.0.10:1
page.dispatch @ page.js?v=4.8.0.10:1
page.show @ page.js?v=4.8.0.10:1
(anonymous) @ approuter.js?v=4.8.0.10:1
show @ approuter.js?v=4.8.0.10:1
onItemAction @ navdrawercontent.js?v=4.8.0.10:1
executeAction @ shortcuts.js?v=4.8.0.10:1
onClick @ shortcuts.js?v=4.8.0.10:1
onClick @ emby-itemscontainer.js?v=4.8.0.10:1

 

 

[SeekingWisdom 1]

opened emby

went to dashboard

hit F12

clicked on (left and right multiple times) "transcoding tests"

above is screenshot of the expanded console section 

I do not get what the previous poster sees - nothing happens for me

I am using a chrome browser version 106.0.5249.91 (official build)(64 bit)

Im going to bed - will check back in morning unless you post again in a couple minutes

Edited October 4, 2022 by SeekingWisdom

[softworkz 4278]

Thanks. I was able to reproduce. I tried replacing with the original files but that didn't work out.

I hope @Luke can tell what's happening.

[softworkz 4278]

@SeekingWisdom @Happy2Play

Sorry for the trouble - this should be fixed now. 

Please restart Emby Server so it can find the updated plugin. Then again to get it installed.
(when it still wouldn't work, a browser cache refresh might be needed).

[SeekingWisdom 1]

i have version 12 installed

i reset my chrome cache (everything)

i restarted emby

when I click on transcoding tests here is a picture of the console (I now realize what I pasted yesterday was not what you wanted - sorry about that)

 

I cleared the console window before clicking transcoding tests so I believe what is shown above is everything

Just to prove I got the new file

What should I do now?

[softworkz 4278]

Can you expand the message by clicking the little arrow:

 

[SeekingWisdom 1]

[softworkz 4278]

Also, does the inputmanager.js get loaded when you click on the link:

[Happy2Play 9140]

Working here

Can't enable this check box but running test from after selecting fininsh now.

[SeekingWisdom 1]

I think that highlighted line showed up when I clicked the indicated link

Edited October 5, 2022 by SeekingWisdom

[softworkz 4278]

1 minute ago, Happy2Play said:

Working here

Can't enable this check box but running test from after selecting fininsh now.

Yes, that's not implemented.

[Happy2Play 9140]

Is @SeekingWisdomissue possibly a proxy issue looking at that ip:port request?

May have to test outside of localhost here.

[softworkz 4278]

1 minute ago, Happy2Play said:

Is @SeekingWisdomissue possibly a proxy issue looking at that ip:port request?

May have to test outside of localhost here.

Hm, but way can it load all others, just no inputmanager?

[softworkz 4278]

@SeekingWisdom Does inputmanager.js exist on the server file system?

In your case probably /volume1/@appstore/EmbyServer/system/dashboard-ui/modules/common

[softworkz 4278]

8 minutes ago, Happy2Play said:

Working here

Can't enable this check box but running test from after selecting fininsh now.

Great! Did you encounter any other issues or flaws?

[Happy2Play 9140]

Is there a case issue somewhere?   Both open the same file.

2 minutes ago, softworkz said:

Great! Did you encounter any other issues or flaws?

Not so far.

@SeekingWisdom Does the Source tab in console show the files?

[SeekingWisdom 1]

I think this is what you wanted to verify?

Edited October 5, 2022 by SeekingWisdom

[softworkz 4278]

2 minutes ago, Happy2Play said:

Is there a case issue somewhere? 

Damn yes - you got it!!!!

[softworkz 4278]

Just now, softworkz said:

3 minutes ago, Happy2Play said:

Is there a case issue somewhere? 

Damn yes - you got it!!!!

Good eye! @SeekingWisdom is on Linux..

[SeekingWisdom 1]

ok - notice in my picture that "inputmanager.js" does not match "inputManager.js"

Edited October 5, 2022 by SeekingWisdom