LLM Integration-based Enhancements

[adminExitium 355]

I am not sure if there are any existing plans for enhancements based on LLM Integrations, but this player for Jellyfin seems to have a very nice set of actually useful enhancements: https://github.com/AyaanZaveri/finetic#ai-assistant

Copying the relevant text from it's README:

## Ask questions like:
"Play Inception"
"Go to Breaking Bad"
"Skip to the action scene"
"Explain what's happening right now"
"What's this movie about?"

## Available AI Tools
The AI assistant has access to a comprehensive set of tools for media library interaction:

###  Content Discovery & Search

searchMedia - Search for movies, TV shows, or episodes by name or keyword
getPeople - Search for people (directors, actors) related to media content
getGenres - Get list of all genres available in the library

###  Library Browse & Management

getMovies - Get a list of recent movies from the library
getTVShows - Get a list of recent TV shows from the library
continueWatching - Fetch list of media items currently being watched/continued
getWatchlist - Get user's watchlist or favorites (popular/highly-rated content)
getMediaDetails - Get detailed information about a specific movie or TV show
getSeasons - Get seasons for a TV show
getEpisodes - Get episodes for a TV show season

###  Playback & Navigation

navigateToMedia - Navigate to a specific movie, TV show, or episode page
playMedia - Play a specific movie, TV show, or episode directly in the media player

###  Subtitle Analysis

skipToSubtitleContent - Intelligently analyze subtitles and find the best timestamp based on user descriptions (doesn't require exact text matches)
explainScene - Analyze subtitles around current timestamp to explain what's happening in the scene
analyzeMedia - Analyze the entire movie/episode using subtitles to answer questions about plot, characters, themes, etc.

###  App Controls

themeToggle - Toggle or set the application theme between light, dark, or system mode

### Smart Subtitle Features

When media is actively playing, the AI can:

Scene Navigation: "Skip to the part where they talk about love" or "Take me to the action sequence"
Context Explanation: "What's happening in this scene?" or "Who is talking right now?"
Content Analysis: "What is this movie about?" or "Summarize this episode"
Uses semantic understanding to find scenes based on descriptions, not just exact text matches

### Search Intelligence

The AI automatically expands common abbreviations and handles vague descriptions:

"b99" → "Brooklyn Nine-Nine"
"the movie with the blue people" → "Avatar"
"the show about meth" → "Breaking Bad"
"the wizard movie" → "Harry Potter"

### Example Queries

#### Content Discovery:

"Show me my continue watching list"
"What genres are available?"
"Find movies with Tom Hanks"
"Show me recent sci-fi movies"

#### Navigation & Playback:

"Go to Breaking Bad"
"Play Inception"
"Show me seasons of The Office"

#### Scene Navigation (during playback):

"Skip to when they arrive at the destination"
"Take me to the emotional conversation scene"
"Jump to the plot twist"

#### Scene Analysis (during playback):

"What's happening right now?"
"Explain this scene"
"What did I miss?"
"Who are the main characters in this movie?"

#### App Control:

"Toggle the theme"

 

[brothom 177]

The player you're providing seems to be external to Jellyfin and therefore not "official" and probably not endorsed either because this project seems to load an LLM locally which probably takes a bunch of resources, maybe even more than the mediaserver itself.

It's all nice and well, but the number of people waiting for AI "assistants" is quickly dropping because of how unreliable LLMs are.
If this or a similar LLM feature is integration into Emby, it should at least be optional because personally, I don't want to be running an LLM anywhere on my network.

[adminExitium 355]

Emby doesn't really need to provide or include an LLM for this. They can just use the standard APIs and allow the users to configure the Endpoint, API Key etc. to allow both self-hosted & commercial providers, depending on what they require and have the resources for.

My primary goal for this is to get all the framework etc. added, because as has been seen before with stuff like the IntroSkip plugin, it's very difficult to introduce UI elements uniformly across clients without core support. The actual choice of an LLM is a smaller concern and can be left up to the user, if they even want it.

And I don't think reliability matters that much here, it's just a helper for entertainment stuff. It doesn't really matter a lot if it's wrong once in a while.

[Luke 42077]

Hi, you may want to check this out: 

 

[adminExitium 355]

Yeah, I am aware of that but I am asking for some kind of LLM client integration in the apps themselves so the users don't need to switch to different wrappers just for those functionality.

It's generally a bad idea to require users to switch between apps while already streaming, it just breaks the immersion.

[thoth23 5]

@adminExitiumI generally agree with you. 

My primary goal in writing Emby.MCP (thanks for highlighting @Luke) was to get some working knowledge of MCP and to learn Python with the aid of Copilot completion while doing so. If it wasn't for Python I would have coded it as a server plugin and learned C# while doing so. Indeed I have contemplated a port, but I can't really justify the time atm. Having the framework you describe in the core would make this task much easier and obviate the need for Claude Desktop or whatever.

That said, I mainly access my Emby server via the Symfonium app on my Samsung phone. So for me, short of LLM integration in Symfonium, I'd want to use the Gemini chatbox to remotely access MCP tools in that Emby plugin. 

There is of course the question of cost. It takes a fair beafy machine to run a decent LLM locally (which is a fairly niche sport in itself), and API calls to the hosted majors can rack up lumpy & unexpectedly costly bills. At least with their provided clients you pay a fixed monthly for a general purpose tool and can choose a lesser model with higher usage limits for simple Emby MCP requests.

Edited September 2, 2025 by thoth23