Add album_artist, year, rating, file_path column to data layer across all providers

[rendyhd]

Capture the original album artist before _select_best_artist() overwrites it with the track-level artist. This preserves the album-level artist (e.g. for compilation albums) in a new album_artist column in the score table, propagated through all media server modules (Jellyfin, Emby, Navidrome, Lyrion), the analysis pipeline, similarity search, song alchemy, path manager, and API responses.

Also fix a pre-existing bug in Emby's standalone track path where _select_best_artist() return tuple was not unpacked.

Also adds:

• Year

• Track Rating (for Navidrome and Lyrion)

• File Path (note on Navidrome)*

• Updated chat (instant playlist) with info in year and rating

---

*Navidrome by defaults reports an "Internal path", this is artist/album/track but not your real file path. If you want to match files across services, it's advised to turn on Report Real Path:

Log into the Navidrome web interface
Go to Players in the right sidebar
Click on the AudioMuse player entry (it appears after AudioMuse first connects)
Toggle "Report Real Path" to enabled

You can also change this as a default setting by setting ND_DEFAULTREPORTREALPATH=true

[NeptuneHub]

Start collecting additional information could be useful as a basis of future functionality. Did you have something in mind?
I'm still thinking that we miss a similar album functionality but I still don't have idea on how to implement it without being repetitive.

About mediaserver to test, just one correction, you need to integrate only Jellyfin, Emby, Navidrome, Lyrion.

IMPORTANT: Navidrome is the configuration for all the Music Server that support the Open Subsonic standard. Ok maybe cover all is not always possible (someone have some small difference) but I usually tested this kind of change on Navidrome AND Lightweight Music Server

For MPD I tried add integration but I was never able to complete basically because I wasn't able to directly download song from MPD for the analysis (maybe if you have any idea integrate MPD or other Music Server in future could be nice). So you can skip it because is incomplete.

[rendyhd]

My goal is to make a Curated Playlist Builder, this would be great metadata to have in there.
The local filepath is to expand in the future and i figured - while I'm expanding the meta data why not include it.

I was thinking, when you have local filepath you can:

• Run analysis directly on local files and generate .m3u based on the existing features
• Open the doors to integrate with providers that don't allow downloads (e.g. plex)
• Connect to multiple providers, using filepath as key identifier

Regarding Album, there's two paths I think:

• Expand album
• Find similar album

The expand album is very much in line for what I want with the second part of my Playlist system - the Expand playlist. You basically expand from N tracks, taking not just the last song as data-point but the group. This would be the same for playlist and album expansion.
A potential endpoint could be "keep playing" instead of track radio. Taking the last N tracks, in use cases where people queue a couple to fine-tune a mood.

Similar album is a different approach, matches groups with groups. I wonder how much that would get used. It'd say it's fun from an exploration perspective if you have a large library, but not the best way to stay in a certain mood.

[NeptuneHub]

Local file path need to be think carefully:

• on one side it can speed up the analysis and open the door to other functionaltiy;
• on the other side you need to think about then how then you use the analysis against the single music server

On this last point it could be nice the idea of a .m3u output for music server that support it. So you don't have to use API at all (and you don't have to match a file path with the ID of the music server). It maybe could be impelmented as a "local-path" music server. The result should be you analyze locally, your output is local file, and then you can import where you want.

[rendyhd]

Successfully tested. I'm going to look into the other metadata fields before pushing the PR.

• Added album_artist in the score table, type text, nullable.
• Ran all providers and 4 AudioMuse instances, after API fix now all 4 have 100% album_artist filled.
• I've also added a folder called "testing". This has 2 docker compose stacks (one for providers, one for AudioMuse instances), a common .env, and a guide how to set it up.

[rendyhd]

I've added and tested the following fields:

• Year (works on all 4)

• File Path (works on all 4, however, Navidrome needs a setting changed)*

• Track Rating (only added for Navidrome and Lyrion, as Jellyfin and Emby don't support it)

• Album Artist was already working on all 4

*Navidrome by defaults reports an "Internal path", this is artist/album/track but not your real file path. If you want to match files across services, it's advised to turn on Report Real Path:

1. Log into the Navidrome web interface
2. Go to Players in the right sidebar
3. Click on the AudioMuse player entry (it appears after AudioMuse first connects)
4. Toggle "Report Real Path" to enabled

You can also change this as a default setting by setting ND_DEFAULTREPORTREALPATH=true

[NeptuneHub]

Thanks for you effort, to recap you added this 4 field all in the score table:

• Year (works on all 4) => numeric?
• File Path (works on all 4, however, Navidrome needs a setting changed) => String?
• Track Rating (only added for Navidrome and Lyrion, as Jellyfin and Emby don't support it) => Numeric?
• Album Artist was already working on all 4 => String?

And for now you use only album artist as as fallback when album is not present in the difference "song search" form? All the other field are only for future implementation right?

In your test, on all the mediaserver, did you try:

• legacy database, already with multiple album in it => it have a migration functionality to add the missing column? you populate them at the first analysis also for the already analyzed song WITHOUT havign to reanalyze with the ML model? (i mean just populate the missing field)
• New deployment, first run => it just create the new table and start populating?
• For music serverthat doesn't support one or more of this field, which is the fallback? maybe some default value that could avoid to future implementation to broke on unsupported music server? (like "uknown" for the string and the number 0 for the number?)
• Which edge case did you tested? => please share the list of all the test that you did automated or manually, and the result.

I'll do my test offcourse, but editing the mediaserver part we need extra attention.

[arsaboo]

@rendyhd This is great. Having the ids along with the ratings is huge....

[rendyhd]

@NeptuneHub

• Yes, I've added it the 4 to score. I considered a separate table, but don't expect any performance issues.
  Year = INT
  File Path = TEXT
  Rating = INT
  Album Artist = TEXT

• album artist, rating, and year are included in the system prompt for the instant playlist

In your test, on all the mediaserver, did you try:
legacy database, already with multiple album in it => it have a migration functionality to add the missing column? you populate them at the first analysis also for the already analyzed song WITHOUT havign to reanalyze with the ML model? (i mean just populate the missing field) - Yes, same as album name
New deployment, first run => it just create the new table and start populating? Yes
For music serverthat doesn't support one or more of this field, which is the fallback? maybe some default value that could avoid to future implementation to broke on unsupported music server? (like "uknown" for the string and the number 0 for the number?) In the table it's just NULL, it doesn't get used elsewhere
Which edge case did you tested? => please share the list of all the test that you did automated or manually, and the result.
Tested load will all servers, with and without ratings.

---

• I just added additional logic for when someone has a full date instead of a year field in the year tag. This hasn't been an issue yet, because the providers have always given a YYYY for me. I fixed it for future local path implementation and knowing sometimes the mp3tag can be filled with a full date depending on where you load the metadata from.

• I just changed the rating from 0-100 to 0-5 since most users will be more used to the 5 star rating.

During my test just now I did find out that album name isn't populated for Navidrome and Lyrion - I haven't seen anything that could cause that regression. I can test tomorrow with the current release

[rendyhd]

Check after last commit, album name is complete now too (rating is just a single song, so 0.3% is correct):
Complete Completion Overview:

Field	Jellyfin	Emby	Navidrome	Lyrion
Total Rows	308	308	308	308
title	100.0%	100.0%	100.0%	100.0%
author	100.0%	100.0%	100.0%	100.0%
album	100.0%	100.0%	100.0%	100.0%
album_artist	100.0%	100.0%	100.0%	100.0%
tempo	100.0%	100.0%	100.0%	100.0%
key	100.0%	100.0%	100.0%	100.0%
scale	100.0%	100.0%	100.0%	100.0%
mood_vector	100.0%	100.0%	100.0%	100.0%
energy	100.0%	100.0%	100.0%	100.0%
year	100.0%	100.0%	100.0%	100.0%
rating	0.0%	0.0%	0.3%	0.3%
file_path	100.0%	100.0%	100.0%	100.0%

[rendyhd]

Finished AB testing the changes to instant playlist, I'll test the navidrome change next if it has breaking impact on file path.
AB_COMPARISON_REPORT.md
I'll the ai instant playlist branch into this one when done: https://github.com/rendyhd/AudioMuse-AI/tree/feature/ai-instant-playlist-upgrade

Not in the AB report is ratings test, I did that in a separate change, added more strict guidelines around that

[rendyhd]

For Navidrome the change was suggested from the developer of Navidrome himself so I assume he liked. He asked to clearly show the audiomuse name, with the version, because then they have some internal statistics.

I looked into the change: The "c" parameter is the Subsonic API client identifier, and Navidrome uses it to create and track players, if we make this change then the player name will change with every version bump (e.g., AudioMuse-AI/v0.9.1)
In order to get the Real Path that we want to match with local file, you need to go into the UI in Players, click AudioMuse and enable Report Real Path.

This change would mean that this Path metadata would break every version bump. Can we check with the Navidrome developer for advise on this?

[NeptuneHub]

Let's keep it easy, if you think that keeping only the application name without the version is better. for me is ok.

The only change at this point is to correct name that is AudioMuse-AI, you wrote only AudioMuse (that is the name of a differente application totally unrelated from us).

Thanks.

[rendyhd]

I've merged the current version, however, there are 3 more potential upgrades I identified in testing. I could look at that later this week.

1. CLAP genre bleed — Text search returns sonically similar but genre-inappropriate tracks (e.g. metal tracks in a jazz playlist) because CLAP matches audio texture, not genre labels. Needs post-filter by genre or a hybrid CLAP+metadata approach.
2. Wallpaper artists — A small set of well-connected artists (e.g. Linkin Park appeared in 13/17 Sonnet playlists) dominate via artist_similarity and song_similarity. Needs cross-prompt artist frequency dampening or per-request artist exclusion after the first occurrence.
3. Near-duplicate tracks — Live, Demo, Remaster, and Deluxe Edition variants of the same song appear as separate entries (e.g. "Numb" and "Numb (Live)"). Needs title normalization/dedup that strips common suffixes before the uniqueness check.

Also, it might be good to do a performance scorecard on a selection of models. I've done extensive testing using Haiku, Gemini Flash, and Sonnet (total of 1200+ requests to openrouter) while optimizing and improving performance. Here I saw a clear difference in using a model like Sonnet vs Haiku/Flash. If we want I can create a test script that runs every model 3x and creates a fair expectation for users. This would be done last though.

What I've merged:
Merge branch 'feature/ai-instant-playlist-upgrade' into feature/album-artist-support

Overhaul the AI instant playlist pipeline with a unified prompt architecture,
stricter filtering, smarter tool selection, playlist ordering, and 2,770 lines
of new unit tests across 4 test files.

=== Prompt & Tool Layer (ai_mcp_client.py) ===

• Add _build_system_prompt() as single canonical prompt for all 4 AI providers
  (Gemini, OpenAI, Mistral, Ollama), replacing per-provider prompt duplication
• Inject real library context (genre/mood/year/rating stats) into system prompt
  via get_library_context() so the AI knows what's actually in the library
• Add energy normalization: AI sees 0-1 range, converted to raw 0.01-0.15
  in execute_mcp_tool using config.ENERGY_MIN/MAX
• Add prioritized tool decision tree (song_similarity > album/decade >
  text_search > artist_similarity > song_alchemy > ai_brainstorm > search_database)
• Correct tool descriptions (artist_similarity returns artist's own songs + similar)
• Expand search_database parameters: scale, year_min/year_max, min_rating, album
• Add system prompt rules 11-12: rating is a hard filter, all user-specified
  filters must appear in every search_database call

=== Backend Tools (tasks/mcp_server.py) ===

• Add get_library_context() with module-level cache for library stats
  (total songs, unique artists, top genres, year range, rating coverage, scales)
• search_database: regex genre matching with (^|,)\s*genre: pattern to prevent
  substring false positives (e.g. "rock" no longer matches "indie rock")
• search_database: relevance-scored ranking by genre confidence sum instead of
  RANDOM() when genre filter is active
• search_database: genre confidence threshold (0.55) to exclude weak matches
• search_database: new filters for scale, year_min/year_max, min_rating, album
• text_search (CLAP): add genre keyword post-filter that checks mood_vector
  overlap, removing off-genre CLAP results (with 40% safety floor)
• text_search: fix column name energy_normalized -> energy
• song_alchemy: add genre overlap post-filter comparing seed track genres
  to result genres, keeping only songs with relevant genre overlap (40% floor)
• ai_brainstorm: replace per-song SQL queries with batched lookup; strict
  2-stage matching (exact case-insensitive then normalized fuzzy) requiring
  BOTH title AND artist to match
• Add album column to all SELECT statements and result dicts

=== Agentic Loop (app_chat.py) ===

• Switch from top-level config imports to runtime config.X module attribute
  access so DB-applied settings (via apply_settings_to_config) take effect
• Add pre-execution validation: reject song_similarity with empty title/artist,
  reject search_database with no filters
• Add rich iteration feedback: artist diversity stats, genres covered per round
• Enforce artist diversity cap via MAX_SONGS_PER_ARTIST_PLAYLIST (default 5)
  with overflow backfill when pool is too small
• Post-collection rating enforcement: track detected min_rating from tool calls
  and filter all collected songs against the database to remove any that leaked
• Cap agentic loop to 2 iterations when rating filter is active to prevent
  the AI from broadening searches to unrelated genres
• Add iteration context instructing AI to never drop original filters
• Integrate playlist_ordering for smooth BPM/energy/key transitions

=== New Files ===

• tasks/playlist_ordering.py (189 lines): Greedy nearest-neighbor track
  ordering using composite distance metric (tempo, energy, Circle of Fifths
  key distance) for smooth playlist transitions
• tests/conftest.py (115 lines): Shared test fixtures with importlib bypass
  to avoid tasks/init.py -> pydub -> audioop import chain
• tests/unit/test_mcp_server.py (1,322 lines): MCP server function tests
• tests/unit/test_ai_mcp_client.py (1,016 lines): AI client and prompt tests
• tests/unit/test_app_chat.py (302 lines): Chat endpoint tests
• tests/unit/test_playlist_ordering.py (130 lines): Ordering algorithm tests

=== Config ===

• MAX_SONGS_PER_ARTIST_PLAYLIST (default 5): artist diversity cap
• PLAYLIST_ENERGY_ARC (default false): energy arc ordering option
• .gitignore: exclude testing_suite/ directory

=== Fix ===

• Update Navidrome client identifier from "AudioMuse" to "AudioMuse-AI" for
  consistency across server versions

[NeptuneHub]

Look there is an unit test that still fail pointing to your some your local directory. Please have a look and fix.

I suggest to postpone any additional feature to future PR.

In the weekend, if I have time, I could make a release to add the Authentication layer to AudioMuse-AI (user+password for the human user, api token for the plugin), if meanwhile this PR is ready and bug free it could be added to the release.

Thanks.

[NeptuneHub]

So I did test against an already exist AudioMuse-AI deployment (on INTEL CPU) that work on top of jellyfin where:

• I run a new analysis => I didn't noticed any bug, new column added and populated;
• searched "2026 song"
  • it did a search from year 1 to year 2026 => I pushed a fix for this
  • it added some rating 3 start filter when not asked => I pushed a fix for this
  • it added some genre jazz random filter => I pushed a fix for this
  • the counting structurer tends to give less than the 100 songs requested => I pushed a fix that try to return 200 songs (not 100) to each iteration. Also here I count the song AFTER the artist filter, to have a real number

All the pushed fix are already against this PR branch, so you can directly pull and start your test.

For the test I used llama3.1:8b because AudioMuse-AI is selfhostable first, so it must work also on model that can run on let's say 6GB VRAM.

I finish my run of test. When you run yours please share here the confirmation that they are completed and no bug is there to have the PR merged.

Because here there is change in the mediaserver files, it's important that you run the test against all the mediaserver to avoid regression.

Thanks.

[rendyhd]

I've done more optimization for ollama/small models.
The models that came out best, in order:

1. qwen3.5:9b (largest tested)
2. qwen3.5:4b
3. gemma3:4b
4. ministral-3:3b (this one is the fastest)

[NeptuneHub]

I did some other test and now the intelligence of the AI seems working good.

I also did a test with this function to check for hidden regression:

• Analysis
• Clustering
• Playlist from Similar Song
• Artist Similarity
• Song Path
• Song Alchemy
• Text Search (CLAP)
• Music Map

And I didn't find any regression. So I'm happy to say that this is ready to be merged. Please after the next release be on the windows in case users raise any issue related to this (I don't think I will release for this week, already too many release :) )

I think you are very strong with this AI related functionality and will be very valuable if in your next PR you would like to keep working on this topic.