Wrong sort order on Photo folders

[cnwinger 7]

Just trying to narrow down the actual issue - I copied my photos from one year's folder to a new folder, deleted all EXIF data from every photo, and then re-named the photos to YYYY-MM-DD-#### with a sequential 4 digit counter.  The resulting sort order when sorting by Folder or by Title was even more random than before.  Something is very wrong here.

[cnwinger 7]

Have you checked the sort title in the metadata editor for these? Is it blank for them?

Has this been added to a bug list to be fixed?  Is there some way for me to determine that myself?

[Luke 37028]

Just trying to narrow down the actual issue - I copied my photos from one year's folder to a new folder, deleted all EXIF data from every photo, and then re-named the photos to YYYY-MM-DD-#### with a sequential 4 digit counter.  The resulting sort order when sorting by Folder or by Title was even more random than before.  Something is very wrong here.

 

Can you give some specific examples? We don't extract the date out of the file name so it will be treated as plain alpha-numeric text.

[cnwinger 7]

Can you give some specific examples? We don't extract the date out of the file name so it will be treated as plain alpha-numeric text.

 

This is what's happening:

 

 

Emby is the only program that fails to sort these filenames correctly. To troubleshoot, what I did next was remove the EXIF data from every photo in the folder, to see if something in the EXIF was causing the issue in Emby.  Same problem.  I then re-named the files as I described (YYYY-MM-DD-#### with a sequential 4 digit counter) to see if different file lengths, underscores, etc. were causing the issue.  Same problem, but it appeared to be even more random then.

 

I'd be happy to provide a log file or whatever else is needed to assist with your investigation, but at this point the app is unusable for me.  I will likely be forced to switch over to Kodi or MrMC on my AppleTV. I'd love to switch back when this is fixed.

Edited December 28, 2019 by cnwinger

[Happy2Play 8252]

Unless there is something specific going on here All my test images are sorted sequentially.
 
Library.db MediaItems table, name column and auto generated sortname column
 


Web Client

 

Edited December 29, 2019 by Happy2Play

[cnwinger 7]

Seeing as I'm not the only user that reported this behavior, is there some additional information or logs that we could provide to you?  Is there an easy way for me to extract the rows of my own Library.db MediaItems table?

 

I did notice that it only seems to happen in folders with a large # of photos named in this manner (500-1000+ photos).  My folders that had 100 or less photos were sorted correctly.

Edited December 29, 2019 by cnwinger

[cnwinger 7]

Ok, I took a look at the Library.db MediaItems table in SQLite browser.  For a folder sorting correctly,:



A folder not sorting correctly:



Formatted to show the values:



So something is causing specific files to not have the SortName generated the same way with the leading zeros.  All those rows with the incorrect SortName were photos taken by a Samsung:

{"CameraMake":"samsung","CameraModel":"SM-G920V","Software":"G920VVRS4DRB1","ExposureTime":0.0031645569620253164,"FocalLength":4.3,"Orientation":"TopLeft","Aperture":1.85,"IsoSpeedRating":40,"RemoteTrailers":[]}

The photos working correctly were all taken by an iPhone. That was the only difference that stood out for those rows, but I could provide a dump if needed.

Edited December 29, 2019 by cnwinger

[lunadmin 0]

Since Update of the actual version, my photo folder also not correct sorted anmyore. In folder name and title name the same issue.

But its simple named, i have just in this format

2014-01-05 text1

2005-05-01 text3

2019-01-02 text2

2015-02 text4

2017-01 text4

2012 text5

 

A few were listed right, the rest is in between. So i found out, instead of list the folder name like explorer do and before update, he do now 

first the xxx then xxx-xx then the xxx-xx-xx Folder names, every group sorted by its own, 

2015-02 text4

2017-01 text4

2005-05-01 text3

2014-01-05 text1

2019-01-02 text2

[pwhodges 1523]

It seems to me that the dates are being concatenated and treated as numbers; in your example:

 

201502

201701

20050501

20140105

20190102

 

which in that form are correctly ordered.  The workaround I'd suggest is to ensure that each date is fully specified, say using 00 for missing days or months.

 

Paul

[Happy2Play 8252]

It seems to me that the dates are being concatenated and treated as numbers; in your example:

 

201502

201701

20050501

20140105

20190102

 

which in that form are correctly ordered.  The workaround I'd suggest is to ensure that each date is fully specified, say using 00 for missing days or months.

 

Paul

 

Since hyphens are a sort remove characters this is true.

  <SortRemoveChars>
    <char>44</char>   = ,
    <char>38</char>   = &
    <char>45</char>   = -
    <char>123</char>  = {
    <char>125</char>  = }
    <char>39</char>   = '
  </SortRemoveChars>

[cnwinger 7]

Not sure when or if this is getting fixed, but for anyone else experiencing this issue, you can manually correct it yourself editing the Library.db MediaItems table in SQLite browser.  Just copy the "Name" field values from your Photos and paste it into the "ForcedSortName" field.  You can select them all at once.  They sort correctly after a refresh of your Photos.

Edited January 7, 2020 by cnwinger

[Happy2Play 8252]

Not sure when or if this is getting fixed, but for anyone else experiencing this issue, you can manually correct it yourself editing the Library.db MediaItems table in SQLite browser.  Just copy the "Name" field values from your Photos and paste it into the "ForcedSortName" field.  You can select them all at once.  They sort correctly after a refresh of your Photos.

So the same as if there were as Sort Title in the UI, but this is a lot faster then doing every item.

[Luke 37028]

There will be improvements coming in the 4.4 release where we will stop the auto-padding of digits in the SortName column in the database. I think this will resolve a lot of these edge cases.

 

Unfortunately this will only apply to brand new installations as there isn't an easy way to upgrade servers into this.

[pwhodges 1523]

Would it be possible or practical to implement a NewSortName column alongside the current SortName column, which could be selected on a library basis - thus enabling any newly created libraries to benefit from the change?

 

Paul

[Luke 37028]

No because some screens will display content from multiple libraries and they'll need to be sorted together.

[pwhodges 1523]

Ah, OK.

[cnwinger 7]

Thanks for the feedback everyone.  Glad this is being sorted in the future release, and my manual workaround can be used to fix this for anyone else experiencing the issue with their current library.

[mtjj 31]

I am also encountering this issue - all of my photo folders are randomly supported, despite the naming convention being exactly the same for all ("yyyymmdd-hhmmss-[original file name]"). I don't really see how this can be caused by a truncation of the file names - see example screenshot below:

• The photos are generated by a lightroom export so
  • I am 99% sure that the metadata is consistent
  • the sequence of file creation should match the sequence of the file name, so it's not a case of photos being processed by emby in a different order.
• I am deliberately picking a folder where all photos were taken with a single camera, processed in the same way by the same software, so there is no question of any inconsistencies from the original photos.
• All photos have a "P" in the name, so even once the hyphens are removed it should not be treating these as numbers.

 

 

For reference, the folder in explorer: 

 

If I click on sort "data added" then the sort order is correct. If I then click on sort "Title" it reverts to the weird sort order above. If I click on the arrow, to toggle ascending / descending it has no effect. Not sure what is happening here    For all photos the "sort title" is blank, but I don't understand how it can be falling back on the title and ending up with the sequence above....

 

 

Not sure when or if this is getting fixed, but for anyone else experiencing this issue, you can manually correct it yourself editing the Library.db MediaItems table in SQLite browser.  Just copy the "Name" field values from your Photos and paste it into the "ForcedSortName" field.  You can select them all at once.  They sort correctly after a refresh of your Photos.

 

 

Is this possible with emby running on Synology? I have no idea how to access Library.db.... 

 

(Luke's quote below, I copied and pasted so it shows up as cnwinger)

 

There will be improvements coming in the 4.4 release where we will stop the auto-padding of digits in the SortName column in the database. I think this will resolve a lot of these edge cases.

 

Unfortunately this will only apply to brand new installations as there isn't an easy way to upgrade servers into this.

 

Does this mean I need to uninstall and reinstall once 4.4 comes out? This is something of a pain, as I have only just finished tidying up from the last time I had to do a full re-install (moving windows -> synology). Isn't it possible to run a script or something to re-generate the sort name field?

There will be improvements coming in the 4.4 release where we will stop the auto-padding of digits in the SortName column in the database. I think this will resolve a lot of these edge cases.

 

Unfortunately this will only apply to brand new installations as there isn't an easy way to upgrade servers into this.

 

[Luke 37028]

It's a bit more complicated than that so that's why it is only applying to new installations. thanks.

[Psyborg 43]

It's a bit more complicated than that so that's why it is only applying to new installations. thanks.

 

Will a re-scan of the library in question be enough to correct the sorting order?

[pwhodges 1523]

No.  The database needs to be created from scratch with the changed search parameters for the relevant table - these parameters are baked into the database table structure.

 

Paul

Edited January 25, 2020 by pwhodges

[Psyborg 43]

Thank you, Paul, for clarifying!