Unicode metadata
Unicode metadata represents text through defined characters and encodings, while normalization, search keys and locale ordering are separate transformations.
Metadata text can contain several scripts, combining marks and punctuation. Correct representation begins with interpreting the file's bytes using the specified encoding.
The word café also demonstrates a second boundary: equivalent visible text can use different character sequences.
One visible word, two stored sequences
| Representation | Code points | UTF-8 bytes |
|---|---|---|
| c a f followed by U+00E9 | 4 | 5 |
| c a f e followed by U+0301 | 5 | 6 |
The second sequence uses an ordinary e plus a combining acute accent. For these strings, NFC normalization produces the composed form; NFD produces the decomposed form.
A byte comparison before normalization can therefore report inequality even though the intended text agrees. A hash of the file's bytes remains a byte relationship; it should not be reinterpreted as a normalized-title identity.
Decoding comes before normalization
A decoder chooses characters from bytes. Normalization then transforms a character sequence under its defined rules.
If UTF-8 bytes were interpreted through the wrong encoding and became garbled characters, normalization generally cannot infer the original bytes and repair that mistake. It acts on the characters it receives.
ID3v2's text-encoding information and Vorbis's UTF-8 value contract provide concrete input boundaries. A library needs the right one before applying any search-oriented text transformation.
Compatibility folding can discard distinctions
NFKC can map the circled digit ① to ordinary 1. That can be useful in a chosen comparison policy, but it changes a distinction present in the original title.
Canonical normalization, case folding, punctuation removal and transliteration are different operations. They should not all be called “make it Unicode”.
An opaque provider identifier also has its own comparison contract. A presentation-oriented normalization policy should not rewrite every identifier in the record.
Preserve the title and derive a useful key
A library can retain the authored text for display while deriving another key for searching. The search key can normalize or fold selected differences to improve candidate recall.
That does not make the resulting strings unique work IDs. Two distinct films can share the same normalized title.
For Emby metadata comparisons, separate source bytes, decoded text, imported display value and any search/sort representation. A correct database title can still be presented with a different font; a missing glyph is a rendering issue rather than proof of bad decoding.
Alphabetical order adds locale rules
Unicode code-point order is not a universal human alphabetical order. Locale collation can place accented letters differently and can apply case or numeric options.
Sort titles develops that relationship with an original locale example. Normalizing a title does not by itself choose the language's sorting policy.