CSV encoding & UTF-8 problems — ü became ü, and other classics

Garbled characters after a CSV import are never random — each pattern names its cause. The three-second diagnosis:

You see	It means
ü, ß, é where ü ß é belong	UTF-8 read as Windows-1252 — re-read as UTF-8
ü where ü belongs	the double-whammy: UTF-8 → 1252 → UTF-8 again
� (replacement characters)	bytes weren’t valid in the assumed encoding — likely 1252 data read as strict UTF-8
Spaced letters / CJK-looking pairs	UTF-16 (Excel “Unicode Text”) read as a single-byte encoding
Name as the first header	UTF-8 BOM not stripped — breaks header matching

The fixes

• UTF-8 mojibake (ü): the file is healthy — the reader chose wrong. Re-open/re-read as UTF-8. If a tool already saved the mojibake, convert back (iconv -f utf-8 -t windows-1252 reverses one round).
• Excel “Unicode Text”: it’s UTF-16 + tabs. Either read it as UTF-16 (the leading BOM FF FE/FE FF tells you the endianness) or re-export as “CSV UTF-8”, which modern Excel offers.
• Legacy exports (� under strict UTF-8): genuinely Windows-1252/Latin-1 files still exist. Detect: valid-UTF-8 check fails but every byte is printable 1252 → decode as 1252.
• BOM in the first header: strip EF BB BF before parsing, or your Name column will never match anything.

What a good importer does automatically

This is detection, not guesswork — and it’s testable. Mildport’s decoder, in order: strip or honor the BOM; detect BOM-less UTF-16 by NUL-byte density (both endiannesses); try strict UTF-8; fall back to Windows-1252 only when UTF-8 is provably invalid. Every case above is a pinned regression test in a byte-level parity corpus, so browser and server decode the same file identically.

Your users never see any of it — ü arrives as ü, headers match, and the file imports.

Test it with your worst file: the playground runs in-browser, no signup. More parsing failure modes: CSV not importing correctly.