Special characters in music library replaced with questions marks

Help and Support Help
#1

I’m writing a Python script to convert M3U playlists exported by Audirvana on my Mac desktop into Volumio compatible playlists. It’s almost working 100%. However, I’m having a problem with special characters (e.g. apostrophe, accented characters, etc) in artist names, album names and song file names.

My music library is on a micro-SD card which is in a USB adapter, which is then plugged into on of the USB3 ports on my Volumio device (a Raspberry Pi 4).

When I look at the library on my iMac running macOS Catalina, I see this Breezin’ song as follows:

896×274 115 KB

However, when I ssh into my Volumio device and look at the file for this song, I see this:

738×206 168 KB

So the apostrophe in this example has been converted into question mark symbols. I have LANG=en_GB.UTF-8 set on both my Mac, and on my Volumio device for the Volumio user.

How can I fix this issue as I am so close to completing my playlist conversion script?

#2

It looks like my issue is to do with missing locales.

I’ve managed to change my LANG to en_US.UTF-8

However, I cannot change my LC_ALL. It is still set to en_GB.UTF-8. This causes the special characters not to be displayed properly. I think this is because the GB UTF-8 locales are not installed. If I manually change LC_ALL to en_US.UTF-8 in an ssh terminal session then I can display the special characters correctly from my music USB drive.

So, how can I change LC_ALL to en_US.UTF-8 permanently?

#3

I have a python script prototype that helps integrate chromecasts with Volumio. It needs to access raw files on disk and serve them as URLs and in a similar vein, I ran into the same issue as yours. I ended up running the script prefixed with…
LC_ALL=en_US.UTF-8

I actually put that into a .sh script wrapper to ensure it always runs with the locale. That might be an option for you.

The default system locale on Raspian is set in /etc/default/locale but I get the feeling it might be over-ridden somewhere along the way as that default seems to be US the very thing I have to override it to.

#4

I’ve managed to import the locales correctly now but there’s still an issue with special characters. I’ve made allowances in my script for these - for example: a questions mark in a song’s URI, a double-quote in a song’s URI, a period symbol at the end of an artist name.

I managed to get both of my large playlists converted successfully BUT only if I removed one song from one of the M3U playlists.

So near, yet so far! I reckon I’m 95% there now and can do without one song in one playlist which messes up the creation of a Volumio playlist.

I’m going to revisit my scrip at the weekend to mop up this final song that contains a special character.