Database update stops after a few artists

I have also changed these values in mpd.conf and it seems to be working fine now. It is still scanning and will be busy for a while as I have over 275K tracks to import, will post an update when it (hopefully) reaches the end…

max_playlist_length to “81920”
max_output_buffer_size to “81920”

Thanks for the help!

By the looks of it /var/log will be full long before it will reach the end of the import, but let’s keep hopes up and see what happens… :slight_smile:

Yeah, /var/log is full now, but this doesn’t seem to stop the import so far, the number of tracks is still increasing…

volumio@volumio:/var/log$ df -h
Filesystem          Size  Used Avail Use% Mounted on
overlay              28G  349M   27G   2% /
/dev/sdb2           2.7G  1.8G  742M  72% /imgpart
/dev/loop0          579M  579M     0 100% /static
devtmpfs            1.5G     0  1.5G   0% /dev
tmpfs               1.5G   64M  1.5G   5% /dev/shm
tmpfs               1.5G  4.9M  1.5G   1% /run
tmpfs               5.0M  4.0K  5.0M   1% /run/lock
tmpfs               1.5G     0  1.5G   0% /sys/fs/cgroup
tmpfs               1.5G  100K  1.5G   1% /tmp
tmpfs               1.5G     0  1.5G   0% /var/spool/cups
tmpfs                20M   20M     0 100% /var/log
tmpfs               1.5G     0  1.5G   0% /var/spool/cups/tmp
/dev/sdb1           487M   32M  455M   7% /boot
//192.168.1.134/1M   19T  8.9T  9.4T  49% /mnt/NAS/musiXerver
tmpfs               302M     0  302M   0% /run/user/1000
volumio@volumio:/var/log$ ls -l
total 20480
-rw-r--r-- 1 root    root       21174 Mar  1 08:13 Xorg.0.log
-rw-r--r-- 1 root    root        1671 Mar  1 08:13 boot.log
-rw------- 1 root    utmp           0 Mar  1 08:13 btmp
-rw-r--r-- 1 root    root    20824064 Mar  1 08:58 mpd.log
-rw-r--r-- 1 volumio volumio    81920 Mar  1 08:57 volumio.log
-rw-rw-r-- 1 root    utmp        1920 Mar  1 08:14 wtmp

Editing mpd.conf with the values suggested and then restarting MPD (via systemctl) has improved things for me.
The scan completes and I have re-gained access to my library. No errors in mpd.log. I hope this persists,

Jim

All good for me, the import has completed… browsing the library really is painfully slow and if I open the list of albums the interface crashes… but one thing at the time, at least all my music is there :wink:
Image5.png

You guys are lucky, in my case it’s still not working. Maybe because I am using a local USB hard disk and not a NAS?

The posted suggestion is still working for me – 11 hours and a re-boot later. Similar to what amatala reported , though, the system is sluggish – cover art doesn’t appear, or appears only very slowly, refreshing the browser display is slow and so on. But there is no memory allocation exception reported in mpd.log and playback seems fine so far.

Since version 2.729 it stopped working again for me and I’ve been waiting for an update for weeks now… Need to give up on this as it’s clearly not stable enough to used in decent conditions…

You have a huge library… Which hardware are you using?

I didn’t have any luck with newer versions than 2.699. Any more recent would have the same issue as described here. 2.699 is the most stable version I’ve found and I’m sticking to it, give it a try.

Which platform are you on?

The issue originates by some buffer sizes of MPD (which can allow to browse big libraries without problems). Issue is, they work on some platform while they absolutely don’t in others. We need to understand how\where they work.

SIDE NOTE: We are working on our own database engine which will be able to handle 1) Big libraries 2) Libraries with corrupted files. Its about one year we work on that, not easy task but we are seeing the end of it…

Sorry don’t know if the question was for Amatala or me - I use a Sparky (Allo USBridge).

That DB engine project sounds like a major work, good luck (and looking forward to it!).

Ok, this is an interesting feedback.
Can you post here the mpd.conf you have?

There you go:

Volumio MPD Configuration File

Files and directories

music_directory “/var/lib/mpd/music”
playlist_directory “/var/lib/mpd/playlists”
db_file “/var/lib/mpd/tag_cache”
log_file “/var/log/mpd.log”
#pid_file “/var/run/mpd/pid”
#state_file “/var/lib/mpd/state”
#sticker_file “/var/lib/mpd/sticker.sql”
###############################################################################

General music daemon options

user “mpd”
group “audio”
bind_to_address “any”
#port “6600”
#log_level “default”
gapless_mp3_playback “no”
#save_absolute_paths_in_playlists “no”
#metadata_to_use “artist,album,title,track,name,genre,date,composer,performer,disc”
auto_update “yes”
#auto_update_depth “3”
###############################################################################

Symbolic link behavior

follow_outside_symlinks “yes”
follow_inside_symlinks “yes”
###############################################################################

Input

input {
plugin “curl”

proxy “proxy.isp.com:8080

proxy_user “user”

proxy_password “password”

}
###############################################################################

Decoder

###############################################################################

Audio Output

resampler {
plugin “soxr”
quality “high”
threads “1”
}

audio_output {
type “alsa”
name “alsa”
device “hw:1,0”
dop “no”
audio_output {
type “fifo”
enabled “no”
name “multiroom”
path “/tmp/snapfifo”
format “44100:16:2”
}

#replaygain “album”
#replaygain_preamp “0”
volume_normalization “no”
###############################################################################

MPD Internal Buffering

audio_buffer_size “2048”
buffer_before_play “10%”
###############################################################################

Resource Limitations

#connection_timeout “60”
max_connections “20”
max_playlist_length “81920”
max_command_list_size “81920”
max_output_buffer_size “81920”
###############################################################################

Character Encoding

filesystem_charset “UTF-8”
id3v1_encoding “UTF-8”
###############################################################################

Hi there,

I am also having a trouble getting volumio working with a large db.

just updatet volumio to 2.861 on a raspberry PI3 with music attached to NAS.
Right now the system is not useable for me.
Some things i recognized so far:

  1. if mpd starts and auto_update is on (default!) it takes more than 90 secs -> so systemd will kill mpd.service! So setting TimeoutStartSec=600 in /etc/systemd/system/multi-user.target.wants/mpd.service helped keep it running after a reboot
  2. mpd can index 1.4TB of music (around 200k files)
  3. volumio process tries to access mpd and kills it… thats why i have connection refused or socket errors
  4. mpc or volumio kann kill mpd with e.g. “mpc ls NAS/music”
    Jan 03 13:14 : client: [8] process command “lsinfo “NAS/music””
    Jan 03 13:14 : exception: std::bad_alloc
    Jan 03 13:14 : exception: std::bad_alloc
    it should return aroung 1800 directorys - mpd gets killed… “bad_alloc”
    while mpc ls NAS/music/Superartist works fine (listing subdirectorys)

So due to the crashing mpd volumio itself can not get any information about the db for its interface, but also webradio does not work :confused:
But mpd itself with mpc add “url to stream” and playing works fine

The volumio image from 02.2020 worked fine with exact the same music db till yesterday.

I am now going to test the buster beta.

Hi everybody,
i have similar problems with my raspi 3 and hifiberry DAC+.
Library update stops somewhere in the middle. Then the indexed part of the library might be searchable for some time. Then the library gets lost. The network drive is still shown as connected in the sources menu but the number of titles is 0.
The library is on a usb 3.0 harddisk connected to the fritzbox router. Network access is with samba share.
I also tried to connect the harddisk directly to the raspi. I also gave it a try with an usb 3.0 ssd drive.
I tried ntfs and ext3 formatted drives.
My library has about 100.000 songs.
The library update stops or hangs not always at the same point. At 2.000 songs at 50.000 songs - what ever. So i guess it is not a wrecked file in the library.
I have tried various smb mounting options from this and other communities.
Changing the mpd.conf parameters as suggested made no difference.
No difference in using volumio-2.853-2020-11-20-pi.img or volumio-2.698-2020-01-16-pi.img
My next try is the 3.x beta

switching to the volumio3 beta helped solve the issue