Logs are here:
More than 3 hours booting this morning, and still not there! Not sure, but it seems that things get worst day after day.
More hints:
-
first boot of the day is always very long. Goes into swapping, and takes more than 30 mn,one hour, etc… After unplugging PSU and restarting, boot takes 5 mn, but at least, it does not stay blocked for hours. So, to summarize, first boot of the day is always a problem. Further ones are better.
-
during these 5 mn, sometimes, I have access to the WebUI (I can navigate through the menus), but I don’t have access to the music (WebRadio or NAS). The system becomes ready only when the “NAS” appears in the music library. This correspond to the end of having the “node” process taking all CPU and RAM.
So I guess that during these long minutes with no trace in the log, the “node” process makes avalaible the music files and library? But why is it so long?
Bad news… I have flashed another SD card, to start again from the beginning.
First boot: eveything comes up in a second. The “node” process took 10 s of CPU time. Good.
I go into declaring my database (access through NFS, wired ethernet). Scan starts immediately. The scanning of the database took almost 3 hours. Not surprising, it was the same with the other SD card.
Reboot volumio: back to the same issue. The “node” process took more than five minutes of CPU time before finishing.
I have not installed anything (no plugin, etc), I just configured the path to my music.
Below the result of the “ps -aux” just after the webUI is accessible: 5:49 minutes of CPU Time.
volumio 1022 51.0 16.9 304024 168976 ? Ssl 15:39 5:49 /usr/local/bin/node /volumio/index.jsSecond chance: reboot again. Same result…
Is this database too large for volumio? I had a much bigger one (twice this size) with mpd alone on my previous platfom (an old eeepc 701…) , without any performance issue.
ls -l /var/lib/mpd/
total 41136
drwxr-xr-x 2 root root 71 Dec 6 23:34 music
drwxrwxrwx 2 mpd audio 3 Dec 12 2014 playlists
-rwxrwxrwx 1 root root 42117562 Jan 18 15:00 tag_cacheFile “tag_cache” is 42 MB large.
cat /var/lib/mpd/tag_cache | wc -l
1357860In the WebUI, “MyMusic” shows me:
5076 Artists
4746 Albums
126859 Songs.
8872 hours of playtime.
Which looks good to me. Is this too much for volumio?
But, from my understanding, this file “tag_cache” is managed by mpd.
Is the “node” process scanning this file at boot time, each time? What for?
Thanks for your help.
What I understand from now: I’m not sure I’m facing a bug, but a true performance issue. I don’t know which one I prefer…
-
the time needed for “/usr/local/bin/node /volumio/index.js” at boot time is closely related to the size of the MPD database. The greater the DB, the longer the needed CPU time. My DB is 42 MB large, it takes 5 mn to “load” it for the “node” process. 5 mn of CPU time, and a lot of RAM memory. So, when memory is missing, this can take hours…
-
things get worst when the touch screen plugin is active, because more processes are present at boot time (chromium-browser for example). So this extra load is not the best thing that can happen…
What I don’t get, is that MPD + GMPC alone work like a charm on a database which is twice the one I have now on Volumio (89 MB), as I have today loaded only half of my DB… I’ve been using this for years on a old eeepc 701, I have never seen such a performance issue.
So, what can I do?
-
I could use a faster SD card. The one I have is a “standard” Class 10 (noname). What about giving a chance to a Kingston or Samsung “pro”. Will it help?
-
any way to postpone the launch of chromium-browser at boot time? I don’t really need the touch screen display when booting, I would prefer to save CPU and RAM for the node process. Are they launched in /etc/init.d? Any way to insert a pause/timer there?
-
Would it make sense to split my DB in parts? The global size would be the same, but I could make 5 or 6 different “first level” directories. Would it help?
-
go back to MPD/GMPC
?
Any other suggestion?
Hi,
I have now reduced as much as I can the size of my DB (mostly suppressing symlinks):
volumio@volumio2:~$ ls -l /var/lib/mpd/tag_cache
-rwxrwxrwx 1 root root 28484728 Jan 19 11:21 /var/lib/mpd/tag_cache
volumio@volumio2:~$ cat /var/lib/mpd/tag_cache | wc -l
915369Number of lines is slighlty lower, but the size of the file was reduced (43MB => 28 MB), mostly because the pathnames are shorter (the way I declare mount points). So, file is much smaller, and holds 10% less lines.
I can’t do more without suppressing music… which I don’t want.
I have also splitted into 5 mount points, instead of one, but I’m not sure it changes anything concerning performance, but this reduces the size of the tag-cache file, because pathnames are shorter.
I have now 2:51 of CPU time needed for the node process at boot time (instead of 5:10). That’s 30% less. This corresponds more or less to the reduction of the file size.
I have not installed the touchscreen plugin yet… And I have not yet declared the other half of my music library. I fear that this will make the system out of use again…
As far as I understand, the “node” process" makes a scan of the mpd database during each boot. Even if this database has not changed. It seems to me that the CPU time needed for the node process is somewhat proportionnal to the size of the “tag_cache” file… Am I wrong?
Can somebody explain me why it is needed to do so? I understand that, for an update, or a rescan of the DB, this is mandatory, but why is it done at each boot?
I have been using mpd/gmpc for years, and it is clear for me that no rescan is needed if nothing was changed in the music library, the mpd clients have an immediate access to the db, whatever its size.
GMPC displays the same information as Volumio (music libraries, albums, tracks, album art, etc…), but it shows it in a second, and does no make any scan.
In volumio, it seems to me that large music libraries have a direct impact on boot time. Not so nice.
I’m very hapy with the audio features in Volumio, quality of sound, user interface, etc… But performance is another story. Can anything be done to solve this?
Situation is now more stable.
With my “reduced” mpd db, node requires now 2:50 before letting other processes do something… After several boots, this time remains the same.
I have installed the touchscreen plugin, and inserted a “sleep 240” at the beginning of /opt/volumiokiosk.sh. This means that the X server and chromium are launched 4 mn after boot, leaving CPU and RAM to “node”. This seem to work for the moment.
I’ll give a chance to a higher speed SD card, I have ordered a “Kingston Sdcg2 32go Micro Sd Sdhc Classe 10 Uhs-i U3 V30”. Let’s see if things are better when it arrives.
The 240 sec waiting time could be adjusted, but I’m wondering how to do that. What could I monitor in order to be sure that the node process has finished its uptdate job? Something in the log?
Of course, if I load the second half of my music library, things will get worst. But for the moment, I can live like that.
It would nevertheless be nice to reduce the load of this “node” process at boot time.
I have now attached a USB HD instead of using my NAS. This solves the issue, don’t ask me why…
Even with my “full” mpd db (71MB, 2,6MLines, 216000 Tracks, 13000 hrs of Music):
volumio@volumio:/var/lib/mpd$ ls -l
total 69852
drwxr-xr-x 2 root root 71 Dec 7 00:34 music
drwxrwxrwx 1 mpd audio 4096 Jan 16 09:31 playlists
-rwxrwxrwx 1 root root 71513293 Jan 17 12:37 tag_cache
volumio@volumio:/var/lib/mpd$ cat ./tag_cache | wc -l
2594913It just boots now in 30 seconds, most of this time taken by the “mpd” process. The main “node” process takes only 10 or 15 seconds of CPU time. So, there’s something strange when accessing from network, at least for big libraries.
I don’t put “SOLVED” in the title.
Over 1 year after this post, I can only confirm what dk31 writes. At the moment I’m still in the test phase with Virtuoso, but it looks like nobody here has a solution for large music libraries. It is always pointed out that the size is not a problem, but that is not true!
Working with a USB hard drive is counterproductive, as synchronization with the NAS must always be done manually. I am very disappointed at the moment because the product is otherwise very well thought out.
Hello,
Back on this topic... I have found a very nice solution (at least for me...) for this issue.
I think that the problem, on Volumio side, is to
-
restart Volumio each day (or each time you want to listen to music),
-
rebuid the full datase, even if nothing has changed; for me, this is a non sense.
This is what the “node” process does at each boot.As far as I understand, mpd is very fast for updating the music library, but this is not the case for volumio, when this library is large. So, for me, the best solution, when using a NAS, is to have your DB located on this NAS, and not in Volumio…
For doing this, I use subsonic.
For me, this solution is perfect because my NAS (RPI4) already runs subsonic (http://www.subsonic.org/pages/index.jsp), which allows me to listen to my music from anywhere through a web interface. And there is a very nice plugin for olumio, named Volusonic, which allows volumio to stream music from subsonic. I gain also a couple of features from subsonic: random albums, newest albums, etc…
So, since I use Volusonic, I have absolutely no performance issues anymore! All you need to check whether you NAS can run subsonic or not.
DenisHi dk31!
so, what you fix boot time with USB, thank you help!
Mac!
The problem persists. I have a rasberry pi 3 with the latest volumio. My music collection (140k songs) resides on a synology nas connected via cifs. Since I installed a 7" lcd display and the touch pad plugin the raspberry freezes completely. It boots up okay but node is taking more and more memory (up to > 60%) and runs out of memory. Eventually it freezes completely (well, I ran out of patience after 24h …). It already took a long time to boot without the touch lcd plugin but now X is also taking memory…
Really crushes the user experience.
Hi,
I totally agree with you. This is for me the BIG issue in Volumio, and the problem is still there in the latest versions. I don’t understand why nobody from the “devt” team has taken a look at this.
Anyway, it looks your are in a situation that is very similar to the one I encountered, so I guess I can help you.
As far as I understand (I may be wrong), when the music library is large “enough”, the scan of this library by volumio takes a lot of time, and also a lot of memory. If you add the touch display plugin, this plugin runs an additionnal web browser, which takes more memory. The whole system comes then into a swap process (you can seen the “kswap0” process running"), but there is no swap on this RPI. So, it takes years…
A workaround is possible in 3 steps. See below. You need to have a ssh connection to your volumio RPI. You must also to be able to edit some files on your RPI (you can use “nano” for this).
First step: you must get sure that the automatic update of volumio database is switched off. When modifying the music library, you will have to tell volumio to update it manually, but only after the boot sequence is complete.
For doing this, you must:
- check this line in /etc/mpd.conf
auto_update “no”
Value must be “no”. If “yes”, put “no” instead.
-
do the same thing in “/volumio/app/plugins/music_service/mpd/mpd.conf.tmpl”
-
in file “/volumio/app/plugins/music_service/mpd/config.json”, you must set this value to false:
“auto_update”:{
“type”: “boolean”,
“value”: false
},
Second step: once you have done this, you must temporarly “unplug” the “touch display” plugin, to prevent it from launching the browser: just edit the “/opt/volumiokiosk.sh” file (you must be root) and insert “exit 0” just after the first line, like this:
#!/bin/bash
exit 0
while true; do timeout 3 bash -c “</dev/tcp/127.0.0.1/3000” >/dev/null 2>&1 && break; done
…
Now, you can reboot your RPI, or just run “killall node”, this will restart everything, but nothing for the touch display. You should see nothing on your LCD display.
Third step: in this configuration, you must now estimate how long it takes for volumio to load your database. When volumio is ready to play music (i.e when you can actually explore you music library in volumio), type “ps -aux | grep node” in you sshconsole
volumio@volumio-sub:~$ ps -aux | grep node
volumio 1322 0.7 9.4 206464 93520 ? Ssl 09:24 0:19 /usr/local/bin/node /volumio/index.js
volumio 1376 0.0 2.6 72636 25908 ? Sl 09:25 0:00 /bin/node /volumio/app/plugins/miscellanea/albumart/serverStartup.js 3001 /data/albumart
volumio 1382 0.0 3.5 115112 35092 ? Sl 09:25 0:01 /bin/node /volumio/app/plugins/miscellanea/albumart/serverStartup.js 3001 /data/albumart
volumio 1383 0.0 3.5 115240 35008 ? Sl 09:25 0:01 /bin/node /volumio/app/plugins/miscellanea/albumart/serverStartup.js 3001 /data/albumart
volumio 1388 0.0 3.5 115132 34864 ? Sl 09:25 0:01 /bin/node /volumio/app/plugins/miscellanea/albumart/serverStartup.js 3001 /data/albumart
volumio 2953 0.0 0.1 4164 1380 pts/1 S+ 10:08 0:00 grep node
volumio@volumio-sub:~$
The first line will tell you how long it takes for the /usr/local/bin/node process to finish its job (19 seconds in my example, see “0:19” just before /usr/local/bin/node on the first line).
Yours might be much longer…
Repeat this operation a couple of times: killall node, wait for volumio to be ready, and check the time with the ps command.
Once you are pretty sure about this duration (it should more or less be stable), edit again the /opt/volumiokiosk.sh file, and replace “exit 0” with “sleep X”, where X is a number of seconds which is slightly larger than the time you need. For example, if the node process needs 2:30 minutes, put 3 minutes (i.e 180), like:
#!/bin/bash
sleep 180
while true; do timeout 3 bash -c “</dev/tcp/127.0.0.1/3000” >/dev/null 2>&1 && break; done
…
And you’re done. Just reboot, or run “killall node” again.
Hope this helps.
Again, it would be very nice to have a feedback from the Volumio team on this issue.
DenisHi Denis,
just only saw your response now. Many thanks for this. I will give it a try. I also followed your suggestion to install subsonic. This works but my NAS (DS216se) is way too slow for it. Anyway, this thread was a kind of lifesaver for me as it clearly describes the problem even though there is - regrettably - no solution by the devs for it.
1 Like
Sorry to see that your NAS is too slow for subsonic. Here, it works on a very old RPI (model B), which is not a very powerful computer.
You may also give a try to minidlna, I don’t use it extensively, but I noticed it worked also, without installing anything on Volumio (you see it appear in “media servers”). Just installed minidlna
Sorry, the answer went away without noticing… So, I just installed minidlna on the RPI server, configured the DB, and it worked immediately.
Hope it helps.
Denis
Back to this topic of very long time needed to be able to play music after boot…
It seems to me that the problem of very long “node” process after boot is now solved with Volumio 3! It was around 2 mns with volumio2 (last version), and was drastically reduced to 10 seconds with volumio 3 (3.173)!
Probably something changed in the pagination method? Hope this stays as it is now!
Thanks to the volumio team, although it was a bit long…
Denis