Plugin Development - Open Questions

Hi,

I’m playing a bit with developing a Volumio plugin. I have read the documentation available at volumio.github.io/docs/index.html, however I have a question concerning the node modules configured in the package.json file as “dependencies”:

The documentation does not cover how these modules are installed or treated. I assumed they would be installed system-wide and if I set a requirement, it would simply be installed. However, this is not the case: if I install the module and try to start it, volumio tells me

2016-12-10T15:03:12.502Z - info: Enabling plugin myplugin
2016-12-10T15:03:12.520Z - info: Loading plugin "myplugin"...
2016-12-10T15:03:12.557Z - info: Error: Error: Cannot find module 'kew'

so even the most elementary modules are not found from some system installation.

Looking at other people’s projects I see that they started to simply copy and paste all required modules into the node_modules directory - which is unfortunate both because the will never get updated ever, and because then we end up with 100 duplicate copies of the same library scattered on the system.

What is the official procedure or idea for this?

Some more thoughts on plugin development:

The current system is very dangerous since badly written (user) plugins can easily pull a parse error or type error and thus crash the volumio process at startup. The process will be respawned, having volumio ending up in sort of a “boot loop” with the web ui unavailable.

Imho somehow the plugin execution should be made more safe, such that an eventual death at startup (or later) will not kill the whole volumio process.

Could this somehow be done (I’m completely clueless when it comes to Node.js or JS in general)

Hey,

The official spotify plugin calls “apg-get install…” during installation via install.sh:
github.com/volumio/volumio-plug … install.sh

In another plugin, “npm install” in the same script during installation:
github.com/ivesdebruycker/volum … -playlists

That way you can do anything including completely trashing the system, but you can probably also install npm packages globally. However, if you’d do that, who would ever clean that stuff up after you remove the plugin? You never know whether the globally installed packages are still needed by other components.

Jochen

Hi simonspa,

We do not install them globally, and plugins need to have their plugins added to the zip. The main reason is that some modules need to be built, and since we don’t have compilers installed by default on Volumio we can’t. And even if we have them, the installation would be veery long.
Having few modules duplicated is not a big deal in terms of space, and when we update the plugin we download again the modules and update them

I agree with you but we haven’t found yet a way to completely decouple plugins from the main system. After all, is just a matter of good coding practices, aka if you handle all possible errors correctly, the plugin should never crash…

If you have suggestions on how to make the system more robust, they are very welcome!

I definitely see the point for user compiled binaries in certain cases, but I think the majority of plugins won’t need this, resulting in the mentioned code duplication that I agree should be avoided.
I also see the point of jochen’s statement about security problems with apt-get, but I think that a proper review before pulling a new plugin can easily solve that. I think the bigger problem is that you can easily install stuff via apt-get, but either you remove it on uninstallation, possibly crashing another plugin or you don’t uninstalll because you’re afraid of doing exactly that.
I think the solution to the two mentioned problems is relatively simple: We need a central plugin manager that takes care of installing, uninstalling and dependencies. Short example:
Plugin A gets installed and informs the manager about its required packages and their respective version, say package P 1.04 and Q 54. The manager will then install these packages using the package manager apt-get etc, maybe P is even released in version 1.05.
Now Plugin B gets installed, requiring package P 1.03 and R 12.6. The manager will check its internal dependency graph for package P and see that it already installed 1.05 and will therefore not install P again, but only R.
If Plugin A gets uninstalled again, it will check its internal dependency graph for packages P and Q and will only find A using Q, uninstalling Q from the system. For P however it will find A and B, thus removing A from the graph, leaving B as package that depends on P and not uninstall P.

Using the package manager also enables easy update of system packages without leaving this crucial security aspect to each and every plugin developer. Only in case that an update breaks a package, the blame should be put on the developer to fix this.

However, I don’t agree with

I think we all know that this is just wishful thinking and every human will always eventually make a mistake, leading to an uncaught error or unexpected behaviour. Therefore all developers should do their best to avoid relying on others not making any mistakes. This holds not only for volumio, but for every project.

I completely agree with you, @Fightclub.
Having a central manager that is simply given a list of dependencies (let’s say, we support apt and npm) and which performs the dependencies check and does the installation would help a lot.

I think this really is a crucial point. Especially for Volumio plugins, many authors will hack the plugin once until it works, publish it and never look at it again since it simply works. This way, we end up with tons of different versions of packages with all sorts of vulnerabilities.