execute .so

Programming applications for making music on Linux.

Moderators: MattKingUSA, khz

Post Reply
User avatar
skei
Established Member
Posts: 337
Joined: Sun May 18, 2014 4:24 pm
Has thanked: 8 times
Been thanked: 57 times
Contact:

execute .so

Post by skei »

hi again.. been busy with some music stuff, but now i'm back to coding again..

in my toolkit, each plugin format is more or less just a thin wrapper that acts as a 'translator' between the host and our internal plugin format, and the plugin code itself is separated from the wrapper code.. so, i recently added the possibility to have all the plugin format wrappers in the same binary .so file.. and it seems to work pretty well..

since i can also compile a plugin as a standalone executable, i wanted to experiment with the possibility of executing the final shared object (.so) as a normal executable too.. after some googling and testing, i found this: https://stackoverflow.com/questions/301 ... bject-file

main.c:

Code: Select all

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

#ifdef INTERPRETER
  const char interp[] __attribute__((section(".interp"))) = INTERPRETER;
#endif

void entry_point(void) {
  fprintf(stderr, "hello executable shared object world !\n");
  _exit(EXIT_SUCCESS);
}
compile:

Code: Select all

gcc main.c -Wall -Wextra -fPIC -shared -o main.so -DINTERPRETER=\"/lib64/ld-linux-x86-64.so.2\" -Wl,-e,entry_point
then i can just execute the main.so file:

Code: Select all

./main.so
and it prints out the "hello executable.." text string..

but is this 'safe'?
what about that /lib64/ld-linux-x86-64.so.2
is this valid across distributions? versions?

- tor-helge
User avatar
mike@overtonedsp
Established Member
Posts: 145
Joined: Mon Apr 24, 2017 5:26 pm
Location: Oxford, England
Been thanked: 55 times
Contact:

Re: execute .so

Post by mike@overtonedsp »

Perhaps I'm not fully understanding the question, but what's to stop you just creating a main.c or main.cpp file containing all the necessary (e.g. standard 'main' ) boiler-plate code which gets the (executable) up and running, creates an instance of your 'plug-in' and does whatever you need with it. You can just build all that as you would a normal executable.
(This is a mechanism similar to the one I used when I provided JACK compatible versions of my code - I created some 'main' boiler-plate code which just instantiated the (VST) 'plugin', created a window for its GUI and connected the audio path to / from JACK to the plug-in's DSP code. This was then all compiled into a standard executable - essentially a mini plug-in host, together with the plug-in code, all wrapped up in a standalone executable).
User avatar
skei
Established Member
Posts: 337
Joined: Sun May 18, 2014 4:24 pm
Has thanked: 8 times
Been thanked: 57 times
Contact:

Re: execute .so

Post by skei »

i know there's not much of a reason.. it's more that i am curios, and it sounds like a fun thing to try.. :-)

the point is to NOT have a separate binary executable.. for example, i could release one plugin_pack.so file, which contained several plugins, each in all the supported formats (ladspa, dssi, lv2, vst2, vst3), including standalone.... you could load the .so in a host, and it acts as a plugin, execute the .so directly (as in my first post), and it would set up jack and midi and all that, and then act as a minihost as you said..

i have around 30 or so simple plugins that i made during the development and testing of the toolkit/framework.. if i want to release all these in all the formats, including standalone, int would mean 180 (30 * 6) files.. if i could combine all of them into one single binary file, it would simplify things.. i could install this single file somewhere, and then use synlimks in my ladspa/lv2/vst/etc directories.. or execute it directly..

but, well..
it's not _very_ high on my priorities list, but something i like to experiment and play around with :-)

- tor-helge
User avatar
davephillips
Established Member
Posts: 592
Joined: Sat Aug 15, 2015 1:05 pm
Has thanked: 35 times
Been thanked: 23 times

Re: execute .so

Post by davephillips »

Greetings,

Your plan sounds similar to the plugin API for VCV Rack. Each plugin developer creates a single plugin for use in the Rack, but that plugin can contain any number of modules to be listed and invoked separately. Thus, one plugin might contain only one module while another might contain fifty. Of course, this method assumes a single format for everything, your plan is more ambitious. :)

Best,

dp
User avatar
skei
Established Member
Posts: 337
Joined: Sun May 18, 2014 4:24 pm
Has thanked: 8 times
Been thanked: 57 times
Contact:

Re: execute .so

Post by skei »

cool! nice to hear that other people have been thinking in similar ways!
my idea is maybe not too far fetched, as i thought, hehe..

multiple plugins in the same binary is supported already by all plugin formats.. a little messy with vst2 and all that shell stuff, but absolutely doable.. with all other formats it's very easy..

having multiple plugin formats in one single binary isn't much of a problem either.. i already tested a .so that works as a ladspa, dssi, lv2, vst and vst3 plugin :-) i put the binary in one place (~/.skei.audio), and then made a bunch of symlinks in the regular plugin directories.. and that seems to work in all the hosts i've tested..

and about that "execute .so as it was a regular standalone executable".. i started looking into it when i discovered that some of the .so files aready on my system could be executed as a standalone binary.. for example, both of these prints out some info when run:

Code: Select all

$ /lib/ld-linux.so.2
$ /lib/x86_64-linux-gnu/libc.so.6
- tor-helge
User avatar
mike@overtonedsp
Established Member
Posts: 145
Joined: Mon Apr 24, 2017 5:26 pm
Location: Oxford, England
Been thanked: 55 times
Contact:

Re: execute .so

Post by mike@overtonedsp »

the point is to NOT have a separate binary executable.. for example, i could release one plugin_pack.so file, which contained several plugins, each in all the supported formats (ladspa, dssi, lv2, vst2, vst3), including standalone....
Ok that makes it clearer - however, I don't know about other users, but personally - when I'm a user, not a developer - I really don't like the idea of a single 'mega plugin' which contains all the other plug-ins a developer ever made.
I've run into this issue a few times in the past with some commercial plug-ins, and while they don't necessarily pack everything into a single binary, they often provide just a single (Windows) installer which only allows you to install everything the manufacturer makes, plus some kind of download manager, plus a license hub, etc etc, all of which end up out of revision, so the first time you use it it triggers yet another multi-gigabyte download.

If you want to build multiple formats, architectures etc, better to invest the time in creating a build system that just builds and packages all the formats for you* (since that's essentially the bulk of what you'll be doing by combining them into a single binary anyway).

Developing for multiple formats, on multiple architectures is just a particular circle of hell that plug-in developers have to endure in the hope of finding enough users (I currently spend 99% of my 'development' time updating and re-building / re-testing some or other of my existing plug-ins for Windows VST, Windows VST3, Windows AAX, Mac, Mac Audio Unit, Mac VST2, Mac VST3, Mac AAX, Linux VST2 (and until recently, ARM / RPi3 / Linux VST2) ARM /MOD (maybe) - not to mention testing with many hosts - all of which will at some time or other do *something* slightly different to either each other or the common sense interpretation of the VST2 , VST3, AAX or whatever specification and require host specific debugging...
(I could go on, but I won't - you get the idea... :) )

*With of course the caveat that the time you save by getting a computer to do something for you is always spent watching it to make sure it doesn't do it wrong...
User avatar
skei
Established Member
Posts: 337
Joined: Sun May 18, 2014 4:24 pm
Has thanked: 8 times
Been thanked: 57 times
Contact:

Re: execute .so

Post by skei »

mike@overtonedsp wrote:I don't know about other users, but personally - when I'm a user, not a developer - I really don't like the idea of a single 'mega plugin' which contains all the other plug-ins a developer ever made.
I've run into this issue a few times in the past with some commercial plug-ins, and while they don't necessarily pack everything into a single binary, they often provide just a single (Windows) installer which only allows you to install everything the manufacturer makes, plus some kind of download manager, plus a license hub, etc etc, all of which end up out of revision, so the first time you use it it triggers yet another multi-gigabyte download.
yeah, i understand what you say.. but.. i'm not planning to release commercial plugins (i prefer to give away everything i make - free/opensource), nor any download managers, service centers or anything like that.. like you, i dislike that kind of crap.. also, i think even one megabyte is gigantic for a plugin, so don't expect gigabyte sized plugins from me... my plugins are just a few 100 kb's, depending on how many formats i have in the binary, and how much additional stuff like editor bitmaps, etc..

multiple plugins in one binary is already pretty common with ladspa plugins, i think?
mike@overtonedsp wrote:If you want to build multiple formats, architectures etc, better to invest the time in creating a build system that just builds and packages all the formats for you* (since that's essentially the bulk of what you'll be doing by combining them into a single binary anyway).
i already have a flexible build system like that! i can already compile single- or multi-plugin binaries, single- or multi-format.. depending on a few #defines and/or cmd line args).. this part is done, works well.. what i wanted to know/learn with the initial post, was if the method of executing a .so file as if it were a normal standalone binary is safe/doable, portable across distros.. not how to do it, or if it is a good idea (i will make that decision later)..

i think it's an interesting thing to experiment with, no matter if it turns out to be useless for making plugins...
mike@overtonedsp wrote:Developing for multiple formats, on multiple architectures is just a particular circle of hell that plug-in developers have to endure in the hope of finding enough users (I currently spend 99% of my 'development' time updating and re-building / re-testing some or other of my existing plug-ins for Windows VST, Windows VST3, Windows AAX, Mac, Mac Audio Unit, Mac VST2, Mac VST3, Mac AAX, Linux VST2 (and until recently, ARM / RPi3 / Linux VST2) ARM /MOD (maybe) - not to mention testing with many hosts - all of which will at some time or other do *something* slightly different to either each other or the common sense interpretation of the VST2 , VST3, AAX or whatever specification and require host specific debugging...
(I could go on, but I won't - you get the idea... :) )

*With of course the caveat that the time you save by getting a computer to do something for you is always spent watching it to make sure it doesn't do it wrong...
yeah, i know.. i have spent ten years or so on this toolkit, ported it back and forth between linux and windows, 32 and 64 bit, and even to object pascal and then back to c++.. i have rewritten the lowest levels (the plugin format wrappers) several times because i wanted to add another format, that worked differently than all the other formats.. if i ever add aax or au formats, or want to support mac (very unlikely), i guess i might need to to it all over again..

- tor-.helge
Post Reply