Linux Audio Programmers: LMMS needs your help

[tatch]

hey diizy, I want to help out with LMMS development but I don't have much experience with application development at and would probably need a bit of guidance initially. Is that something someone would be able to do with me or do you strictly require experienced audio programmers?

[diizy]

hey diizy, I want to help out with LMMS development but I don't have much experience with application development at and would probably need a bit of guidance initially. Is that something someone would be able to do with me or do you strictly require experienced audio programmers?

Well, we can answer questions to the best of our ability, but a large part of it is also self-learning... programming isn't that hard though - start from small things, do some small and simple bugfixes, ask if you don't understand something and don't worry - none of us were born with programming knowledge. Hey, I still feel like I barely know what I'm doing half the time...

[Eino]

I would like to help out, but my programming skills are very little to almost none.
But I did figure out a way to get lmms to record, My portable Grand piano, to sound like your sitting in front of it.
I split the keyboard it to two channels, above middle C and below middle C.
I'm a musician, not a programmer, But I would like to help out anyway I can.

[raboof]

First off I think it's awesome you're asking for help so publicly here!

As you're probably well aware we all have our pet projects that are often already taking up way too much time, so it's hard to find contributors - but I think it's fun, important and educational to try and collaborate as much as possible.

I just wanted to mention it's really important to treat your contributors well. This is hard - I feel I'm falling short in this respect in my own projects - but essential to build a more collaborative world.

I guess my feedback from 2010 is probably now outdated . I'll try and see if I can do anything useful tonight. (to be clear: I'm not criticizing you for not acting on that issue - we're all doing this in our spare time. It does however kind of show how important it is to respond to any contributors quickly and keep that momentum going...)

[diizy]

I would like to help out, but my programming skills are very little to almost none.
I'm a musician, not a programmer, But I would like to help out anyway I can.

Well, right now we mostly need developers. But all help is always welcome. You can report bugs, test new releases, or just make music and tell people you've made it with LMMS.

First off I think it's awesome you're asking for help so publicly here!

As you're probably well aware we all have our pet projects that are often already taking up way too much time, so it's hard to find contributors - but I think it's fun, important and educational to try and collaborate as much as possible.

Agreed. Any form of collaboration would be most welcome for the LMMS team.

I just wanted to mention it's really important to treat your contributors well. This is hard - I feel I'm falling short in this respect in my own projects - but essential to build a more collaborative world.

Yeah. We all do the best we can to create an open and collaborative environment.

I guess my feedback from 2010 is probably now outdated . I'll try and see if I can do anything useful tonight. (to be clear: I'm not criticizing you for not acting on that issue - we're all doing this in our spare time. It does however kind of show how important it is to respond to any contributors quickly and keep that momentum going...)

2010 was before my time with LMMS, so I can't really say for sure... from a cursory look, I think that might have been addressed in this commit: https://github.com/LMMS/lmms/commit/2b7 ... f80bbe96b0 but if you really want to be sure, you can look at the Jack backend here: https://github.com/LMMS/lmms/commits/st ... ioJack.cpp

[raboof]

One other thing where I think people who are experienced with Linux Audio could help LMMS would be Jack support. LMMS has a Jack backend, but I think there are some problems with the implementation... I've been trying to look at it myself but so far I can't find any obvious fault in it, and I don't know much about Jack anyway...

If anyone wants to take a look at our Jack backend code, maybe submit some patches or just point out if we're doing something seriously wrong with it, that would be very much appreciated and would help with getting LMMS a bit closer to working well together with other Linux audio applications.

I've looked into some stuff, unfortunately I don't have enough time to really propose any big fixes. I'll just drop some general feedback here. Would be interested to hear if any other devs here can chime in with addition insights/solutions/best practices.

The essence of a good JACK backend is that the JACK process callback should never block.

You're using a fifo buffer of size '1' between the main LMMS code and the JACK audio backend, which blocks reading when there's nothing yet to read and blocks writing when there's still something to read. This *can* help (there is a chance that data becomes available before the JACK deadline), but at a huge risk: if your application doesn't meet JACK's deadline, it might get booted altogether.

I'm not sure if you could eliminate this fifo buffer entirely. In any case, 'Mixer::renderNextBuffer' should be lightning-fast (whether it's called from the tread filling the fifo or directly from the JACK callback). And it's in this method where there appears to be a classical mistake. Before actually rendering the sound, you call:

Code: Select all

// methods needed by other threads to alter knob values, waveforms, etc
	void lock()
	{
		m_globalMutex.lock();
	}

This could cause a lot of trouble. I'd take a good look at which processes take this lock (I haven't had a chance to). If any of those are doing anything non-trivial (like simple assignments), at least fix that. However, there could be more complicated problems lurking. The 'proper' fix would be to do this entirely without locking and using atomic writes and memory barriers and all that jazz, which is *really* hard to get right. You could cheat by using a lock, but you could run into priority inversion problems. If it's posix mutexes those could perhaps be mitigated by using priority inheritance, but you're using QT locks here, I'm not so sure those are implemented as posix locks...

I guess the bottom line here is there's plenty going on that's illegal in principle, but it's hard to say which of these are likely to result in actual real problems. Not sure if that's helping much ...

[diizy]

I've looked into some stuff, unfortunately I don't have enough time to really propose any big fixes. I'll just drop some general feedback here. Would be interested to hear if any other devs here can chime in with addition insights/solutions/best practices.

The essence of a good JACK backend is that the JACK process callback should never block.

You're using a fifo buffer of size '1' between the main LMMS code and the JACK audio backend, which blocks reading when there's nothing yet to read and blocks writing when there's still something to read. This *can* help (there is a chance that data becomes available before the JACK deadline), but at a huge risk: if your application doesn't meet JACK's deadline, it might get booted altogether.

Ok, that makes sense.

I'm not sure if you could eliminate this fifo buffer entirely. In any case, 'Mixer::renderNextBuffer' should be lightning-fast (whether it's called from the tread filling the fifo or directly from the JACK callback). And it's in this method where there appears to be a classical mistake. Before actually rendering the sound, you call:
...
This could cause a lot of trouble. I'd take a good look at which processes take this lock (I haven't had a chance to).

I think the mixer lock is mostly used by things that have to modify some parameter, eg. GUI-thread events that want to modify a knob value or some other parameter of an effect, or something that could lead to errors if it were modified mid-period. But there's also some other stuff like MIDI-event processing... I guess that may be something we should handle differently.

In any case, these are known deficiencies... Paul Giblock had at one point planned to rewrite the entire LMMS core in an RT-safe way, but then I guess he got too busy with rl and never could follow through with that, and now we're severely lacking in developers who really understand RT-coding or thread optimizations... so we just kind of have to do what we can.

The 'proper' fix would be to do this entirely without locking and using atomic writes and memory barriers and all that jazz, which is *really* hard to get right. You could cheat by using a lock, but you could run into priority inversion problems. If it's posix mutexes those could perhaps be mitigated by using priority inheritance, but you're using QT locks here, I'm not so sure those are implemented as posix locks...

I'm not sure, I'll have to check that in Qt docs...

But speaking of atomic writes. We have a function InstrumentTrack::processInEvent() which processes all incoming MIDI-events for a given instrument track, and this entire function locks the mixer mutex, presumably the only reason it does so is to prevent race conditions with the vector that keeps track of active MIDI-notes. This vector is a simple c++ array of pointers to noteplayhandles, so I'm wondering if it'd be possible to use atomic pointers here and do away with the mutex?

I'm going to give it a shot and see if it works that way...

[diizy]

I'm going to give it a shot and see if it works that way...

I did this and got it working otherwise, except that now it crashes for some reason when chaining instruments together via MIDI... hm.

[diizy]

Actually, Qt's atomic pointers don't seem very suitable for this task, so I'm instead going to try to just use a local mutex in the MIDI event processing function, so that the global mixer mutex doesn't have to be touched.

[diizy]

Ok, the crashing with chained MIDI was actually unrelated and a separate bug, so now I've been able to eliminate some global mutex locks from LMMS. It seems there are a few that are simply pointlessly locked, or maybe as a precaution of some sort, where removing the locks doesn't seem to make any difference...

[diizy]

Hypothetically, if we were able to pay for X work hours to a developer to work on improving the core engine of LMMS... would anyone be interested in such a project?

[fundamental]

Hypothetically, if we were able to pay for X work hours to a developer to work on improving the core engine of LMMS... would anyone be interested in such a project?

I'd image that if a reasonable pay/hours was setup there are people out there who would work as a consultant for the project.

[diizy]

Hypothetically, if we were able to pay for X work hours to a developer to work on improving the core engine of LMMS... would anyone be interested in such a project?

I'd image that if a reasonable pay/hours was setup there are people out there who would work as a consultant for the project.

We'd need a bit more than consulting... it'd have to be someone who's well-versed in RT-audio, DSP- and systems coding. We have a halfway-finished engine developed by Paul Giblock which could be used as basis for the work. The job would involve developing the new audio engine in an RT-safe way, then integrating that into the current LMMS codebase.

This is, again, all just hypothetical at this point. We're looking at options for possibly raising funds for an endeavour such as this (possibly with a crowdfunding campaign). So I wanted to gauge if there's anyone who'd be interested in such work.

[diizy]

Latest developments in the LMMS saga:

I'm currently working on a memory manager for LMMS, which should somewhat improve RT-safety by cutting down heap allocations. It's still a long way to go to make LMMS RT-safe, but it's a step forward.


On another note: FalkTX, what's the status on the LMMS/Carla plugin? Are you still working on it? I'd love to see it in LMMS 1.2...

[diizy]

On another note: FalkTX, what's the status on the LMMS/Carla plugin? Are you still working on it? I'd love to see it in LMMS 1.2...

How is 1.1 coming?

We're at the final RC... I think we can do a release very soon - there may be small issues but nothing that would be a showstopper and couldn't be left to a later bugfix...

I haven't done anything on carla-lmms since I posted the picture, but Carla2 is now at beta2 and a lot more stable.
One thing that would really help me (and I guess other devs) was having a null plugin that did nothing, maybe prints midi notes.

I've considered the same thing - writing a well-documented example/reference plugin. The problem is, that the API is sort of a messy hodgepodge of different kinds of instruments - we have multistream instruments, which render each note in their own thread, then we have two kinds of single-stream instruments, ones using MIDIevents and ones using the native note handles... so it's kind of hard to write a single example plugin that would show all that functionality. I'm hoping we could simplify the instrument APIs in the future...

Will the API change much from 1.1 to 1.2?

For now, I don't think so - currently, the changes to the API mostly only affect multistream instruments. However, if you have any problems rebasing to master, or any other questions about the codebase or API, feel free to ask me, I should be able to help with that...