ALSA, JACK and other approaches (e.g. Windows Audio System)

[raboof]

I appreciate the situation you're in, raboof, but I have to say I'd rather have the thread deleted entirely than have the posts be redacted. When people can still browse these forums and find (both) Jeffs' misleading and incorrect statements left apparently uncontested

By all means contest them! I'm sure we can have an on-topic discussion without the ad hominems.

[j_e_f_f_g]

Where it counts, however, which is for the audio buffers, JACK uses shared memory (shm/mmap), right? So I don't really see the problem here.

No, the whole design of the system contributes to its total latency. Because jack's engine runs as a separate process than all the apps connected to it, that adds overhead (ie, latency) due to dealing with virtual address spaces, task switching, and inter-process communication (ipc). Let's contrast how jack does it, versus how a non-jack app could do it. (For the latter example, I'll choose an app called "eDrummer" since I'm sort of familiar with it).

1) JACK apps (clients) open a "connection" with which to communicate with the daemon. They do this by calling a function jack_client_open in jack's userspace lib. (ie jacklib runs in the app address space, and the daemon outside). jack_client_open allocs some mem (primarily a jack_client_t struct) to keep track of "jack stuff", and opens a shared (ie AF_UNIX) socket that the daemon created. It's via writing/reading this socket that apps communicate with the server, and vice versa. (That's the IPC mechanism). If the socket opens, the app knows the daemon (jack server) is running. jack_client_open does a series of writes to this socket to give the daemon info, and reads to get info from the daemon. For example, writing a jack "ClientExternal request" tells the daemon "here's another app". With that the daemon creates a JSList struct, fills it in with info about the app, and links it into a list of JSLists from all the other connected apps. The daemon needs this list so it can coordinate i/o, for example going through the list of "input apps" for each alsa out buffer fill.

2) The app calls various other jack lib functions to create "ports" (ie tells the daemon which apps are supplying audio data, and which apps want audio data given to them for processing), and to register callback functions (ie tells jacklib which app function to call whenever the server tells jacklib to do something). Basically what jacklib functions like jack_port_register, jack_set_process_callback, etc, do is allow jacklib to store the app's information in that jack_client_t struct. The daemon also creates numerous shared memory areas for the various ports (of every app). This is not any card's hardware buffer. The daemon needs all these separate buffers to sum various apps' input data, figure out which buffers (output ports) go to which input ports... well, we get into this mess later. The daemon writes the shared mem details to its socket so alsalib can store the app's jack buffer addresses.

3) Finally the app calls jack_activate. jacklib starts up a thread (running in app's space) which polls on that socket, waiting for the daemon to give it instructions.

4) The jack daemon sets up ALSA audio output (and input) using MMAP, does a bunch of details I'll skip over (which edrummer does as well. Consult Audio.c for details. I comment my sources profusely). Eventually it calls a function to wait for alsa to say "I need some audio to output to this card" (or "I got some audio input for you". We'll use the audio out scenario for our following example).

Mind you, nothing so far affects playback latency, because playback hasn't begun. All the above does is use up RAM/CPU and mostly gives you dozens of points of failure above and beyond ALSA such that hardly a day goes by someone doesn't post "I can't get jack working. Why did I ever leave Windows?". But you need the above to understand what comes next. Playback begins. DUN DUN DDUNNN!

5) Alsa wakes the daemon and says "this card need audio to play. Here's its buffer to fill."

"Wait! I've got to get that data from the apps supplying it.", says JACK. The daemon goes through its list of JSLists, looking for apps with jack output ports. It finds the first app, and calls an os function to wake that app's alsalib thread. Remember that the demon itself is userspace, so every os call it makes is a ring 0 transition -- costly operation. The alsalib thread wakes from its call to poll. Another ring 0 transition... and... Thread Switch! No wait. That thread is running in a different process... Process Switch! This is costly, and all the while alsa wants its data NOW! Now the daemon can't do anything until it gets that data, so it calls an os function to wait for the app to write "I'm done" (not literally) to the shared socket. Ring 0! The jacklib thread calls the app's process callback which fills the buffer, and returns. jacklib thread calls an os function to write "I'm done" (Ring 0!) and then poll (Ring 0!) to await more instructions from the daemon. The daemon returns from poll (Ring 0! Process switch!) and read()s (Ring 0!) the "ack".

"Finally. You got my data?", asks alsa.

"Wait! That's the data from just one out port. There may be more out ports in my list!", cries jack. jack continues going through those JSLists, and for each out port, it does the above. Ring 0! Process switch! Process switch! Etc! Finally, jack ends up with a bunch of intermediate buffers of data.

"Done?", scowls alsa.

"No, I've got to mix (sum) all these buffers!", sobs jack.

"Well hurry up and mix them to the card's buffer.", growls alsa.

"No I can't!", weeps jack. "Some of these buffers (out ports) are connected to jack input ports. That means I have to mix these intermediate buffers down to another set of intermediate buffers to be sent off to apps that want to process the data further. Those apps will process the data into their shared mem buffers. When all that's done, then I can mix those buffers to the card's buffer. So excuse me while I go through my list again, just like I did with the out ports, and repeat it with each in port."

"You're joking!", gasps alsa.

"No, I was actually designed this way!", screams jack above the din of Ring 0, Process switch, Process switch, Ring 0!

"Why??", exclaims alsa.

"So you can take a youtube mp3 quality audio and add some nasty aliased filter to it in realtime!", boasts jack, who having now gathered his second set of intermediate buffers, furiously mixes them into the card's buffer, and nods he's done.

"Thanks.", sighs alsa. "Oh and while you were d**king around, I had 12 xruns. Better increase your latency buffer another 512 frames".

==================================

Now let's look at how that non-jack app (we'll call "ed") does things:

1) ed sets up ALSA audio output using MMAP, does a bunch of details I'll skip over, and calls a function to wait for alsa to say "I need some audio to output to this card".

2) Alsa wakes ed and says "this card need audio to play. Here's its buffer to fill.". ed directly mixes his waveforms into that buffer. Done.

=================================

Do I really need to explain how someone went from a jack app with 10 ms latency, to ed with a 4 frame buffer (surprised me) and 2 ms latency?

I stand by my recommendations to shebang, even though he's probably given up on linux a whole 24 hours after his first visit here.

[j_e_f_f_g]

JACK doesn't add any latency.

c'mon jeremy. You know that all those ring transitions, buffer copying, list searching, socket reading/writing, etc, and especially process switching, add latency versus not doing any of that. It would be metaphysically impossible otherwise. Those are a lot of CPU cycles eaten by the jack server (and clients).

It's not a question of if jack adds latency. It's a question of "how much?".

But if anyone can figure out how to do all this without any latency, there's a senior level engineering position available for him at intel.

JACK doesn't maintain any connections or sessions.

Jeremy??? What do you think that linked list of JSList structs (stored in the "client" field of the server's jack_engine_t struct) is for?

It does mixing but the mixing being done is really minimal, JACK actually only adds streams together.

Right, that's mixing -- you sum each individual sample point of the waves to be mixed. Like how edrummer's mixPlayingVoices() sums all the "playing" drum waves directly into the card's buffer. Except jack doesn't do that. It needs to mix the various out (port) buffers to intermediate buffers that get sent off to various in ports, and then mixes the returned in buffers to the card's buffer. More mixing. More overhead. More latency.

[autostatic]

c'mon jeremy. You know that all those ring transitions, buffer copying, list searching, socket reading/writing, etc, and especially process switching, add latency versus not doing any of that. It would be metaphysically impossible otherwise. Those are a lot of CPU cycles eaten by the jack server (and clients).

It's not a question of if jack adds latency. It's a question of "how much?".

Does using JACK add latency?

Jeremy??? What do you think that linked list of JSList structs (stored in the "client" field of the server's jack_engine_t struct) is for?

No idea, I'm unfamiliar with the JACK codebase, I'm just an end user and I thought JACK didn't maintain any sessions or connections, I thought you needed external tools to do this.

Right, that's mixing -- you sum each individual sample point of the waves to be mixed. Like how edrummer's mixPlayingVoices() sums all the "playing" drum waves directly into the card's buffer. Except jack doesn't do that. It needs to mix the various out (port) buffers to intermediate buffers that get sent off to various in ports, and then mixes the returned in buffers to the card's buffer. More mixing. More overhead. More latency.

Like I said, I'm just an end user, I have no exact idea how JACK is dealing with the buffers you mention. If you say it adds more latency and the original author of JACK states on his site that it does not then I tend to believe the latter.

[j_e_f_f_g]

My feeling is that you don't care though. But please prove me wrong.

Really? Well as long as we're impuning and second-guessing the motivations of others, here's my "feeling".

I think the technical discussion went way over your head (even though I tried classic "dumbing down" techniques such as making it conversational). In fact, i'd be willing to bet you didn't even read much of it before summarily and derogatorily dismissing it as a "rant" (how would like if I took to callin KxStudio "bitrot"?), and indeed had and have no interest in understanding anything about it because you (at most) know it's critical of jack (and you don't like that). If you actually wanted to understand it, you would have stayed on-topic (note the topic is a tech discussion about jack, not "what has jeffg done to help JACK?"). You would have quoted particular points you didn't understand, and asked for clarification... if you cared about the topic. You would have cited particular points you disagreed with, and offered coherent rebuttal... if you cared about the topic. But my feeling is you don't care. Why? Because you instead engaged in a classic (and surprisingly transparent) evasion of "I don't like the message, so I'm going to try to shoot the messenger". Here's how this cheap ploy works:

1) Because i can't offer any on-topic rebuttal to the message, I ignore every bit of it. I dismiss it summarily. I call it pointless, or misinformation, or whatever. My proof for this claim? Mere assertion. (ie, It's pointless because I said it's pointless). I cannot cite any specific content of the message to support my allegation, because my allegation is contrived. If anyone calls me on my lazy, underhanded ploy, I'll just try to blame my own lack of initiative on the messenger somehow. That's all I care about anyway -- getting attention away from the message and onto the messenger.

2) Immediately go off-topic and change it to a discussion about the messenger. Hope for, and encourage, everyone else who wants to shoot the messenger to join in likewise. Make it personal, and make it the biggest off-topic mess possible. Try to bury the message in a multitude of messenger bashing so that it "disappears", even if it's not on a forum where the moderation is so heavy-handed that, instead of telling the messenger bashers to grow a pair and learn how to effectively dissent, the moderators make it all disappear, thus giving the bashers exactly what they wanted, and teaching them the way to get it there is to off-topic attack the messenger if you don't like the message. If it worked once, twice, thrice, why not four, five, six etc times?

You'll notice no answers to your transparent baiting. I'm way too smart to fall for the cheap parlour tricks. If you have some on-topic response, questions, rebuttal, etc to the content of my jack posts (ie the "message") then behave like someone who cares about the topic. Quote the part of my post to which you're responding, and actually respond to that content. If all you can do is baiting (ie every one of male's posts), then grow a pair or get the hell out of the thread.

Prove me wrong.

P.S. This much-needed, and clearly way overdue, message better not "disappear" (unless it goes into some "policy page" people are referred to whenever they start this "attack the messenger" nonsense).

[nedko]

shebang, the point of JACK is about open culture. If you are going to compare features, you should compare JACK to ReWire. We have plenty of all-in-one (monolithic) apps for Linux too. Some of them use ALSA, some allow inter-operation with other apps via JACK. The usefulness of JACK is that when you miss a feature in one app, you can use several apps together. As for the stability, Linux can be more stable and fine tuned than Windows. Because it is open source. If you are a user that can code software then this will probably give you some value. It definitively does for me. I switched to Linux because I was not able to tweak the annoying properties of the Windows apps I've been using back then. Now I can even tweak my OS kernel.

As other Linux Audio Developers pointed already, JACK is not perfect. If you want to implement all in one solution, then ALSA (or even OSS) is a viable choice. As for this Windows thing (WAS, new to me), don't you think it is off-topic in this forum?

[raboof]

Where it counts, however, which is for the audio buffers, JACK uses shared memory (shm/mmap), right? So I don't really see the problem here.

The daemon also creates numerous shared memory areas for the various ports (of every app). This is not any card's hardware buffer. The daemon needs all these separate buffers to sum various apps' input data

Aah you're right, you need separate buffers when you want to support mixing sound from various sources, which JACK wants to support. Makes sense, thanks.

that adds overhead (ie, latency)

"Thanks.", sighs alsa. "Oh and while you were d**king around, I had 12 xruns. Better increase your latency buffer another 512 frames".

I think these are key comments. You (quite dramatically) describe how JACK introduces overhead. This is no surprise to anyone: after all, JACK uses ALSA as a back-end, so *anything* it does will be overhead compared to talking directly to ALSA. Also, while unmistakably adding

However, 'overhead' and 'latency' are very different (though obviously related) things. If JACK is configured to use equivalent period sizes and buffer sizes compared to an ALSA application, it will have exactly the same latency.

So the real question here, where your second quote touches upon, is this one: is the overhead introduced by JACK (on a correctly configured system) so big that it forces you to use a larger buffer (because of missed deadlines aka xruns)? Or is the difference generally eclipsed by the processing needed to, well, do the actual audio processing (which you'll have to do regardless of whether you're doing it through JACK or through ALSA directly)? These are not rhetorical questions, they need to be backed up with data for the use case you're interested in.

due to dealing with virtual address spaces

I'm still confused what the performance impact of virtual address spaces would be.

Do I really need to explain how someone went from a jack app with 10 ms latency, to ed with a 4 frame buffer (surprised me) and 2 ms latency?

It's not clear to me whether this difference is caused by the JACK overhead, or by the fact that perhaps ed was simply coded more efficiently compared to the JACK app tested.

[raboof]

a forum where the moderation is so heavy-handed that, instead of telling the messenger bashers to grow a pair and learn how to effectively dissent, the moderators make it all disappear, thus giving the bashers exactly what they wanted, and teaching them the way to get it there is to off-topic attack the messenger if you don't like the message.

Even though I think this is also off-topic, as this is basically directed at me personally, I felt the urge to respond.

Yes, I have heavily moderated this thread. If you look closely, I have also put a considerable amount of work into making sure the message did not disappear along with the ad hominem attacks going back and forth. So, no, I don't feel I gave the bashers what they wanted or taught them ad hominems serve any purpose.

It would have been much easier to remove/lock the entire topic and/or ban the posters causing the most trouble - in fact, people asked me to. I didn't. Rejoice.

This much-needed, and clearly way overdue, message better not "disappear"

I totally reserve the right to make anything I don't like disappear (including the message I'm responding to now, which is 'on the line'). I honestly believe it makes this forum a more fun and useful place.

[raboof]

c'mon jeremy. You know that all those ring transitions, buffer copying, list searching, socket reading/writing, etc, and especially process switching, add latency versus not doing any of that. It would be metaphysically impossible otherwise. Those are a lot of CPU cycles eaten by the jack server (and clients).

It's not a question of if jack adds latency. It's a question of "how much?".

(...)

Right, that's mixing -- you sum each individual sample point of the waves to be mixed. Like how edrummer's mixPlayingVoices() sums all the "playing" drum waves directly into the card's buffer. Except jack doesn't do that. It needs to mix the various out (port) buffers to intermediate buffers that get sent off to various in ports, and then mixes the returned in buffers to the card's buffer. More mixing. More overhead. More latency.

Does using JACK add latency?. Like I said, I'm just an end user, I have no exact idea how JACK is dealing with the buffers you mention. If you say it adds more latency and the original author of JACK states on his site that it does not then I tend to believe the latter.

It's a bit of a terminology game. When running with consistent buffer sizes and everything, JACK and ALSA have exactly the same latencies. However, because JACK introduces some overhead, an otherwise equivalent application might cause xruns when ran through JACK while it runs fine when ran directly into ALSA.

Whether this is just a theoretical possibility or a real-world problem is another question, though.

Jeremy??? What do you think that linked list of JSList structs (stored in the "client" field of the server's jack_engine_t struct) is for?

No idea, I'm unfamiliar with the JACK codebase, I'm just an end user and I thought JACK didn't maintain any sessions or connections, I thought you needed external tools to do this.

Again sort of a terminology issue here: of course JACK is aware of the currently active connections (that's how it knows which audio to put where). However, to intelligently manipulate those connections (i.e. saving/restoring them across JACK server or even machine restarts), you need a JACKlib-based application to handle that.

[j_e_f_f_g]

The web page Jeremy cites is so brief in its content that I'm going to quote in its entirety below:

There is NO extra latency caused by using JACK for audio input and output. When we say none, we mean absolutely zero. The only impact of using JACK is a slight increase in the amount of work done by the CPU to process a given chunk of audio, which means that in theory you could not get 100% of the processing power that you might get it if your application(s) used ALSA or CoreAudio directly. However, given that the difference is less than 1%, and that your system will be unstable before you get close to 80% of the theoretical processing power, the effect is completely disregardable.

That's it??? This is how he explains away all that jack overhead I detailed?

Nils, you ever think to ask this guy to "deliver data and measurements in a fashion that anyone else can run the same experiment" with jack, alsa, and coreaudio? Because if that's important to you (and it must be because you demanded it from me), maybe you should demand he show evidence to prove his claims?

Jeremy??? What do you think that linked list of JSList structs (stored in the "client" field of the server's jack_engine_t struct) is for?

No idea, I'm unfamiliar with the JACK codebase, I'm just an end user and I though JACK didn't maintain any sessions or connections, I thought you needed external tools to do this.

That's cool, Jeremy. I appreciate that you're an enduser. (And an effective one. Those vids you made are examples of effective advocacy. Sure your your drum sound and features aren't going to wow the guy sitting at his v-drums connected to an i7 running Superior Drummer. But obviously the point was portability, convenience, and price. You focussed on the correct selling points. The v-guy pro could look at his large setup, and say "yeah that pii is portable". He could look at the cables pouring out of his v and say "yeah that pii sets up quick". He could look at his bill of sale and say "yeah that's cheap". And you didn't ruin it by saying "And this Linux JACK Session stuff, with its spaghetti maze of lines connecting a bunch of nebulously labeled, um, software things... is so much better than your Win/mac setup. Cuz' it lets me take a youtube audio fart and add cheap effects to it". Because then the v-guy would sneer in ridicule and forget the selling points. Would that others knew how to advocate instead of some of the crap I've seen pulled).

I appreciate that you're an enduser. And I really try to explain things in layman's terms, so let me take a shot here at explaining the "session manager" stuff. The server needs to know who is connected to it, who is going to provide data to it (let's say a soft synth), and who wants a shot at modifying that data before it goes to the sound card (let's say an EQ effect). So the synth calls jack_client_new to tell the server "I supply data. Add me to your list.". And the EQ calls jack_client_new to tell the server "I can modify that data. Add me to your list.". And maybe a reverb calls jack_client_new to tell the server "I can modify that data too. Add me to your list.". So the server thinks "I can take the synth (out port) and route it to the EQ's in port. Then route the EQ's out port to the reverb's in port. Then route the reverb's out port to the sound card. There. Now the synth goes to the eq then the reverb then the sound card."

But wait. Maybe you want the synth to go directly to the reverb, and you don't want the eq in there at all. Or maybe you want the reverb first, eq second. Wouldn't it be nice to have a tool that asks the server for the names of all the in and ports in that JSList? Then the tool displays all the ins on the right, and outs on the left, and lets you draw lines between them how you want them connected. And the tool gives this info back to server so the server can set that routing. A tool like qjackctl? Yes these external tools exist solely for your benefit -- to let you influence the server's routings. The server still maintains the connections. (it has to). The external tool simply "gives you a say" in the routing. Oh yes, and it introduces more points of failure.

If this ungodly mess weren't enough, people started inventing "session managers". All these things do is record the information you would manually enter into something like qjackctl, like some glorified keylogger, and automatically spit it back at jack under certain conditions. Oh yes, and they introduce more points of failure. Some of the more intrusive ones even have abstraction layers that do things like intercept data an app is trying to send to alsa, and reroute it to jack. They usually call these things "bridges" or "emulation". hey let's take a nice low latency alsa app and give it the latency of jack. And yes, this introduces more points of failure (with bridges being especially problematic as they can really undermine an app's operation.

Stop this madness! Kill this three-headed monstrosity, and replace it with what is really needed -- a well-designed integrated plugin system using shared libraries (ie, LV2... well better designed than LV2. Turtle?? c'mon!).

Uninstall JACK. Unistall Pulse Audio. Get rid of all "session managers" and "bridges". Get rid of any app that can't use alsa directly. Use alsa. Fix alsa. Extend app functionality will LV2 plugins, not via qjackctl.

Then and only then will we see performance and usability comparable to WASAPI, CoreAudio, ASIO, etc.

If you say it adds more latency and the original author of JACK states on his site that it does not then I tend to believe the latter.

I dragged you kicking and screaming through jack, detailing every point of latency in excruciating detail. Was there something in particular I got wrong, Jeremy, which would account for a total repudiation of my analysis? Upon what basis do you side with the original author -- because he posted a paragraph that says only "There's no latency"? Surely you wouldn't expect him to post something like this about his own software (although if it were me I'd do it for laughs... cuz that's me):

There is NO extra latency caused by using JACK for audio input and output. When we say none, we mean absolutely some. A bit actually. Ok, there's more latency than the amount of dirty magazines dad hides in the garage so mom won't find them. The only impact of using JACK is a slight increase in the amount of work done by the enduser just to get this thing even running, let alone without xruns. "In theory"[\i] you could not get 100% of the processing power that you might get it if your application(s) used ALSA or CoreAudio directly. "In theory"[\i]. But in practice, you will. However, given that the difference is less than 1% (of what I don't know), and that your system will be unstable before you get close to 80% of the theoretical processing power (i'm totally guessing), the effect of using JACK is completely disagreeable -- especially if you previously used Windows or a mac or... hell even an Atari ST.

[ssj71]

As I understand, Jeff, your point is ALSA has some issues and should be replaced or fixed (which is difficult due to the poor documentation), and JACK is adding a broken front-end on top of a broken back-end. But how would you recommend app->app audio be mixed such as JACK allows? If I want edrummer to send drums to an FX program is this possible using ALSA and if not, how should this be architectured? A change in ALSA or something added on top?

[tux99]

Uninstall JACK. Unistall Pulse Audio. Get rid of all "session managers" and "bridges". Get rid of any app that can't use alsa directly. Use alsa. Fix alsa. Extend app functionality will LV2 plugins, not via qjackctl.

Why, just why..?

Top be fair he does have a good point, a plugin based model is a lot more user-friendly than having numerous separate apps all glued together by Jack and session managers.
And that's even the case if you completely disregard latency.

[j_e_f_f_g]

you don't seem to care about helping JACK development.

Of course not! Did you not read anything I wrote?? Let me repeat:

For all the problems JACK introduces (and I've been detailing) -- increased latency, convoluted configuration, more points of failure, the fact that it sits on top of alsa rather than replaces it (so it inherits all the underlying limitations/problems, and can't do anything about it), the fact that it's yet another alsa abstraction layer and there are already too many of them such that they're killing any chance of there ever being a unified audio api (alsa) being universally used (and thereby actually decreasing app interoperability/compatibility) -- I don't want to help it. I want it to cease to exist. I want alsa (or some other kernel level audio system) to rise from the tarpit that currently is linux audio and save us from the misguided, poor compromises that are currently keeping linux audio from matching what the competition is doing and planning.

Why would I want to help something which I, in utmost earnest, believe is a major detriment to linux audio?

I want to dance upon jack's (and pulse audio's, and port audio's and every other cheesy abstraction layer's) grave.

I lost interest in this conversation anyway...

When did you start to show interest?

[tux99]

Sure, but in there you assume we all want monolithic, all-in-one applications, which is not the case for some linux audio users including myself.

There is nothing monolithic about using a base application (sequencer or DAW) where you then can add softsynths and effects via plugins, in fact it's exactly the same thing as using a base application and then adding softsynths and effects via Jack routing, only a lot more user-friendly!

Don't forget that some plugins actually introduce latency, reason why some hosts have something called PDC (plugin delay compensation).

In that case they would also introduce latency when used via Jack.

[tux99]

I want alsa (or some other kernel level audio system) to rise from the tarpit that currently is linux audio and save us from the misguided, poor comprises that are currently keeping linux audio from matching what the competition is doing and planning.

Well, then you could start collaborating with the ALSA devs helping them to make ALSA better and hopefully better documented too.
From what I can see you do have the skills for that so why not contribute to ALSA?

I'm very much in favour of getting ALSA improved and therefore make all other audio daemons built on top of it redundant, but unfortunately I don't have the skills to do that so all I can do is hope that people like yourself take the initiative.