A word of warning

[j_e_f_f_g]

Some of you may not be aware that the MIDI Manufacturer's Association (MMA), who is in charge of managing the MIDI specification, has finally ratified the long-awaited update to MIDI. The initial specification, called MIDI 1.0 was adopted in 1985. The new update is called MIDI 2.0.

Tashi... Tasch... Itchy has started (and completed some, but not all of) ALSA's MIDI 2.0 implementation. Without getting into details about the differences betwee 1.0 and 2.0 (I've got a lot of studying to do myself), the differences are dramatic enough to necessitate updates to alsa drivers, and no doubt data structures used internally by alsa.

The alsa library that application code is supposed to use should protect existing MIDI programs from breaking due to the alsa updates. (ie, Those programs will continue to use MIDI 1.0 without problems. Of course, those programs will need to be updated if they are going to use MIDI 2.0's new features). It is the dictate of Linux Torvalds that updates to the operating system should not break apps.... as long as those apps use "the official way to interact with the OS". But apps that bypass the OS are fair game to break.

Jack2 bypasses alsa's official "API". It directly interacts with audio drivers (without alsa's involvement). And it creates and modifies internal data structures that no app is supposed to ever touch. Because of this, there's a real possibility that jack2's MIDI features (and any emulation based upon it -- Pipewire?) will break when the new alsa update arrives. (Could be in the next linux final release).

This is a heads-up. If your MIDI software no longer works as soon as you update your kernel, this is a possible cause that you should definitely consider. Also, consider this a warning that, making a backup prior to updating your system, is a good idea for the next alsa update. The MIDI part of alsa is undergoing a complex update.

What are just a few of MIDI 2.0's features? For one thing, the 16 channel limit per MIDI port/vst is raised to 255 channels. MIDI 2.0 also increases bandwidth significantly, so latency and jitter can be expected to be lower. Midi devices can communicate their patches by name, not just numbers. So for example, there will be no need to make any "nam" files that tell an app the patch name for each number. An app will be able to directly ask any MIDI device for that information.

[novalix]

cough
https://github.com/jackaudio/jack2/issu ... -602761123

[tseaver]

@j_e_f_f_g

Jack2 bypasses alsa's official "API". It directly interacts with audio drivers (without alsa's involvement). And it creates and modifies internal data structures that no app is supposed to ever touch.

Hmm, I'm not contradicting you, because I'm unsure, but I'd like to see where you got this info: I have read for years that JACK functioned as a layer "on top of" ALSA. E.g., see this discussion on the Ardour discourse, where Paul Davis and Robin Gareus appear (to me) to make that same claim: I might misread them, but I'm confident they know more than I about the relationship between JACK and ALSA.

[j_e_f_f_g]

link above

I don't know what the person on that mailing list is testing. From what little context is provided, he appears to be making some hardware that uses MIDI 2.0, and connects to the computer via an ethernet port. I don't know of any standardized protocol for midi-over-ethernet except for RTP-MIDI (ie, Apple's midi-over-ethernet scheme for CoreAudio). The poster mentions RTP-MIDI, but doesn't provide any details about what software he's actually testing. (He could be testing jack1, set to use alsa rawmidi api, for example). I don't know what he means by "passing UMP packets". To where/what? His post is dated 2020, which precedes the alsa MIDI 2.0 implementation. So he couldn't possibly be testing whether alsa's MIDI 2.0 update breaks jack, nor even testing a working 2.0 implementation. It sounds like his hardware is simply tossing MIDI 2.0 messages at alsa, and not actually doing anything with those messages except noting that they don't cause alsa (pre-2.0 support) to crash. Well, I would hope so. UMP packets are designed in such a way that MIDI 1.0 devices (and a MIDI 1.0 stack) should simply ignore them (2.0 messages look like "undefined messages" to MIDI 1.0 devices, and historically, all MIDI devices pre-2.0 are supposed to ignore undefined messages). But this in no way tests whether alsa's 2.0 update will break jack2 because jack2 touches components that it shouldn't, and therefore is not protected against breakage due to OS updates.

[j_e_f_f_g]

Jack2 bypasses alsa's official "API".

I'd like to see where you got this info

I got it from the only place you'll ever get important information like this (because linux developers tend not to document this stuff). And that place is directly from jack2's source code. I saw how it ripped lines of code right out of alsa's asound lib (ie the library functions that apps are supposed to use) so jack2 could directly call the ioctl function of audio drivers (which is supposed to be the job of asound lib), passing structures/values that should be known only by asound lib (which jack2 knows about because it links with asound's private header files, instead of the asound header file apps are supposed to link with).

I saw it all for myself. Download the jack2 source code and you can verify all this for yourself. Incidentally, this is the same way I learned a whole bunch of things about jack1 and alsa that wouldn't be known to anyone who didn't study the source code (because it isn't documented anywhere else). It's not that I have any particular "advanced ability to know things". I just sat down with the code, and took the time to learn what it does. Anyone can do it (well, given some knowledge of C/C++ programming).

And I recommend all linux audio programmers do so (because typically there's a lot of undocumented stuff in there).

JACK functions as a layer "on top of" ALSA.

That's true. Both jack1 and jack2 use parts of alsa to make sound. The difference is that jack1 uses only the parts that an app is supposed to use, whereas jack2 uses some parts it isn't supposed to use (and therefore is not protected from breaking due to os upgrades).

[novalix]

The poster mentions RTP-MIDI, but doesn't provide any details about what software he's actually testing. (He could be testing jack1, set to use alsa rawmidi api, for example).

Well, it is a discussion inside the issue tracker of jack2. Reading the whole thread one gets the impression that those people developing the software do care and closely observe the development on the alsa side.
If you have identified possible problems of the implementation of jack2 regarding the use of the alsa api, then there should be a dedicated bug report, where those issues can be discussed.

[j_e_f_f_g]

it is a discussion inside the issue tracker of jack2.

One would suppose then that it was some sort of jack2 test. But what he's using, what the purpose of his test is, and what conclusions, if any, one can draw about jack2 compatibility with a future (then non-existent) alsa update cannot be determined from that link.

Here's a more detailed explanation of what I "think" he's testing (but maybe not, because there's not enough info there to know for sure):

You can't have a situation where some midi 2.0 device (we'll call him simply "2") starts sending out 2.0 specific messages while some midi 1.0 device (call him "1") is "listening". That will totally confuse 1, and probably make him start creating sounds like "Gas Music From Jupiter". (Kudos to whomever gets that comedic reference. It's obscure). 2 needs to specifically ask if anyone connected to him understands MIDI 2.0.

Here's how it goes between 2 and 1:

2: Let me ask whomever is connected to me if he understands MIDI 2.0 specific stuff. According to MIDI 2.0, I should send out this particular sysex message that any MIDI 2.0 device will recognize me as asking "Do you speak 2.0?".

<2 sends the sysex message, and waits to receive back another sysex message that 2.0 defines as a response of "Yes I do.">

<1 gets the "Do you speak 2.0?" sysex message, and thinks:>

1: Hmmm. This is a sysex message, but I don't recognize it. It's not a "GM Reset" sysex message like 1.0 defines, It isn't a "Master Volume" sysex message also defined by 1.0. It isn't any of 1.0's sysex messages. I'm supposed to ignore a sysex message like this. So that's what I'll do.

<2 never gets a timely "Yes I do" response. So 2 concludes>

2: Ok, whomever is connected to me doesn't understand 2.0 messages. So I need to stick to 1.0 messages only with this guy.

That's what I think this guy was testing. His 2.0 hardware device tossed a "Do you speak 2.0" at ALSA (and Windows MIDI stack) and ALSA/Windows didn't crash, or play gas music from jupiter, or email your internet porn cache to your grandmother. Instead, alsa/windows blindly passed on that sysex message to a MIDI software program (ie, Reaper, QTractor, Cubase, Ardour. etc) so maybe that program can say "AHA! Someone is asking me if I speak 2.0! I do! Alsa/Windows, please pass back to that guy this here "Yes I do" sysex message for me, so we can do 2.0.". Or maybe the program is 1.0, in which case he won't respond to "Do you speak 2.0?" either.

That's what I think he's testing. But regardless of whether that's the case or not, he definitely isn't testing jack2 compatibility with an updated alsa.

those people developing the software do care and closely observe the development on the alsa side.

I don't know what you're saying here. There are lots of interpretations. I'm not sure if you're saying "The jack2 devs must have had their reasons for bypassing asound lib. And I assume those reasons are more important than risking potential breakage on alsa updates". Or maybe you mean something else.

You'll have to rephrase the above, and/or expound upon it, in order for me to know what you mean.

If you... identified... problems of... jack2, then <make> a dedicated bug report

There's no need to file any bug report here. I'm sure whomever is maintaining jack2 already knows that it bypasses asound lib, and understands the potential ramifications of that. If the dev doesn't, then you've really got problems.

[novalix]

I don't know what you're saying here. There are lots of interpretations.

Your broad claim of jack2 bypassing libasound is definitely wrong. That means it is not true.
Emitting a "warning" about a severe flaw of a specific software in a user centric forum, paired with the reluctance to discuss this on the development side seems like classic FUD to me.

I hope that was clear enough.

[merlyn]

Phoronix was a bit less doom and gloom in this article. We're due to get MIDI 2.0 support in kernel 6.5, which is due in late August/early September. It's not really too long to wait to find out if jeff is talking shit or not.

Oh, and it's Takashi Iwai. Not that difficult, is it?

[tseaver]

@j_e_f_f_g

I got it from the only place you'll ever get important information like this (because linux developers tend not to document this stuff). And that place is directly from jack2's source code. I saw how it ripped lines of code right out of alsa's asound lib (ie the library functions that apps are supposed to use) so jack2 could directly call the ioctl function of audio drivers (which is supposed to be the job of asound lib), passing structures/values that should be known only by asound lib (which jack2 knows about because it links with asound's private header files, instead of the asound header file apps are supposed to link with).

OK. Just to check the claim superfically, I'll look for inclusion of the "asound private header files" bit first:

Code: Select all

$ git clone https://github.com/jackaudio/jack2.git
Cloning into 'jack2'...
...
Resolving deltas: 100% (23945/23945), done.
$ cd jack2/
$ git grep "#include.*alsa"
linux/alsa/JackAlsaAdapter.h:#include <alsa/asoundlib.h>
linux/alsa/JackAlsaDriver.h:#include "alsa_driver.h"
linux/alsa/alsa_driver.c:#include "alsa_driver.h"
linux/alsa/alsa_driver.c:#include "alsa_midi_impl.h"
linux/alsa/alsa_driver.h:#include <alsa/asoundlib.h>
linux/alsa/alsa_driver.h:#include "alsa_midi.h"
linux/alsa/alsa_midi_impl.h:#include "alsa_midi.h"
linux/alsa/alsa_midi_jackmp.cpp:#include "alsa_midi_impl.h"
linux/alsa/alsa_rawmidi.c:#include <alsa/asoundlib.h>
linux/alsa/alsa_rawmidi.c:#include "alsa_midi_impl.h"
linux/alsa/alsa_seqmidi.c:#include <alsa/asoundlib.h>
linux/alsa/alsa_seqmidi.c:#include "alsa_midi_impl.h"
linux/alsa/generic_hw.c:#include "alsa_driver.h"
linux/alsa/hammerfall.c:#include "alsa_driver.h"
linux/alsa/hdsp.c:#include "alsa_driver.h"
linux/alsa/ice1712.c:#include "alsa_driver.h"
linux/alsa/usx2y.c:#include "alsa_driver.h"
linux/alsarawmidi/JackALSARawMidiDriver.cpp:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiDriver.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiPort.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiReceiveQueue.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiSendQueue.h:#include <alsa/asoundlib.h>

The includes wrapped in double quotes are for local headers (files in jack2/linux/alsa itself). All the others are just the main, public ALSA library header, 'alsa/asoundlib.h':

Code: Select all

$ git grep "#include.*alsa" | grep -v "\""
linux/alsa/JackAlsaAdapter.h:#include <alsa/asoundlib.h>
linux/alsa/alsa_driver.h:#include <alsa/asoundlib.h>
linux/alsa/alsa_rawmidi.c:#include <alsa/asoundlib.h>
linux/alsa/alsa_seqmidi.c:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiDriver.cpp:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiDriver.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiPort.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiReceiveQueue.h:#include <alsa/asoundlib.h>
linux/alsarawmidi/JackALSARawMidiSendQueue.h:#include <alsa/asoundlib.h>

Can you please point me to the location in the JACK2 source which includes "private" ALSA headers?

[tramp]

I'm running kernel 6.5.0-rc5-rt2 and jackdmp 1.9.21now and didn't noticed any issue with midi, not with jack, not with alsa and not with interact jack <-> alsa midi.

[glowrak guy]

edit:

1. Jack2 bypasses alsa's official "API"...

2. This is a heads-up. If your MIDI software no longer works as soon as you update your kernel, this is a possible cause that you should definitely consider. Also, consider this a warning that, making a backup prior to updating your system, is a good idea for the next alsa update. The MIDI part of alsa is undergoing a complex update.

3. What are just a few of MIDI 2.0's features? For one thing, the 16 channel limit per MIDI port/vst is raised to 255 channels.

In reverse order,

1. I can only think of 4 composing players using linux, who would directly benefit from 255 channels based on their reported workflow (and that's counting you among them, without confirmation). (I hope there are many more, and I can only name two of the four, and the other fellow might expand his horizons, but doesn't likely need to, from a progressive rock standpoint.)

2. Thanks for the heads-up. Midi 2 will only be tested here on a new install of some historically sound distro where it is verified (by the spearcatching travails of others) to work with both my old computer, and the the common existing tools and plugins contained therein, that I enjoy, and rely on.

3. I thought the whole idea of 'open source' was to allow deprecation of the old carved-in-stone 'official' rules, and to allow for fresh discovery, innovation, and improvement over once 'best practices', last years model, and the status quo. etc Maybe 'open' is a little too broad a term for core developers of linux?

But here's to the shifting winds of the future, may genious be victorious over the mobs