Alsa Midi port name ambiguity trouble

[Impostor]

Ideally, all 3 should. But in practical terms, your problem will be fixed with only the alsa bug being fixed. Once that happens, QJackCtl and Muse will work.

Okay, is the following an acceptable bug report for Alsa, or should I include more info:

*Issue with midi port naming*

The way Alsa is naming midi ports (starting somewhere between kernels 5.11 and 5.13) is prone to give issues in certain applications/hardware combinations.
As an example, take as hardware the midi keyboard M-Audio Keystation 49 MK3, and as software MusE Sequencer.

The Keystation offers two midi ports over a single usb cable. Both ports are now assigned the same name by Alsa (at least on kernels 5.13 and 5.15):

0:'Keystation 49 MK3 Keystation 49'
1:'Keystation 49 MK3 Keystation 49'

MusE apparently does not take into account port numbers, only the naming string, when scanning for midi devices. This leads to only port 1 being usable in MusE, but port 1 only carries commands from the keyboard's transport buttons, which makes it rather useless.

Given that there are plenty of Midi device manufacturers, and software application developers, and Alsa is the singular connection in between, this unwanted behaviour is best solved by adjusting midi port name assignment by Alsa, such that distinct ports -always- get distinct names.

[Linuxmusician01]

First of all, thanks for the clear explanation.

[...] because MacOS/Windows don't use it

I'm afraid that's where the Alsa developers way of thinking goes wrong then. As a company you have to go with the greatest common denominator. That's Windows and secondly Apple. Manufacturers will never do things the way Linux prefers it. I wouldn't if I were a business man.

for the 'M-Audio Keystation 49 Mk. 3' there aren't even drivers for Windows.

There's probably one 'maudio Windows usb midi' driver that supports their entire line of usb keyboards on Windows. This is how yamaha, casio, roland, etc do it.

Do you mean that there's a general driver on the M-Audio website or that Windows and MacOS have one standard "on board"? Because I can't find such a general driver on M-Audio's website...

[folderol]

As a matter of interest Rosegarden does manage and display both the port and the name along with the type.
e.g.
16:0 Hua Xing MIDI 1 (read)
14:0 Midi Through Port-0 (duplex)

[j_e_f_f_g]

You're giving a lot more info than they need, while omitting a pertinent point (that alsa's current query of USB endpoint names doesn't account for two or more endpoints having the same name).

I'd report it as so:

In kernel 5.13, alsa changed the way it obtained descriptive names (strings) for USB devices. In previous kernels, alsa constructed a name from the endpoint's index (number). For example, the first USB MIDI endpoint is named "MIDI 1", the second such endpoint is "MIDI 2", etc. This ensures each device has a unique name. In kernel 5.13, alsa now queries a device's USB controller chip for each endpoint's name, and uses any such provided name for alsa's device name. Unfortunately, alsa doesn't take into account that a USB device may provide the same name for one or more of its endpoints. The m-audio Keystation 49 USB keyboard is an example of such a device. By not checking for identical names, alsa now allows different devices to have the same name. This presents confusion when an app lists two supposedly distinct devices, but both with the same name. Some audio apps, such as Muse sequencer, rely only upon distinct names when saving their configuration. The net result is that such a config can be broken when multiple devices have identical names.

Therefore, alsa needs to do a strcmp() on the endpoint names, and if two or more are the same string, revert to using the old convention of simply naming the devices numerically.

Optionally, a "quirk rule" for the maudio keystation 49 may be added to alsa USB-AUDIO driver. This rule should apply the name "MIDI Piano keyboard" to the first MIDI endpoint, and the name "MIDI Transport buttons" to the second MIDI endpoint.

You'll want to run the lsusb command to locate the keystation 49's vendor id number (VID) and model id number (MID). Alsa devs will need those 2 numbers if they want to add that quirk rule. Each number should be a 4 digit hex value such as:

VID = 0030
MID = 0F1C

Of course, they will be different values than the above.

[Impostor]

16:0 Hua Xing MIDI 1 (read)

Hmm, and you use kernel 5.18? If your port gets 'MIDI 1' appended to the name, then maybe the issue has already been fixed after kernel 5.15...alas, kernel 5.15 is the newest I can readily install, so no way of testing.

[folderol]

Yes.
uname -a
Linux devuan 5.18.0-4-rt-amd64 #1 SMP PREEMPT_RT Debian 5.18.16-1 (2022-08-10) x86_64 GNU/Linux

[Impostor]

Just tried AVLinux 21 live usb, which has kernel 5.16, but there the issue persists. I also tried kernel 5.18 from ubuntu testing, but got errors both installing and booting. Did boot though, and could open terminal to run aconnect -i. Issue here is still present (but due to all those errors maybe not a reliable result).

[j_e_f_f_g]

If your port gets 'MIDI 1' appended to the name, then maybe the issue has already been fixed

Not really. It's more
likely that the MIDI endpoint for his "Hua Xing" device (Whatever that is... some cheap asian MIDI interface?) has the simple name of "MIDI 1".

[Tim E. Real]

Hi. Sorry I'm late to this. I have a few things to say.

These are the MusE settings in the nonworking config:

First, hey @Impostor, in your first image of MusE on the first page of this thread,
I see letters instead of icons in the Arranger Track List header columns like mute, record etc.
How is that possible? Are you using a custom stylesheet or something? Did someone change this
or put an option that I was not aware of? Or have I have completely forgotten some change occurred?
Thanks.

Second, about the ports.
Unfortunately MusE kind of relies on unique port names.
Two ports with the same name is bad for MusE, whether Jack ports or ALSA ports.
In usage, it may have trouble listing and connecting such ports.
In storage, the song files use port names, to automatically re-establish connections,
so they need to be unique.
And in usage, I use port names to automatically re-establish connections if a device
was not available at song load but then came on line, like a USB MIDI device was plugged in.

I call this persistent connections.

I tried very hard to use port numbers or identifiers.
Unfortunately those can change each time a device is plugged in.
I found only the port name is reliable to work with each time.

And with Jack I automatically search for the best possible port name to use since with Jack
there are up to two alias names that can be used. I found the first or second alias is best.
The default (normal non-alias) Jack port names were unusable because they contain a
number that changes each time the device is plugged in - they are not the same every time.
I have read it has been suggested that Jack remove these aliases in favour of the meta-data
now that it works. But I have argued against it. Still, as long as ultimately all the ports have
reliable, repeatable, unique names, it should be OK.

Third, I would like to know something important:
Your image of PianoTeq's port list shows both ports - but - if you choose one, does it properly
remember that selection the next time you start it, and not accidentally select the
other one?
Check carefully which one it really ends up sending data to...
If the device is USB MIDI, does PianoTeq properly remember which port you chose if the
device is unplugged and re-plugged?

I kinda feel your pain with this trouble, hope it can be worked out.
Thanks.
T.

[Impostor]

Hi, thanks for looking into it.

First, hey @Impostor, in your first image of MusE on the first page of this thread,
I see letters instead of icons in the Arranger Track List header columns like mute, record etc.
How is that possible? Are you using a custom stylesheet or something?

That's the ' deep ocean' theme, included with the appimage of MusE.

I tried very hard to use port numbers or identifiers.
Unfortunately those can change each time a device is plugged in.

The main device number does indeed change from time to time, depending on when I turn the device on, but in my very limited experience, my keyboard seems to be, by Alsa, consistently assigned subdevice port number 0 for the keyboard data, and port number 1 for the transport data.

The default (normal non-alias) Jack port names were unusable because they contain a
number that changes each time the device is plugged in - they are not the same every time.

True: the Jack port names (as reported in QJAckCtl under device "system") of my keyboard are named "midi_capture_x" and "midi_capture_y" where x and y vary each time I turn the device off and on again. Makes persistent connections in the Patchbay a bit of a pain to realize.
That 's mainly why I used Alsa Midi for MusE: up until the kernel upgrade past 5.11, these ports did have consistent subdevice numbers (0 and 1), and distinct names. The port numbers 0 and 1 are still consistently assigned to keyboard data and transport data respectively, but the naming protocol has seemingly changed, alas, in my specific case for the worse.

Your image of PianoTeq's port list shows both ports - but - if you choose one, does it properly
remember that selection the next time you start it, and not accidentally select the
other one?
Check carefully which one it really ends up sending data to...

Yes, I can select either one, and that one is remembered when restarting Pianoteq. Also the upper port is always the one carrying the keyboard midi data, and the lower port, with (2) appended, is always the one carrying data from the transport buttons of the keyboard. This doesn't change with restarting the midi keyboard.

I kinda feel your pain with this trouble, hope it can be worked out.

Not so much a pain for me personally: simply routing both ports via 'midi through' to MusE works fine. I don't use the transport for anything, so merging ports 0 and 1 in this way makes no difference to me, in any practical sense.

At least, until the Alsa devs deprecate midi through..

But I can imagine there are people with hardware who actually use the multiple ports of their single device, who may run into the same problem someday. So I think it's best the Alsa devs solve it. But I'm not yet ready to make an account there just to be able to file a bug report. I'm rather conservative with doling out personal data.

[Tim E. Real]

Oops. The Deep Ocean theme. I forgot @kybos added that. My bad.

My 'persistent connections' scheme aims to ensure that hot-plug devices
are properly re-discovered and re-matched with the MusE ports they had
been assigned to.

Alas, it's not perfect. I suspect for example that if two identical products
are plugged in, it might have trouble if the port names are the same,
but I was unable to test this, not having multiple devices here.

I concluded that what I really need is an absolute identifier of a particular device
that I can keep track of, some kind of serial number or UUID.
But that isn't available through normal programmatic means.

But... I just read this great article that describes exactly the kind of problem
I faced and how to solve it - with pre-2.6 kernels or without UDEV:
https://alsa.opensrc.org/MultipleUSBAudioDevices

Which leads to this page.
UDEV to the rescue?
It says that UDEV can solve all of these problems:
https://alsa.opensrc.org/Udev

Maybe something there can help you permanently rename your devices upstream?

As you can see it's technical and involved, not something the average user,
including me, would want to do.
If I could just use some of that info within the program...

[Impostor]

It says that UDEV can solve all of these problems:
https://alsa.opensrc.org/Udev
Maybe something there can help you permanently rename your devices upstream?

I'll wait until midi through is deprecated before I'll attempt that:)

By the way, I've looked a bit more into how Pianoteq saves its midi setting:

If I start my midi keyboard after booting, it gets device number 24. If I boot with the keyboard active, it gets device number 20.

Say, in Pianoteq's config file, the last config is saved as

<device name="Keystation 49 MK3 Keystation 49" id="24-0"/>,

because I always turn on the keyboard after booting. But for once I start the keyboard before booting, so its device number becomes 20 for a change. Pianoteq now does not automatically connect, so I do it manually, to the second port this time, because I like to play piano with the transport buttons.
Now the config reads

<device name="Keystation 49 MK3 Keystation 49 (2)" id="20-1"/>
<device id="24-0"/>

If I now reboot, and turn the keyboard on after, then Pianoteq defaults to connection 24:0, because that was the previously saved connection with device number 24, and the config file currently reads:

<device id="20-1"/>
<device name="Keystation 49 MK3 Keystation 49" id="24-0"/>

So it does consistently apply the appendage (2) to the name of transport control port 1, and it saves the last connection setting for each device number, but the name string only for the current connection.
Of course, if I had multiple devices, Pianoteq could in this way easily connect to the wrong device...

[j_e_f_f_g]

Here's my take on "discovering" a particular alsa device.

First of all, it's true that you never use the card (audio interface) number for saving a configuration. USB hot-swapping and such can cause the card number to change. You always have to save the card's name in your config file, and use that name when you subsequently look for that card. But you have 2 choices for the name. A "long name" or a "short name". The long name identifies the card definitively, even if there are 2 of the same model product attached to the computer. An example of the "short" name for a mythical "FocusWrong 666" card that attaches to the computer via USB could be:

Code: Select all

FocusWrong 666

Where does alsa get this name? From the USB controller chip in the FocusWrong 666. Most protocols have a way of asking for a product's "identification". In the case of USB, there's a message the computer sends over the USB cable that says "Tell me your product's name so I can show it to the enduser". And so the FocusWrong 666 will return that information.

This is what happens when you run the lsusb command. Linux asks each USB device for its name, and displays each name. So you'll see a list of product names like:

Code: Select all

FocusWrong 666
Microsoft Overpriced Series Mouse
Apple iThing made by Chinese Slaves

Now, if you have 2 FocusWrong 666 models attached to your computer, they are both going to report the same name. This is where alsa's "long name" comes in. For usb products, alsa appends which usb jack (referred to as a "buss") the product is attached to. And since more than 1 product could be daisy-chained to one buss, alsa also appends the usb unit number. For example, maybe our long name for one of our FocusWrong interfaces is:

Code: Select all

FocusWrong 666 device 2 on buss 1

And our other FocusWrong interface may be:

Code: Select all

FocusWrong 666 device 1 on buss 3

They each have a unique long name.

So to definitively find a particular card, you store the long name in your config file. But wait! What if someday the user moves his FocusWrong to a different usb jack? Now the long name is going to change. You'll then need to use the short name to find (at least one of) the FocusWrong to attach to. In other words, you need to first go through all the alsa card names looking for the long name. If you find it, you're good to go. If not, you had better have also stored the short name in your config file. Because now you need to go through the card names again, looking for that.

Next, you'll probably need to locate a particular "device" on that card. Device numbers typically don't change on most products. But if a product has various "modes" in which some parts of the product are temporarily enabled or disabled, devices could possibly appear and disappear, thus causing all device numbers to change. Less likely, but still possible, is that the name could change too. For this reason, it's best to store both the device name, and device number, in your configuration. Later when you're looking for the device, go through all the device names, looking for a matching name. When you find the matching name, then check its device number to see if it's the number you expect. If so, you're good. If not, you should probably prompt the user about a possible error in locating that device. Refer to the device by that name you stored. Ask the user if the device you found is the correct one. If he answers yes, you're good. If not, present a list of all the device names, and ask him to pick the one he wants.

If you need to look further for a sub-device, handle it the same way as a device above.

Hint: You don't need to store the card's long name. What I do is run the string through the following hash function to convert it to a unique 32-bit binary value. And that's what I store in the configuration. When I later look for the card, I run each card name through the hash function and compare the two hashes. In fact, you could do this hash conversion with all the name strings. But I'd advise keeping at least the short card name in case you need to identify the product in some error message to the enduser.

Code: Select all

// "str" is a pointer to a nul-terminated string.
// Returns a uint32_t (32-bit) hash.
uint32_t hash_string(const char * str)
{
   register uint32_t hash;
   register uint32_t len;

   hash = 0;
   while ((len = *((unsigned char *)str)++))
   {
     hash += len;
     hash += (hash<<10);
     hash ^= (hash>>6);
   }
   hash += (hash<<3);
   hash ^= (hash>>11);
   hash += (hash<<15);

   return hash;
}

[Tim E. Real]

Amazing. I see you've struggled with this too.
Thanks very much for the tips. My code could certainly be improved.
(OT: Your Backup Band looks quite interesting, I must try it...
guitarbackingtrack.com is sadly gone and I'm reeeally bored these days.)

I am running kernel 5.13.8-1 and I have an MAudio MidiSport 2x2 which
shows up as two devices. But I see no trouble with port naming. I should try a later kernel...

[j_e_f_f_g]

Ironically, Muse is my favorite linux music app. I'm using it right now to compose something. Unfortunately, I'm stuck with the AppImage version because there's noone in debian packaging muse. I wish you'd consider making a (non-AppImage) binary that can run under AV Linux. This could serve as a "reference binary" that would run under a very popular music distro, and likely be runnable on most other distros as well. (My experience is, if it runs on AV Linux, it's usually going to also run on Debian and Ubuntu at least).