[SOLVED] Is a realtime kernel needed?

Optimize your system for ultimate performance.

Moderators: MattKingUSA, khz

Dominique
Established Member
Posts: 68
Joined: Sat May 26, 2018 10:24 am
Has thanked: 3 times
Been thanked: 2 times

Re: [SOLVED] Is a realtime kernel needed?

Post by Dominique »

Jack Winter wrote:Is the use of cgroups really a problem with systemd based distros?
The use of cgroups is mandatory on system using systemd because systemd will just not work without them.

The exact cgroups setup will depend on the distribution in use. I give it a try on my gentoo it was 2 or 3 years ago. It was with the gentoo-sources kernel and the cgroups configuration from the JACK wiki. It was working fine at the beginning, but with time the rt cpu groups begin to be polluted by some kernel threads. I am not even sure if it was normal or not, and nobody was able to answer my questions. I just don't have the time to read and understand the huge cgroups documentation, so I shifted back to the rt kernel. It was possible to do so as my init system use openrc.
Jack Winter
Established Member
Posts: 381
Joined: Sun May 28, 2017 3:52 pm

Re: [SOLVED] Is a realtime kernel needed?

Post by Jack Winter »

I'm not sure I understand the cgroups vs rt kernel discussion. I use Archlinux which most certainly is a systemd based distro. I can run it with or without rt kernel, and don't think I've seen any negative impact for jackd from it's use of cgroups.. I remember that we were afraid (many years ago) that systemd/cgroups would start partitioning rt cpu runtime, but afaik that threat has never materialized.

I'm also unwilling (at least so far) to sit down to read and try to understand the cgroups documentation...:S
Reaper/KDE/Archlinux. i7-2600k/16GB + i7-4700HQ/16GB, RME Multiface/Babyface, Behringer X32, WA273-EQ, 2 x WA-412, ADL-600, Tegeler TRC, etc 8) For REAPER on Linux information: https://wiki.cockos.com/wiki/index.php/REAPER_for_Linux
Drumfix
Established Member
Posts: 299
Joined: Mon Jan 26, 2009 5:15 pm
Been thanked: 11 times

Re: [SOLVED] Is a realtime kernel needed?

Post by Drumfix »

Jack Winter wrote:Isolate one cpu and run the interrupt handling thread and jackd's processing thread(s) on it? Or more than that?
Basically that's it.

The usual kernel options you would want to set are

Code: Select all

isolcpus=1-n nohz_full=1-n rcu_nocbs=1-n nowatchdog acpi_irq_nobalance
(n depending on the number of core you want to reserve)

in my case (quadcore with core 0 = for nrt-stuff, cores 1-3 isolated) it is

Code: Select all

isolcpus=1-3 nohz_full=1-3 rcu_nocbs=1-3 nowatchdog acpi_irq_nobalance
Then move all irqs except the audio/midi irqs to the non-rt core(s) and pin the audio/midi irqs to one of the isolated cores and also disable rt throttling.

As an example here is the script i run as root after login:

Code: Select all

#!/bin/bash
# move non-rt irqs to core 0
for i in 1 3 4 5 6 7 8 9 10 11 12 13 14 15 17 18 19 22 24
do
  echo 1 >/proc/irq/$i/smp_affinity
done

#move rt irqs to core 1 (here audio irq =16 and midi irq = 20)
echo 2 >/proc/irq/16/smp_affinity
echo 2 >/proc/irq/20/smp_affinity

#disable rt throttling
echo -1 >/proc/sys/kernel/sched_rt_runtime_us
Add the following patch to jack2 (posix/JackPosixThread.cpp):

Code: Select all

--- JackPosixThread.cpp 2016-02-29 23:14:39.027647000 +0100
+++ JackPosixThread.cpp.patched 2018-05-28 07:51:37.980423538 +0200
@@ -24,6 +24,7 @@ Foundation, Inc., 59 Temple Place - Suit
 #include "JackGlobals.h"
 #include <string.h> // for memset
 #include <unistd.h> // for _POSIX_PRIORITY_SCHEDULING check
+#include <sched.h>

 //#define JACK_SCHED_POLICY SCHED_RR
 #define JACK_SCHED_POLICY SCHED_FIFO
@@ -132,6 +133,15 @@ int JackPosixThread::StartImp(jack_nativ
             return -1;
         }

+        cpu_set_t cpuset;
+        CPU_ZERO(&cpuset);
+        CPU_SET(1, &cpuset);
+
+        if ((res = pthread_attr_setaffinity_np(&attributes, sizeof(cpu_set_t), &cpuset))) {
+            jack_error("Cannot set affinity for RT thread res = %d", res);
+            return -1;
+        }
+
     } else {
         jack_log("JackPosixThread::StartImp : create non RT thread");
     }
@@ -240,6 +250,15 @@ int JackPosixThread::AcquireRealTimeImp(
                    strerror(res));
         return -1;
     }
+
+    cpu_set_t cpuset;
+    CPU_ZERO(&cpuset);
+    CPU_SET(1, &cpuset);
+
+    if ((res = pthread_setaffinity_np(thread, sizeof(cpu_set_t), &cpuset))) {
+        jack_error("Cannot set affinity for RT thread res = %d", res);
+        return -1;
+    }
     return 0;
 }
Jack Winter
Established Member
Posts: 381
Joined: Sun May 28, 2017 3:52 pm

Re: [SOLVED] Is a realtime kernel needed?

Post by Jack Winter »

Drumfix wrote:The usual kernel options you would want to set are

Code: Select all

isolcpus=1-n nohz_full=1-n rcu_nocbs=1-n nowatchdog acpi_irq_nobalance
(n depending on the number of core you want to reserve)

.
.
Thanks for writing such complete instructions, I'll try that when I get some time. I did try to boot w isolcpus and move everything off the reserved cpu and then move the soundcard irq thread and jack1's processing thread to the isolated cpu, but that gave me lots of xruns with jackd idle and no connected clients. But it might have been something else I did wrong, sometimes it takes a few tries to get these things worked out :)

IMO, /proc/sys/kernel/sched_rt_runtime_us is probably not relevant as that will only throttle rt threads when they have consumed 95% of the runtime cpu during a whole second.
Reaper/KDE/Archlinux. i7-2600k/16GB + i7-4700HQ/16GB, RME Multiface/Babyface, Behringer X32, WA273-EQ, 2 x WA-412, ADL-600, Tegeler TRC, etc 8) For REAPER on Linux information: https://wiki.cockos.com/wiki/index.php/REAPER_for_Linux
Taika-Kim
Established Member
Posts: 20
Joined: Mon Sep 07, 2015 3:16 pm

Re: [SOLVED] Is a realtime kernel needed?

Post by Taika-Kim »

Update: I'm not sure what caused those problems back then.

I'm now on Ubuntu 18.04 and M-Audio Profire 2626, and without the low latency kernel I get glitches in the sound. So at least with Firewire the stock kernel is not good. Don't know about a realtime kernel though, I didn't try one yet.
Jack Winter
Established Member
Posts: 381
Joined: Sun May 28, 2017 3:52 pm

Re: [SOLVED] Is a realtime kernel needed?

Post by Jack Winter »

"uname -a" will tell you what you have. If the output contains "PREEMPT RT" it's a realtime and if only "PREEMPT" a lowlatency kernel.
Reaper/KDE/Archlinux. i7-2600k/16GB + i7-4700HQ/16GB, RME Multiface/Babyface, Behringer X32, WA273-EQ, 2 x WA-412, ADL-600, Tegeler TRC, etc 8) For REAPER on Linux information: https://wiki.cockos.com/wiki/index.php/REAPER_for_Linux
User avatar
protozone
Established Member
Posts: 181
Joined: Tue May 08, 2018 9:02 pm
Contact:

Re: [SOLVED] Is a realtime kernel needed?

Post by protozone »

Wow, amazing tips in this thread.
I think we ought to continue the conversation in this direction by way of talking more about good kernel boot paramters.
I know for me, just adding "threadirqs" seems to speed up every distro I ever use except for the ones that already have it. I guess it's a hardware type of thing that finally matured in a good way. I don't know much about the hugepages settings; I'm not sure if they change much.

Also, I do like the idea of reducing the swappiness from 60 or whatnot to 10. That's a decent tweak.

I'm of the opinion that for every type of Windows DAW optimization/tweak, there's a Linux equivalent too. And then Linux has some extras that Windows does not have available for most users most of the time even with Registry hacking and control panel settings, special utilities, etc.

One thing which I think makes a heckuva lot of sense is the "noatime" /etc/fstab option. I'm happy that it's starting to show up in non-DAW distros.

Last but not least, I took a look the ext4 developer site pages and although I don't use it yet, eventually it will probably be even more appealing than it is now. That is unless there's too much fiddling and feature creep. Once my system wouldn't boot because I'd used a more modern gParted to format my partitions as ext4 and ext3 with some backwards compatible ext4 features (yeah, it's possible to do without making the system not boot).

The problem was that the OS versions of ext4 (and maybe even ext3) wasn't as modern as what gParted was doing to the partitions. I didn't realise it at first because the versions of ext and e2fsprogs as listed on sites such as distrowatch.org seemed to match, but I had to read the fine print from the developer and the error/boot logs more closely.

I came to the conclusion back then, that although now almost every common distro uses ext4 formatting by default, it's still actually in development, not necessarily a final phase of development. Even on the ext4 website they acknowledge that it's a work in progress and some performance features are still risky. It's for that reason and other reasons I just WON'T use ext4 for another couple of years. It needs to stabilize and be vetted. Ext3 has been thoroughly stabilized and vetted; it's pros and cons are pretty well understood it seems like (except for it's speed benchmarks(!) hmmm!). I've seen some speed tests comparing ext4 to btrfs or whatnot, but hardly any tests of just plain old ext3.

Well, that's all for now. I could go on about this stuff forever. But the masses wouldn't stand for it :D :lol:
Post Reply