using multiple hard drives

[tatch]

https://www.ableton.com/en/articles/opt ... ive-setup/ any thoughts when applying this to a linux setup?

[turquoisesky]

Works fine for me, where's the problem?

[Frank Carvalho]

Me too. Two hard drives, no problem. I think the most important thing to be aware of, is to make sure the OS resides on one drive, while the DAW works on the other drive. The OS regularly accesses the drive for reading system files, writing to log files etc., and this will stress the disk if it is also used for reading and writing to the same drive. Therefore my setup has the OS, and more static resouces, like samples, on one drive, while I use the other drive for DAW operation.
If you want a more advanced setup, you could have several drives mounted using a fast type RAID for fast read/write operation (and still keep the OS on a separate drive). This way it is left to the RAID setup to offload disk traffic.

/Frank

[freshmelon]

Also here no problem.

[tatch]

I guess the purpose of this post isn't to ask "do you use multiple hard drives" because that in itself is trivial; I'm more interested in what exactly can be gained from it besides increased storage space. Is getting the DAW to "work" on the other drive just a matter of setting the project directory to that drive? I'm also interested in the usage of the 3rd drive mentioned in that article as dedicated to multitrack recording and how the recordings are automatically moved (and presumably re-linked) to the project folder upon saving. How detrimental is the read/write "traffic jam" they mention in their scenario? I'm sure it wouldn't typically break your system but I'm curious about the extent of the benefits.

[Frank Carvalho]

Is getting the DAW to "work" on the other drive just a matter of setting the project directory to that drive?

Yes.

I am not aware of any native method to get the DAW to spread disk usage across several disks. Ardour has one folder for audio, as is common for DAWs. You could probably distribute files on other disks, using soft links to the physical files, so as to spread disk IO, but this is not terribly practical - and saving a project becomes unnecessarily complicated.
Then rather use a suitable type of RAID and have the RAID controller manage disk IO.
But I think in most cases disk IO is not much of a problem anyway. Just make sure you have OS on one disk and DAW on another. DAWs will usually cache files or parts of files in memory.

/Frank

[Luc]

https://www.ableton.com/en/articles/opt ... ive-setup/ any thoughts when applying this to a linux setup?

Thought #1:
I am very familiar with that concept because I've come across quite a few people who advocated the use of several disks to improve performance. The idea back then was to make the computer use the two disks on the same IDE controller, because it can't use two controllers at the same time, but it can use two disks on the same controller at the same time. It was a poor man's kind of 'striping.' The second IDE controller was best left for the CD drive. I don't know if that concept still applies on SATA world.

I tried it myself a couple of times and I always hated it. It is a bit faster, but not that much, and making backups was a nuisance. After many experiments, I have settled with one hard disk at all times, I don't care if I have to buy a bigger one, and always have another disk of the same capacity to clone the production hard disk. I use rsync to update my backups. That arrangement is especially useful with Linux, because you can just swap disks and get productive in no time. On Windows, it's never too easy.

Thought #2:
I've seen a few people advocate the separation of system from data in dedicated disks, arguing that "if the system fails or the hard disk dies, my data is safe."

Erm, no. System failure is no big deal, you can just pop in a CD and restore it, though you lose your configuration, but more importantly: what if the data disk dies instead of the operating system disk? Those people don't seem to take that into consideration. That is one more reason to put everything in one disk, but have another with full backups.

Thought #3:
The article says:

"Should no slot be available, or if you work with a laptop computer, you will need to install the drive in a USB, Firewire or Thunderbolt enclosure: note that doing so will slightly compromise the drive speed."

That is a bad idea. A USB drive will be much slower and defeat the entire purpose of the idea (though I have no experience with USB 3.0). And USB isn't very safe, errors may occur and you may lose data. My backup procedure fails halfway through it sometimes because USB gets disconnected, and I have to restart it. It happens.

If you really want to do something like that, then do proper RAID. And get a powerful machine, with a fast processor and lots of RAM.

[Frank Carvalho]

I've seen a few people advocate the separation of system from data in dedicated disks, arguing that "if the system fails or the hard disk dies, my data is safe."

Agreed, that is not a good reason. The good reason for separating OS and DAW, is that the DAW should have uninterrupted access to IO, but so does the OS. You never know when the OS is going to kick in with an urgent request for disk access. Having the OS on its own disk keeps its requests away from the flow of data of the DAW.

Oh, and I totally agree. The DAW disk should be complemented with a backup disk and rsync. I do that as well.

[asbak]

Assuming the idea is to maximise performance opportunities

- Get a lot of RAM. Don't be shy. 8GB minimum. 16GB preferable.
- Get a SSD and install the OS on it. 120GB is way more than you need for an OS. 240GB if you want to dual boot with Windoze and still have tons of space left.
- Don't create any swap partitions. It will affect drive performance and impact the SSD's lifespan. (This is why you got that extra RAM).
- Obtain a separate large internal SATA harddrive, preferably a reasonably speedy one and not a "Green" type. Mount /home on it and keep your projects, samples and data under /home/<username>.
- Edit your /etc/fstab file and mount your drives with noatime.
- Disable unnecessary system services and services which pester your drives with endless activity such as tracker-miner-fs on Debian. It's a real stupid service.
- Check what's happening on your system with iotop -ao. Leave it running over a long period and identify services or processes making unnecessary use of the drives, especially during idle time. Get rid of those processes and services if practical and possible.
- Use rsync to backup your data to a separate backup drive. USB drives are fine for taking backups. Do this while you're not using the system for audio production.

This way you get snappy performance and reasonable data security.

If your data is valuable enough then keep another backup drive off site and rotate it from time to time with your original backup drive.

[Luc]

- Don't create any swap partitions. It will affect drive performance and impact the SSD's lifespan. (This is why you got that extra RAM).

I wonder if that is true. I've always worked with little RAM (not with music, though; I am a newbie in this area), and I've had many resource meters installed that always told me that my swap partition was rarely ever used. On Linux, of course. Windows is a very different story.

So even if the swap partition exists, I believe it won't be used if you have enough RAM.

[EdRez]

So even if the swap partition exists, I believe it won't be used if you have enough RAM.

swappiness is parameterised; the operating system will resort to swapping before memory is full. You can set at what point that kicks in, and I have that set at over 90% memory usage (how much over depends on total system RAM, the more there is the higher I set it). On my laptop, I have the swap set to slightly larger than RAM. I still never want it used when doing audio work, but I like the option to suspend the system via RAM content being pushed to swap. I also do some digital image work which exceeds the 8GB or 16GB RAM of either of my systems. In those cases, I need the swap. If anything gets kicked to swap whilst I'm in the middle of audio work, the session will probably be wasted anyway.

As a back-of-an-envelope calculation: Assuming you're letting something like Ardour write 32-bit sample data per-channel, and that you're recording 16 channels at 96kHz; you then need to be able to support around 50Mbps of write bandwidth to the disk (circa 6.2 MB/s). That's below the USB 2.0 spec of 480Mbps (~ 60 MB/s) and way under the USB 3.0 spec of 4800Mbps (~ 600 MB/s). You'll never get those performance figures out of either (conservatively, lop off 25-30% for overheads), but you'll certainly get enough to not have problems if you're using a dedicated USB disk for your DAW.

Modern systems handle writing uncompressed HD video data directly to disk just fine, so why would you worry about audio?

[asbak]

- Don't create any swap partitions. It will affect drive performance and impact the SSD's lifespan. (This is why you got that extra RAM).

I wonder if that is true. I've always worked with little RAM (not with music, though; I am a newbie in this area), and I've had many resource meters installed that always told me that my swap partition was rarely ever used. On Linux, of course. Windows is a very different story.

So even if the swap partition exists, I believe it won't be used if you have enough RAM.

The problem is that once you start running a lot of apps, huge samplesets and so on your RAM gets eaten up quickly.

Right now a quick glance at free tells me that I'm already down 3.5GB and that's just with Muse + Linuxsampler + some not huge piano samples, a softsynth, some effects and a webbrowser running. No doubt one can run with lean resources or memory usage can be improved but do I really want to spend hours tuning the system and running barebones? Not really. RAM's cheap and I'm not in a puppy linux competition.

Once the RAM's approaching exhaustion it'll start swapping assuming there's a swap device available. If not the system will probably slow to a crawl or hang which won't be conducive to one's good humour or meaningful results.
So for me at least, the system needs 4GB just to get by. 8GB gives some breathing space and 16GB obviously leaves a nice cushion.

Yes it can be made to work on less RAM & with swap partitions. Personally I just don't see the point, not if the aim of the game is to maximise performance gains and this can be achieved for relatively modest costs.

[Luc]

So I suppose you can keep the swap partition and use the swapon/swapoff command to toggle it.

[tramp]

So even if the swap partition exists, I believe it won't be used if you have enough RAM.

Indeed. And it would never harm your system to set it up with a swap partition enabled.
Just, for the case you have enough RAM, but care about disk-space, you could leave it out.
But at least, if you run out of RAM, your swap partition will keep your system alive.