Hey everyone, can someone please review my rationale behind my selection for data drives, parity drive, cache drive and comment if I'm wrong, misinformed or if there is a better selection?

 

Background: I plan to use a Norco 4224 for mostly torrents, Plex, daily TimeMachine backups, pictures. I also want the option for virtualization tasks in the future. CPU = Xeon® Processor E5-1650 v3, MOBO = Supermicro X10SRA-F.

 

1) Drives will be all green drives (i.e., 5400 rpm) because:

- I won't see much of an improvement with 7200 rpm drives

- The decreased power requirement and cooler temps will reduce monthly electrical costs

- the slower speed will make the drives last longer

- Since I'm using an SSD for a cache drive, my perceived write speeds will already be very fast negating the benefits of 7200 rpm data drives

 

2) Parity drive will also be 5400 rpm because:

- no benefit to have a 7200 rpm parity drive if all data drives are 5400 rpm

 

3) Cache drive will be a 500GB - 1TB SSD because

- perceived write speeds will be blazing fast negating the benefits of 7200 rpm drives. Obviously, the overnight transfer of data from the cache drive to the data drives will be limited to the 5400 rpm speeds.

 

My questions:

 

Q1) If the data I'm writing to the cache drive is not very important (i.e., it's not a big deal if I lose the data), is there any benefit to having multiple cache drives?

 

Q2) If I'm downloading torrents, will the data be written to the cache drive or directly to the data drives.

 

Q3) If I'm seeding torrents, does the data get read from my data drives directly to the internet? Or does it go from my data drives to the cache drive and then to the internet?

 

Q4) It seems to make the most sense to have all my seeded torrents on a single data drive so that I can minimize the number of data drives being spun at any given time, correct?

 

Q5) Is the bottleneck of a drive's throughput at the level of the drive's RPM speed, the SATA connection or the network card? A lot of fuss is made over RPM so I assume RPM is the slowest limiting factor.

- no benefit to have a 7200 rpm parity drive if all data drives are 5400 rpm

If you write to multiple drives at the same time, then a 7200 RPM parity drive helps.

This is especially true when creating a brand new parity drive. Not so much for parity check operations, but parity generate, yes.

multiple parallel writes, yes to some degree.

If you are doing torrenting directly to an array drive, then the torrent drive and the parity drive should be 7200 RPM.

 

In my builds, the parity drive and the most used drive in my systems (I have about 6 mini systems) are 7200 RPM.

 

So usually drive 1 and parity are 7200 RPM. I found I did not need to use a cache in each of these systems when configured like this.

60MB/s writes with a burstable throughput of up to 100MB/s for the first few GB works for me.

 

Where an SSD cache saves is not having to spin up drives, but since I write all day long on some servers, it doesn't really matter.

 

If I'm downloading torrents, will the data be written to the cache drive or directly to the data drives.

Depends on where you configure the torrents to be written.

If user share, then you may want to confine it to specific drives with INCLUDE.

If configured for user share, files are written to the cache first (if enabled), and moved later (if they are not open)

 

Many of my systems use the HGST 6TB 7200 RPM NAS drives for parity. They get about 225MB/s on the outer tracks.

I get excellent burst write speeds after a few kernel tunings.

SSD vs HDD won't make much difference for caching user share writes on GB ethernet since the network will be the limiting factor.

 

But the usual method is to use your cache drive not only to cache user share writes, but also as working storage for your applications. This especially makes sense with SSDs. Take Plex for example. If you put your Plex library (its database) on your parity array, then it will make the data disk + parity spin when Plex updates its library.

 

I personally don't even cache user share writes and use my cache (2 x 120GB btrfs raid1 cache pool) exclusively for applications. Most of the writes to my array take place completely unattended anyway, with torrents, NZB downloads, scheduled backups, etc., so there is no benefit to me of speeding up writes and I get instant parity protection by not going to cache.

 

Q1) If the data I'm writing to the cache drive is not very important (i.e., it's not a big deal if I lose the data), is there any benefit to having multiple cache drives?

Redundancy does have some benefit with application storage mentioned above.

Q2) If I'm downloading torrents, will the data be written to the cache drive or directly to the data drives.

Up to you how you configure the application and the share it writes to.

Q3) If I'm seeding torrents, does the data get read from my data drives directly to the internet? Or does it go from my data drives to the cache drive and then to the internet?

Nothing ever gets moved back to cache.

Q4) It seems to make the most sense to have all my seeded torrents on a single data drive so that I can minimize the number of data drives being spun at any given time, correct?

That's what I do.

Q5) Is the bottleneck of a drive's throughput at the level of the drive's RPM speed, the SATA connection or the network card? A lot of fuss is made over RPM so I assume RPM is the slowest limiting factor.

Unless you have faster than GB ethernet, network speed is the bottleneck.

Thanks guys, this is good information. I admit, much of it goes above my head.

 

If you write to multiple drives at the same time, then a 7200 RPM parity drive helps.

 

What applications or situations make sense to write to multiple drives at the same time? I plan to expand my unRAID drives one at a time (as opposed to buying 24 drives for the Norco at once) as I reach the capacity of the system, so I imagine I will only be writing to the newest drive.

 

If you are doing torrenting directly to an array drive, then the torrent drive and the parity drive should be 7200 RPM.

 

Should I be torrenting directly to an array drive instead of the cache? I don't know the benefits/cons of either method.

 

If user share, then you may want to confine it to specific drives with INCLUDE.

If configured for user share, files are written to the cache first (if enabled), and moved later (if they are not open)

 

Not too sure what is meant by user share but I'll read some more about it. I don't know what is meant by "INCLUDE" either, but I think this may just be a result of me never having used unRAID before.

 

SSD vs HDD won't make much difference for caching user share writes on GB ethernet since the network will be the limiting factor.

 

This is not something I realized ... perhaps I should just use a standard HDD instead of an SSD for my cache ...

 

But the usual method is to use your cache drive not only to cache user share writes, but also as working storage for your applications. This especially makes sense with SSDs. Take Plex for example. If you put your Plex library (its database) on your parity array, then it will make the data disk + parity spin when Plex updates its library.

 

I'm confused, how can I use my cache drive as the working storage for Plex? My Plex library is well over the size of any hard drive that currently exists.

 

And I didn't realize that all disks containing parts of my Plex Library would spin everytime Plex updates its library ... I'll have to set this to update manually or only when new media is added.

 

I personally don't even cache user share writes and use my cache (2 x 120GB btrfs raid1 cache pool) exclusively for applications. Most of the writes to my array take place completely unattended anyway, with torrents, NZB downloads, scheduled backups, etc., so there is no benefit to me of speeding up writes and I get instant parity protection by not going to cache.

 

It seems that I probably don't need a cache drive and should save my torrent files directly to the parity-protected array.

Thanks guys, this is good information. I admit, much of it goes above my head.

 

If you write to multiple drives at the same time, then a 7200 RPM parity drive helps.

 

What applications or situations make sense to write to multiple drives at the same time? I plan to expand my unRAID drives one at a time (as opposed to buying 24 drives for the Norco at once) as I reach the capacity of the system, so I imagine I will only be writing to the newest drive.

 

Let's say you are torrenting or using some other downloader to an array drive. Then you decide to save the latest edited image from your workstation or update a set of mp3 tags in a directory. The extra speed of a 7200 RPM drive will come into play (at a minor, but noticeable level).

 

The more high speed streaming you've got going on, the more noticable it is.  When I used to torrent to a 5400 RPM data drive and 5400 RPM parity drive, anything else I did on the array would drag and I would be waiting.  Todays drives are larger and faster, but I would still recommend 7200 RPM drives for the parity and the busiest drive in your array.

 

 

Movie, Media and read mostly can easily be on slower drives without much impedance.

 

If you are doing torrenting directly to an array drive, then the torrent drive and the parity drive should be 7200 RPM.

 

Should I be torrenting directly to an array drive instead of the cache? I don't know the benefits/cons of either method.

 

I did and still do. In fact I set up an HP micro server with a 3tb 7200 RPM parity drive and 3TB 7200 RPM data drive (disk1) for the live torrents and move them to the 5400 RPM 3TB drives after I've used or choose to retire them after seeding.

 

If user share, then you may want to confine it to specific drives with INCLUDE.

If configured for user share, files are written to the cache first (if enabled), and moved later (if they are not open)

 

Not too sure what is meant by user share but I'll read some more about it. I don't know what is meant by "INCLUDE" either, but I think this may just be a result of me never having used unRAID before.

 

This will become more evident as you use unRAID and see how to configure user shares. It's a way of configuring a user share that spans multiple disks and isolating that virtual join to specific disks.

 

SSD vs HDD won't make much difference for caching user share writes on GB ethernet since the network will be the limiting factor.

 

This is not something I realized ... perhaps I should just use a standard HDD instead of an SSD for my cache ...

 

There will be a measurable difference, with direct array writes you can burst at max ethernet speed and then it will slow down to somewhere ebetween 35~60MB/s.  With an SSD cache, it can sustain at a high rate of almost 90MB/s.  It depends on how much data will be moved at one time and if you can wait.

 

I move and edit mp3's all day from captured streams. I get good speed without a cache.

 

However the cache drive is needed if you are going to run VM's or Dockers.

 

I personally don't even cache user share writes and use my cache (2 x 120GB btrfs raid1 cache pool) exclusively for applications. Most of the writes to my array take place completely unattended anyway, with torrents, NZB downloads, scheduled backups, etc., so there is no benefit to me of speeding up writes and I get instant parity protection by not going to cache.

 

It seems that I probably don't need a cache drive and should save my torrent files directly to the parity-protected array.

 

As I mentioned above, I write torrents directly to the array, but I use top speed drives for the torrent drive and parity.

When I get a good high speed torrent I can sustain 7.5MB/s for as long as it takes with no impedance.

But the usual method is to use your cache drive not only to cache user share writes, but also as working storage for your applications. This especially makes sense with SSDs. Take Plex for example. If you put your Plex library (its database) on your parity array, then it will make the data disk + parity spin when Plex updates its library.

 

I'm confused, how can I use my cache drive as the working storage for Plex? My Plex library is well over the size of any hard drive that currently exists.

 

And I didn't realize that all disks containing parts of my Plex Library would spin everytime Plex updates its library ... I'll have to set this to update manually or only when new media is added.

Your Plex Library is not all of your movies, etc. It is the data plex keeps about all your movies, etc.

 

For example, I have a Movies share, and it actually uses multiple drives to store my Movies. But my Plex library, its database, is stored on cache. When I watch a movie, Plex updates the library to keep track of my position in the movie so I can resume it later. It keeps track of which movies have been watched. If there are new movies, it adds them to the list, etc.

 

As far as drive RPMs, depends on your specific use case whether it matters or not. Almost everything written to my server after the initial load is written by applications on the server, or by backup applications on other PCs. Most of this happens automatically in the middle of the night and nobody is waiting. Reads are typically streaming, and slower drives can usually keep up with my movies, etc. Maybe if you have full BD rips and multiple users it would be an issue, but not for me.

 

As far as drive RPMs, depends on your specific use case whether it matters or not. Almost everything written to my server after the initial load is written by applications on the server, or by backup applications on other PCs. Most of this happens automatically in the middle of the night and nobody is waiting. Reads are typically streaming, and slower drives can usually keep up with my movies, etc. Maybe if you have full BD rips and multiple users it would be an issue, but not for me.

 

I would have to agree here. How the array is used at an interactive level is how I determine drive choice. Batch operations can easily wait.

All of my workstations and laptops have small 256GB SSDs.

All of the big data and/or user/personal files is stored on the the unRAID server(s).

I'm writing and updating all day long.

Let's say you are torrenting or using some other downloader to an array drive. Then you decide to save the latest edited image from your workstation or update a set of mp3 tags in a directory. The extra speed of a 7200 RPM drive will come into play (at a minor, but noticeable level).

 

The more high speed streaming you've got going on, the more noticable it is.  When I used to torrent to a 5400 RPM data drive and 5400 RPM parity drive, anything else I did on the array would drag and I would be waiting.  Todays drives are larger and faster, but I would still recommend 7200 RPM drives for the parity and the busiest drive in your array.

 

Got it. So would having a cache to download new torrents also achieve this result you are talking about (avoiding the wait/drag)?

 

However the cache drive is needed if you are going to run VM's or Dockers.

 

Got it, for this reason alone I will probably get an SSD for a cache drive, or at least one in the future. I presume this is what was meant by trurl when he said the SSD cache should be used as working storage for applications.

 

As I mentioned above, I write torrents directly to the array, but I use top speed drives for the torrent drive and parity.

When I get a good high speed torrent I can sustain 7.5MB/s for as long as it takes with no impedance.

 

This is great info, thank you.

 

But the usual method is to use your cache drive not only to cache user share writes, but also as working storage for your applications. This especially makes sense with SSDs. Take Plex for example. If you put your Plex library (its database) on your parity array, then it will make the data disk + parity spin when Plex updates its library.

 

I'm confused, how can I use my cache drive as the working storage for Plex? My Plex library is well over the size of any hard drive that currently exists.

 

And I didn't realize that all disks containing parts of my Plex Library would spin everytime Plex updates its library ... I'll have to set this to update manually or only when new media is added.

Your Plex Library is not all of your movies, etc. It is the data plex keeps about all your movies, etc.

 

For example, I have a Movies share, and it actually uses multiple drives to store my Movies. But my Plex library, its database, is stored on cache. When I watch a movie, Plex updates the library to keep track of my position in the movie so I can resume it later. It keeps track of which movies have been watched. If there are new movies, it adds them to the list, etc.

 

Got it, just the Plex metadata is stored on the cache, not the actual library of media which is on the parity array.

 

I now get what you meant when you said "working storage for your applications."

 

As far as drive RPMs, depends on your specific use case whether it matters or not. Almost everything written to my server after the initial load is written by applications on the server, or by backup applications on other PCs. Most of this happens automatically in the middle of the night and nobody is waiting. Reads are typically streaming, and slower drives can usually keep up with my movies, etc. Maybe if you have full BD rips and multiple users it would be an issue, but not for me.

 

Most of my movies are uncompressed Blu-Ray Remux (typically ~30GB per movie). This makes Plex transcoding very CPU intensive (hence my choice for E5-1650 v3). I don't foresee more than 3 simultaneous transcodes, but if all 3 transcodes are from the same 5400rpm drive, do you foresee problems?

 

Thanks guys, these were great responses and very informative. Just a couple questions.

 

Q1) If I choose to save torrents to the cache drive, and when the torrent eventually moves to the parity protected array overnight, do I have to manually "force re-check" the seeded torrent through torrent software (Transmission) in order to continue seeding or is the torrent software/unRAID smart enough to know the new location of the seeded torrent? I don't want to have to manually "force re-check" every torrent I download after it moves from the cache to the parity protected array.

 

Q2) Based on what trurl and WeeboTech have been saying, I should be saving all programs (Plex, VM) on the SSD cache, but leave the large library databases (Plex) on the parity-protected arrays? (this is super obvious/dumb question but I want to make sure I understand the basic rationale).

Q1) If I choose to save torrents to the cache drive, and when the torrent eventually moves to the parity protected array overnight, do I have to manually "force re-check" the seeded torrent through torrent software (Transmission) in order to continue seeding or is the torrent software/unRAID smart enough to know the new location of the seeded torrent? I don't want to have to manually "force re-check" every torrent I download after it moves from the cache to the parity protected array.

The user share where you save torrents can be cached, so logically there is no difference between whether a file is on cache or on the parity array. However, open files will not be moved. You should probably study the wiki about user shares.

Q2) Based on what trurl and WeeboTech have been saying, I should be saving all programs (Plex, VM) on the SSD cache, but leave the large library databases (Plex) on the parity-protected arrays? (this is super obvious/dumb question but I want to make sure I understand the basic rationale).

The actual media files belong on the parity array. The library database belongs on cache. The applications and VMs also belong on cache. When you get ready to setup dockers this will probably become clearer.

Q1) If I choose to save torrents to the cache drive, and when the torrent eventually moves to the parity protected array overnight, do I have to manually "force re-check" the seeded torrent through torrent software (Transmission) in order to continue seeding or is the torrent software/unRAID smart enough to know the new location of the seeded torrent? I don't want to have to manually "force re-check" every torrent I download after it moves from the cache to the parity protected array.

The user share where you save torrents can be cached, so logically there is no difference between whether a file is on cache or on the parity array. However, open files will not be moved. You should probably study the wiki about user shares.

Q2) Based on what trurl and WeeboTech have been saying, I should be saving all programs (Plex, VM) on the SSD cache, but leave the large library databases (Plex) on the parity-protected arrays? (this is super obvious/dumb question but I want to make sure I understand the basic rationale).

The actual media files belong on the parity array. The library database belongs on cache. The applications and VMs also belong on cache. When you get ready to setup dockers this will probably become clearer.

 

Thanks so much for your help. As you astutely pointed out, I think much of my basic questions will become obvious/answered once I setup my unRAID. Thanks again for the pointers, I've decided to use an SSD cache because:

 

1) I don't want to have a parity-protected array of mixed 5400 and 7200 rpm drives. If I get a 7200rpm drive for my parity and for my most used data drive, those 7200 rpm drives will eventually be moved to the regular data array since the parity and most used drive will be replaced by newer, higher capacity drives in the future. This is especially true for the parity since that has to be the highest capacity drive in the array. By having an SSD cache, I won't have the problem of "lagging" as WeeboTech mentioned that he gets when saving torrents directly to a 5400 rpm drive.

 

2) I like the idea of only having 1 drive spun up (the cache drive) for most of the day rather than the 2 drives (parity + data drive) when saving torrents.

 

3) I'll save vital data (e.g., family photos) directly to the parity-protected array and save non-vital data (e.g., torrents) to the cache drive.

 

4) I want the option of launching VM and dockers directly from the SSD Cache. Since the cache isn't parity protected, I can just backup the settings for those apps to the parity data array on a regular basis.

 

Thanks again for the input guys.