New build. Expand as needed or all at once?

[jebusfreek666]

Ok, so I just completed my first ever build (of any computer). I have run all 8 drives through preclear, and decided to run memtest while I am at work today, just to get a better feel for longevity of the system. I will be using this sever mostly as a plex server, but also for document/picture back up. Further down the road I plan to research VM and play with that as well, and maybe some home automation. So, my question is this. Generally speaking, is it better to add all the storage capacity all at once or wait until more space is needed to assign the drives? Like I had said, I have 8 drives. I will run single parity for the time being. My media is currently taking up about 8tb of data on externals. The drives I have purchased are all 6tb. So, at minimum, I will need to add 1 parity, and 2 data drives. Which will leave 5 more drives left over. Should I leave these 5 unassigned (as warm spares I guess they would be considered) until such time as I need to expand my storage capacity, or should I assign them all (except maybe one as a spare) right off the bat.

 

Also, super dumb question, but when transferring files over from my externals through crusader, do I add them directly to the array or go through the cache and let mover handle adding them to the array while I am sleeping? I do have 2 500gb SSD, which I plan to use as cache (raid1). I realize this would limit me to less than 500gb at a time, and ultimately increased the amount of time it takes to transfer to the array. Just wondering if there is some other added benefit that I should consider, or if it is just quicker, easier, and smarter to toss them directly on the array. (see.... super dumb question, which I am 99% certain I already know the answer to!)

Edited January 28, 2018 by jebusfreek666

[tdallen]

Well, first - congrats on building your first system!

 

On your question, how quickly will you fill up your array?  If you plan to fill it up quickly (say, you're converting a big DVD/Bluray library) then I'd add enough disks now.  If you plan to fill it up slowly then I'd leave the other drives cold/offline for now.  Actually, I would have advised you not to buy them until you need them because now the the warranty clock is ticking...  But since you have them there's no point subjecting them to power/heat/spin-up cycles in the server.

 

On the transfer, I'd turn off write-caching for now.  Then, turn on Turbo Write (in my signature).  Or you could just unassign Parity until the transfer is complete.  Either approach is better than using the cache drive for the amount of data you want to transfer.

[JonathanM]

Only keep the drives you need in the array. Multiple reasons, most important is how parity protection works means every disk must be read perfectly to recreate the dead drive, regardless of content of all the drives. So, if you have 5 empty drives and one of your full drives fails, you are depending on those empty drives to reconstruct your data. Every added spindle increases risk.

 

Much better to keep tested spares on hand to replace drives any time they show signs of failure. I'd make sure you do what is necessary to keep those spare drives spun down if you leave them connected electrically, to keep the run time hours down. Better yet just disconnect them after they test good, and wait for them to be needed in one capacity or another.

 

You didn't mention the actual case hardware, but a large percentage of disk issues are caused by cable stress when mucking about inside the case. I'd strongly recommend using hot swap bays so you don't have to touch the cables when dealing with drive changes.

 

You are keeping the data on the externals as well for backup, right?

[jebusfreek666]

14 minutes ago, tdallen said:

Well, first - congrats on building your first system!

 

On the transfer, I'd turn off write-caching for now.  Then, turn on Turbo Write (in my signature).  Or you could just unassign Parity until the transfer is complete.  Either approach is better than using the cache drive for the amount of data you want to transfer.

 

Thanks! Surprisingly, I have not run into a single issue as of yet. Everything has been plug and play, and I am figuring things out as I go. But so far, everything has worked how I have intended it to, and haven't plugged anything in the wrong spot..... yet  I will read about the Turbo Write, hopefully that cuts down a ton of time.

 

11 minutes ago, jonathanm said:

Much better to keep tested spares on hand to replace drives any time they show signs of failure. I'd make sure you do what is necessary to keep those spare drives spun down if you leave them connected electrically, to keep the run time hours down. Better yet just disconnect them after they test good, and wait for them to be needed in one capacity or another.

 

You didn't mention the actual case hardware, but a large percentage of disk issues are caused by cable stress when mucking about inside the case. I'd strongly recommend using hot swap bays so you don't have to touch the cables when dealing with drive changes.

 

You are keeping the data on the externals as well for backup, right?

 

I was thinking about keeping 1 connected, but unassigned, so it could be used as a warm spare and disconnecting the others. Then I could use my warm spare to either replace a failed drive (unlikely at this stage) or add to the array when needed and slap in another to replace the warm spare. I am debating pulling them out entirely and re-boxing them so they are not even subjected to the heat in the tower. But, honestly, I kind of went for overkill on this thing and it doesn't get that warm. I think if I pulled them out, it would stay a few degrees cooler though, since there would be larger gaps between the drives.

 

As for the hardware, I have a Define R5. No hot swaps. But I did spend a few extra bucks and pick up all locking SATA cables. So, hopefully, that wont be an issue.

 

As of right now, I will keep backups of what I have on externals. But as the collection grows, I will probably not backup the media as it should be fairly easy (albeit time consuming) to replace. I will however continue to backup the pictures/documents that I can not replace. Honestly, this whole build has been fun so far so I have been kicking around the idea of building a server to "give to my mom". That way she can store whatever she wants, and I would have an offsite backup for all my most important things should something happen in the house.

[JonathanM]

34 minutes ago, jebusfreek666 said:

But I did spend a few extra bucks and pick up all locking SATA cables.

https://support.wdc.com/knowledgebase/answer.aspx?ID=10477

This isn't limited to WD, make sure your cables are the correct type for the specific drive. Locking cables may not make good connections on some drives.

 

Hot swap bays are worth their weight in gold for avoiding cable issues.

36 minutes ago, jebusfreek666 said:

I am debating pulling them out entirely and re-boxing them so they are not even subjected to the heat in the tower.

Nah, disconnecting them is plenty. The storage temp limits are WAY wider than operating limits, they'll be perfectly fine hanging out in their permanent homes until needed.

38 minutes ago, jebusfreek666 said:

That way she can store whatever she wants, and I would have an offsite backup for all my most important things should something happen in the house.

That is a solid plan. Mirror each other's irreplaceable stuff in a geographically separated location.

39 minutes ago, jebusfreek666 said:

I will probably not backup the media as it should be fairly easy (albeit time consuming) to replace.

As you age, time starts to be more valuable, and money less so. Carefully weigh the value of your hours vs. the cost of storage. How many hours do you have to work to aquire a 6TB drive for backup vs. how long it would take to deal with 6TB of media replacement. That equation will change over time, sometimes quicker than you would like.

[jebusfreek666]

44 minutes ago, jonathanm said:

As you age, time starts to be more valuable, and money less so. Carefully weigh the value of your hours vs. the cost of storage. How many hours do you have to work to aquire a 6TB drive for backup vs. how long it would take to deal with 6TB of media replacement. That equation will change over time, sometimes quicker than you would like.

 

That makes perfect sense. And if it was only ever going to be 6TB, it would be a no brainer. However, the thought of keeping backups of an entire "large" collection, say 60TB, seems like it could be a huge pain. Especially if that collection changes frequently. I do see your point though. IT would be an awfully painful experience either way, but probably more so to have to reacquire it all.

[SSD]

5 hours ago, jebusfreek666 said:

My media is currently taking up about 8tb of data on externals. The drives I have purchased are all 6tb. So, at minimum, I will need to add 1 parity, and 2 data drives. Which will leave 5 more drives left over. Should I leave these 5 unassigned (as warm spares I guess they would be considered) until such time as I need to expand my storage capacity, or should I assign them all (except maybe one as a spare) right off the bat.

 

With hot swap cages adding drives JIT works well. Without them, I'd opt for fewer tinkerings inside the case. If you are like most, you're first few months will see dramatic increases in space usage. If you add just 2 data drives, you'll have about 4T free. No enough IMO. I would add 2 extra (4 data drives), giving you 16T to grow.  That should keep you for a year or more, depending on your usage pattern. 

 

Another disk I would suggest for dual parity. Dual parity protects you from a very unlikely second failure, but for a new user, its more important duty is to protect you from certain types of mistakes / missteps. It is not absolute protection, but it can protect you from certain mistakes that new users encounter. Dual parity does not need to be a permanent decision, but I suggest new users opt for it even with smaller arrays.

 

That will leave you 2 disks. You should always have one or two precleared and waiting in the wings for a failure. 

 

Maybe by the time you've filled up your 4 data disks and are ready to add more drives, you'll have considered adding hot-swap bays and can do at that time.

[trurl]

Make sure you setup Notifications so unRAID can notify you of problems. Since unRAID will emulate a missing disk, it is possible to have access to all your files even if you have a serious problem. Notifications will keep you aware of any problems so you don't wind up with multiple problems.

 

And use the forums!

 

Probably the worst situations are people who don't know they have a problem, and keep running till they have even more problems. Then when they finally notice the problems they try to fix it without knowing how.

 

[trurl]

33 minutes ago, trurl said:

Probably the worst situations are people who don't know they have a problem, and keep running till they have even more problems. Then when they finally notice the problems they try to fix it without knowing how.

 

Case in point: