Mat1926 Posted December 15, 2017 Share Posted December 15, 2017 Okay I am expecting all the parts to be delivered today or tomorrow. I do have 2 WD NAS appliances, each one contains 4 x 8 TB WD Red HDDs -so total 8 HDDs-. They are new, like 3-4 months old and I did not use them that much. In the 1st NAS I have like 1 TB left -~ 24 TB used, Raid 5-, and regarding the 2nd WD NAS I only used 3-4 TBs only. So I plan to order new HDDs from Amazon, maybe 3 x 10 TB WD Red. I plan to initially start using those 3 HDDs and transfer my 4 TB from my second WD NAS, and when I'm done. I will use the 4 x HDDs in the UnRaid array. Then I will start moving my data from my 1st WD NAS -~ 24 TB-, and when I'm done I will also plug the 4 HDDs in the UnRaid system. So in the end I will have 8 x 8 TB WD Red and 3 x 10 TB WD Red. Your thoughts about this please? Can I have different sizes? What if 10 TB failed, then I need to replace 2 x 8 TBs? Also, what is the best way to transfer the data from my old WD NAS to the UnRaid NAS? Just basic copy over the network using total commander for instance? Is that reliable? I want to make sure that all the data was transferred accurately before using the old HDDs in the new system. I have thousands of small files, not just big files... Any general recommendations and precautions? Thnx in advance guys, i am looking forward to use your system... Link to comment
ashman70 Posted December 15, 2017 Share Posted December 15, 2017 With unRAID, if you choose to have a parity drive, it has to be larger than or equal to your largest drive. So in your case, a 10TB drive would be used for parity. You can also choose to have two parity drives, they don't have to be equal in size, so in your case you would have to use a 10TB for your first parity drive, but could use an 8TB as your second. unRAID is not like hardware RAID, there is no striping of parity. If you lose one drive, its contents are emulated by the parity drive and other drives, until you can replace it. Copying the data over the network seems to be the best way to do it, so yes. There are several popular file compare programs you can use for Windows, system commander was one I believe, you could use that to compare the data from the NAS that you have copied to the unRAID server when it's done. Link to comment
Mat1926 Posted December 15, 2017 Author Share Posted December 15, 2017 2 minutes ago, ashman70 said: With unRAID, if you choose to have a parity drive, it has to be larger than or equal to your largest drive. So in your case, a 10TB drive would be used for parity. You can also choose to have two parity drives, they don't have to be equal in size, so in your case you would have to use a 10TB for your first parity drive, but could use an 8TB as your second. unRAID is not like hardware RAID, there is no striping of parity. If you lose one drive, its contents are emulated by the parity drive and other drives, until you can replace it. Copying the data over the network seems to be the best way to do it, so yes. There are several popular file compare programs you can use for Windows, system commander was one I believe, you could use that to compare the data from the NAS that you have copied to the unRAID server when it's done. thnx. So I will start by using 1 x 10 TB as parity and 2 x 10 TB for data, and after migrating my data from my 2nd WD NAS, I will plugin the 4 x 8 TBs to the system and use 2 x 8TB as parity and 2 x 8 TB for my data. The I will have like ~32 TB of usable space so that I can move the data from the 1st WD NAS and when I am done I can also add another 1 x 8TB as parity and 3 x 8 TB for data... So in total I have 1 x 10 TB and 3 x 8 TB for parity, and 2 x 10 TB plus 5 x 8 TB for data...so I have roughly 60 TB of total space. Is that correct? What is the max number of drives if failed I am still protected? The problem is that I am mixing different sizes so maybe 2 x 10 TB will fail or 3 x 8 TB might fail simultaneously.... Now, regarding the migration of my data, I was referring to a 2 panel file manager called Total Commander - similar to that old DOS utility Norton Commander -, it can move that data w/o issues but do I need to double check the data or this transfer over the network is already protected against data corruption automatically? Thnx Link to comment
ashman70 Posted December 15, 2017 Share Posted December 15, 2017 You can only have a maximum of two parity drives in unRAID. With a PRO license you can have a maximum of 28 data drives and two parity drives, so 30 drives, this does not include drives you may use for cache. Data corruption can happen at any time, but if you are certain the cables and network cards are in good order and you haven't seen any issues transferring data over your network, you should be ok. That being said, whether you currently have any data corruption or not is pretty hard to tell. There are plugins you can download for unRAID where you create hashes of your data and compare them on a monthly bases to see if you have any data corruption and this may be something you want to look into down the road. Personally I use the BTRFS file system on one of my unRAID servers along with ECC to help protect against possible data corruption. Link to comment
Mat1926 Posted December 15, 2017 Author Share Posted December 15, 2017 3 minutes ago, ashman70 said: You can only have a maximum of two parity drives in unRAID. With a PRO license you can have a maximum of 28 data drives and two parity drives, so 30 drives, this does not include drives you may use for cache. Data corruption can happen at any time, but if you are certain the cables and network cards are in good order and you haven't seen any issues transferring data over your network, you should be ok. That being said, whether you currently have any data corruption or not is pretty hard to tell. There are plugins you can download for unRAID where you create hashes of your data and compare them on a monthly bases to see if you have any data corruption and this may be something you want to look into down the road. Personally I use the BTRFS file system on one of my unRAID servers along with ECC to help protect against possible data corruption. I will use the pro license. So basically I cant have any number of parity HDDs, is that what you are saying? my RAM is " 2 x Supermicro Certified MEM-DR480L-SL01-EU21 Samsung 8GB DDR4-2133 ECC Un-Buffer LP Server Memory"... Regarding the file system, what is the best one? Link to comment
ashman70 Posted December 15, 2017 Share Posted December 15, 2017 Correct, you cannot have more than two parity drives. "Best" is up to you, you really have two choices XFS or BTRFS. Out of my four unRAID servers three are XFS and one is BRTFS. If I could do it all over again, I think three would be BTRFS and one XFS. BTRFS is a relatively new file system however it supports the capability to do snapshots, something that has not yet been made available in unRAID. Link to comment
Mat1926 Posted December 15, 2017 Author Share Posted December 15, 2017 Just now, ashman70 said: Correct, you cannot have more than two parity drives. "Best" is up to you, you really have two choices XFS or BTRFS. Out of my four unRAID servers three are XFS and one is BRTFS. If I could do it all over again, I think three would be BTRFS and one XFS. BTRFS is a relatively new file system however it supports the capability to do snapshots, something that has not yet been made available in unRAID. Can I ask you why you are having 2 different file systems? You said even if you did it all over again you will still have 1 XFS...Can I ask why is that? Thnx Link to comment
ashman70 Posted December 15, 2017 Share Posted December 15, 2017 Because the one server that would be XFS isn't doing or storing anything important that I care about, however the other three servers do store data I care about. It's not like I'm really losing much by having XFS over BTRFS, I just found out about BTRFS after I had already made my servers XFS. XFS is a mature file system that works very well. I've not had any problems with either. Link to comment
Mat1926 Posted December 15, 2017 Author Share Posted December 15, 2017 Thank you very much for your helpful replies, I appreciate it a lot... Link to comment
ashman70 Posted December 15, 2017 Share Posted December 15, 2017 Anytime, good luck with your build. Link to comment
Mat1926 Posted December 23, 2017 Author Share Posted December 23, 2017 Just a quick update... Received all parts few days ago, and I just finished building my system, did install the trial key for the latest unraid stable build. My plan is to purchase the pro license key since my system supports 24 bays. I will have 2 WD 10 TB Reds for parity, and the rest of HDDs are for data. I have 3 raid controllers, each provides 2 mini sas ports, so total 6 cables for my system. What is the best way to attach the HDDs? Is it preferable to install each parity HDD on a separate raid controller, or its better to have them on the same controller? Is it better to start by having the parity HDDs on their own controller, and start populating the data HDDs using the other 2 raid controllers? Thnx Link to comment
SSD Posted December 23, 2017 Share Posted December 23, 2017 33 minutes ago, Mat1926 said: is the best way to attach the HDDs? Is it preferable to install each parity HDD on a separate raid controller, or its better to have them on the same controller? Is it better to start by having the parity HDDs on their own controller, and start populating the data HDDs using the other 2 raid controllers? Not sure what cards you are trying to use to connect the drives, but you generally need HBAs and not RAID cards. Some RAID cards can be flashed with IT firmware to make them HBAs, and some RAID cards have passthrough modes and can work with unRaid (e.g., Areca cards), but generally you want vanilla HBAs Shouldn't matter which controller you hook which drive to. Hopefully you have read about and understand how parity works, and understand that normal unRaid writes are not going to be extremely fast. Look into reconstruct (a.k.a. turbo) write mode for your initial load to get much faster I/O at the expense of requiring all drives to be spinning to do writes. Most users use this to load bulk data faster, but then turn it off for normal use. Good luck with your build! Link to comment
tdallen Posted December 23, 2017 Share Posted December 23, 2017 Hi - What motherboard and CPU are you using? Link to comment
Mat1926 Posted December 23, 2017 Author Share Posted December 23, 2017 Motherboard 14 minutes ago, SSD said: Not sure what cards you are trying to use to connect the drives, but you generally need HBAs and not RAID cards. Some RAID cards can be flashed with IT firmware to make them HBAs, and some RAID cards have passthrough modes and can work with unRaid (e.g., Areca cards), but generally you want vanilla HBAs Shouldn't matter which controller you hook which drive to. Hopefully you have read about and understand how parity works, and understand that normal unRaid writes are not going to be extremely fast. Look into reconstruct (a.k.a. turbo) write mode for your initial load to get much faster I/O at the expense of requiring all drives to be spinning to do writes. Most users use this to load bulk data faster, but then turn it off for normal use. Good luck with your build! IBM M1015 controllers, are these suitable? Link to comment
Mat1926 Posted December 23, 2017 Author Share Posted December 23, 2017 3 minutes ago, tdallen said: Hi - What motherboard and CPU are you using? Supermicro X11SAT-F-O and Xeon E3-1230 V5 Link to comment
CHBMB Posted December 23, 2017 Share Posted December 23, 2017 My system is similar and I cabled it in an ordered fashion to make troubleshooting straightforward, all cables labelled both ends.Consequently my parity is on the same controller with no issues that I've noticed.Sent from my LG-H815 using Tapatalk Link to comment
Mat1926 Posted December 23, 2017 Author Share Posted December 23, 2017 2 minutes ago, CHBMB said: My system is similar and I cabled it in an ordered fashion to make troubleshooting straightforward, all cables labelled both ends. Consequently my parity is on the same controller with no issues that I've noticed. Sent from my LG-H815 using Tapatalk Are my raid controllers suitable? IBM M1015... Link to comment
CHBMB Posted December 23, 2017 Share Posted December 23, 2017 Yep, flashed to IT modeSent from my LG-H815 using Tapatalk Link to comment
Mat1926 Posted December 23, 2017 Author Share Posted December 23, 2017 Just now, CHBMB said: Yep, flashed to IT mode Sent from my LG-H815 using Tapatalk From where can I get the FW? Link to comment
uldise Posted December 23, 2017 Share Posted December 23, 2017 20 minutes ago, Mat1926 said: From where can I get the FW? see here: Link to comment
Mat1926 Posted January 2, 2018 Author Share Posted January 2, 2018 Okay guys. I just got my 3 new 9211-8i cards, they are already in IT mode. Please check the screens The only issue is that I cant reach the cards bios, when I click ctrl+C the system shows me 2-3 messages and then just hangs. I am able to switch it off/reboot it, it is not totally dead. Also, in order to boot successfully w/o waiting forever, I adjusted the PCIe OPROM in my system's bios to EFI instead of Legacy. My motherboard is Supermicro. So I am asking if the LSI bios menu is necessary or not? My cards are already okay now or am I missing a setting in the LSI BIOS? And if that EFI oprom option good/bad? *edit* I just noticed that my bios is not the latest. The latest is 7.39.02.00. The FW is the latest...shall I update the bios? Link to comment
JorgeB Posted January 2, 2018 Share Posted January 2, 2018 1 hour ago, Mat1926 said: So I am asking if the LSI bios menu is necessary or not? No, only if you need to boot form a device connected there. Link to comment
Mat1926 Posted January 2, 2018 Author Share Posted January 2, 2018 2 minutes ago, johnnie.black said: No, only if you need to boot form a device connected there. No I dont plan to do that. So based on the screens, am I okay? Link to comment
JorgeB Posted January 2, 2018 Share Posted January 2, 2018 Only thing that matters is IT mode and firmware 20.00.07 Link to comment
Mat1926 Posted January 2, 2018 Author Share Posted January 2, 2018 2 minutes ago, johnnie.black said: Only thing that matters is IT mode and firmware 20.00.07 Great...I will ignore the bios then. So it is okay to go ahead and boot unraid and start attaching my hdds then? Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.