Should I buy a NAS ore build my own Unraid nas, concerned about write speeds. HELP!!

[Hrridly]

Hello people and sorry for crashing the party so abruptly so please forgive me if I have posted in the wrong place as I am just looking for some pointers before I either buy a NAS or build my own using Unraid. I was thinking of purchasing the below.  These nas devices cost roughly £3000 pounds sterling or $3680.

 

So, I got thinking I could build a system myself will top of the shelf parts for half the money and have a home self-build nas as good as the ones mentioned below.

 

With warranty and unable to repair parts it is a worry what happens when the warranty dies if your device dies so does your data along with-it bad situation to say the least.  With a self-build just replace the part and carry on from where you left off well maybe I am sure it is possible.

 

The nas units I was considering buying.

 

QNAP TVS-h1688X QNAP TVS-872X-i3-8G 8-Bay NAS

QNAP TVS-872X-i3-8G 8-Bay NAS

Asustor Lockerstor 10 Pro AS7110T 10 Bay NAS Enclosure

 

For the past four weeks I have been reading up on Unraid software and the benefits of building my own nas server.  I have always built my own gaming pc’s so apart for the Unraid software learning curve should be an easy process you know what I mean when I say easy still a steep learning curve but one which I will learn slowly and just take it bit by bit until I have a handle like everyone else you have to start somewhere.

Anyway, I purchased a Qnap TS8753A nas to figure out what they are capable of  I installed two m.2 NVME segate 510 2tb Firecuda drives for the cache, and a qnap 10gbe ethernet card to connect to the 10Gbe port on my motherboard as I prefer the direct connection without any internet related connectivity for security and peace of mind so I only need a connection between PC & NAS.

 

I was able to achieve a write speed using the 10Gbe without cache 271Mbs which was disappointing to say the least could not figure out what was going on everything pointed to 10Gbe connectivity as I checked it in the Qnap software and on my PC, everything pointed to 10Gbe active but was not getting 10Gbe speeds might have been the processor on the nas which was an AMD Ryzen™ Embedded V1500B 4-core/8-thread 2.2 GHz processor, anyway installed and activated the cache and it went to 1100MB’S write and 1249Mbs read.

 

So, my question is this before I go out and buy all the aprts to build my nas and I would be  buying top of the range parts to build my own beast of an  Unraid server that will last me for years for backup of my media movie data, which will be around 65Gb a file will I be able to achieve the write and read speeds of say my Qnap TS873A with cache.

 

Has anyone experience of the maximum read and write speeds using Unraid  with conjunction of using a 10Gbe ethernet connection between the Unraid server and the pc direct connection as this is the only situation that is suitable for me between my PC and my nas.  Just don’t what to go paying a fortune on a load of electronics to arrive fit them all together to be underwhelmed again as I have read people reporting slow write speeds the Asustor Lockerstor 10 Pro AS7110T promises 1100 write speeds using m.2 nvme though it costs £2299 with 10 bays and is surely way over priced for what it is to say the least. That is why I would love to buld the same server but for half the cost and be 4 times better in build quality than the Asustor Lockerstor 10 Pro AS7110T makes sense right but need the write speeds to be where I wnat them. 

 

Looking forward to your reply.

 

I will appreciate anyone who reads my post and replies with a positive reply as I would have to purchase the Unraid pro Pack for the number of hard drives I would eventually be installing into my server all Segate 18Tb Exos hard drives.  Sorry for the long post but wanted you all to know my story where I am coming form and where I want to end up.  Thank you to all who reply.

[trurl]

https://wiki.unraid.net/Manual/Storage_Management#Performance

 

[trurl]

https://wiki.unraid.net/Manual/Storage_Management#Pool_.28cache.29_Operations

[Hrridly]

Nice forums guys and thanks for all the valuable advice. I cant really take in how many members gave real good advice on buying a ready made system or going it on my own with unraid but with many reports of slow writes maybe i am better off sticking with a pre built.

 

That answers my question. 

 

Farewell folks.   Dont be talking about me now ya’ll.

[JorgeB]

Write speed with Unraid depends mostly on the hardware used, also if you just want to use the array or can use pools.

 

For the array, and using turbo write, if there are no controller or other bottlenecks write speed will be limited by the slowest disk in the array in that zone, e.g., with fast empty disks write speed will be around 200/250MB/s.

 

For pools, using a multi device SSD pool or a fast NVMe device/pool I get around 1GB/s with 10GbE.

[Hrridly]

Thank you very much for your reply I think I could actually live with that write speed should not be a problem at all it was just I was expecting write speeds of 35 to 56Mbs and obviously that is not the case just needed to have someone say it even though I have been reading the forums for ages trying to get answers.  So I will maybe have A go at building my very own Nas server.  Will be a challenge and and exciting road ahead buying all new stuff and having the raid array working will be a great accomplishment to have done it for myself.

 

Thank you for the reply thats all I wanted to read.  I am sure I will make a big boo boo so will play about with the software before storing data on it to get a feel to how it works before I commit data to it and maybe do something and delete a hard drive or something silly.

 

Great job well pleased. Just one more last question I dont have to write code or anything of that nature to change settings it is a Linux operating system shell like windows desktop point and click.

 

All the very best.

[Hrridly]

Just to clarify I will be adding a SAS controller so will that slow the array down. Anyone.

[JorgeB]

11 minutes ago, Hrridly said:

35 to 56Mbs

I assume you mean MB/s, Mbs is Mbits per second.

 

Also note that without turbo write array writes are considerably slower, again depending on the hardware, but speeds from 40 to 60MB/s are normal, more details about both write modes here:

 

 

[Hrridly]

Thank you I shall go and inwardly digest.

[nasforthemass]

You got answers from 2 pretty experienced members. nice! maybe you'd like to hear some feedback from someone that was in your position not too long ago...
Before i do, I would like to make sure your understanding and expectation of data read/write speed is on the same page of different raid set-ups and how your storage is connected/managed. I only say this because I run LARGE 80TB+ storages in both of the following setups.

DAS is "direct attached storage".  NAS is "network attached storage". It seams like you are trying to create a NAS that works as a DAS.
BOTH a DAS and NAS can run a traditional raid set-up, but I am only aware of a NAS being able to run other methods of raids. The basic "storage" foundation of UnRaid (this OS) is as follows:

• you can have multiple HDDs of mixed capacity and speeds all work in unison, while still having 1 or 2 drives as Parity
• you can add more drives and substitute drives at any time, with minimal work
• a file is written entirely to one drive only, meaning that drive can be read individually without needing the other drives from your system
• you can create a faster drive pool/cache for higher speed volumes

Let's pretend you only have 10x 18TB Seagate Exos in your UnRaid system and you choose 1x  of those drives as parity. (we'll save the SSD's and cache for later)  Your Raw storage is 180TB, but your Volume/Usable storage would be 162TB. (-1x 18TB drive used for parity)  Imagine you had a folder called "Movies" with 999 files/folders in them, each one around 65GB. Although you would see the "Movies" Folder and everything in it like a traditional single HDD/SDD, the folder would actually be physically across all the drives without "splitting" the individual files; an easy hypothetical would be:
999 movie files split between 9 drives (because the parity doesn't store the useable date)... meaning each drive would have around 111 movies stored on it. Cool? Have we understood the basics of it?
Let's say you want to transfer/read/copy one of those movies files: The main limiting speed will be the drive that it is stored on. Yes, a Seagate Exos can do upto 260MB/s, but that is only a portion of the HDD that can read/write that fast. In the real world, some parts of that same Seagate Exos will only get around 50-80MB/s. Different areas of a HDD have different read/write speeds (depending on how full it is). A good SSD on the other hand, can read/write almost the same speed on any part of the drive.

Good practice for users new to UnRaid: 
Not all storage needs to be wicked fast! But even if you do want higher speed storage, UnRaid has other tricks up it's sleeve, like cache pools, being able to mix in SSDs, creating folders that only use the faster storage array and other folders being stored only on the "slower" HDD setup. 
Example from my setup: (Notice how there are 3 levels of speed/importance)
I have 8x 8TB of HDDs. I store pictures, videos and back-ups on there. Especially older files that i don't need regularly. (ski vacation pictures from 8 years ago)
I have an 4TB SSD that only has 1 folder in it and the data in that folder doesn't get split to the HDDs. This is where I keep recent pictures that i don't need on my laptop.
I have a 800GB PCIE/NVME storage that is used as a write cache. If i transfer 50GB of new pictures and drone footage to my Unraid, this storage is the fastest at receiving it, and then transfers it to the SSD or HDD (depending what folder I choose) at a pace that suits it. If I were to transfer the 50GB of new files directly to the HDD volume, my transfer speed would be limited by the write speeds of my drives. (They're around 88% full, so its around 100MB/s


UnRaid is awesome, but understanding it's core functions are pretty important when it comes to your expectations and usage. BUT you're right. there is a learning curve to being a confident user. there are lots of functions, settings and best-practices that will improve or make your server work best for you!
If you build your own system, you will learn more about system bottlenecks (when one component is slowing everything else down) and solving this. 

PS, I built both my systems from used server parts one bay (from reliable vendors). the first system was great, i learned a lot. but i then built a second system because i saw the bottlenecks/limits of my first system and was more informed on how to build a more up to date one.

Edited June 23, 2022 by nasforthemass

[Hrridly]

Well kind Sir thank you ever some much for the explanation actually had to read that twice to fully understand it all now as they say the penny has dropped what you are referencing as to what I would want to achieve. I do believe that building my own NAS is the way to go only because these Qnaps, Synology and all the rest of the NAS manufacturers for home use not professional and what they are charging is a total rip off to me at any rate.  Anyway without getting side blinded here you are correct I only want a connection between my main PC because that is where I will be transferring my backups import documents, business data, invoices, bank statements, movies, pictures backups my digital life it soon builds up to something very massive it will be a depository with many files named for different purposes.  You have opened my eyes about creating these folders suppose I would have wanted to do that at any rate, did not know you can allocate to different hard drives now that is clever this will be a long slog figuring it all out as you have made me aware and you can only do that by delving into it and actually doing it for real in the real world to begin as a practice setup then if I cock up I can just start from the beginning again without losing any data, just to get a feel for how it all works maybe by a stroke of genius it will work first time and no upset. I love building PC’s I love tinkering this is a dream situation I love struggle and figuring out how it all works this is all new to me the world of NAS. 

 

I appreciate the reply I really do I can not thank you enough as it is a nervous process as there are a lot of parts to buy. Can I just be a nuisance one more time concerning HBA cards I was thinking of buying the Broadcom 9305-24i SAS Controller then I read you can buy a cheaper flashed for IT mode HBA card and pair it with a expander far cheaper than buying the 24 controller card was going to do it that was to have less pcie cards hanging off the inside of the motherboard.  I take it if that HBA card went bad can I buy a replacement and everything should be there or would I lose the whole lot now that is something I have just thought right there, scary indeed.

 

As for connection would I be better to buy a 10GBE pcie card or upgrade to 40Gbe between my nas and pc I ahve currently a 10Gbe on my pc motherboard suppose a 10Gbe card for the nas would suffice now please dont laugh I am only putting that out there would the 40Gbe get me any faster speeds or as you explained it is all to do with the speed of the hard drives but then using Cache will speed that process up somewhat then the cache can do its thing later on at a later time on its own so to speak transferring data to the array.

 

Thats a wrap and now for the very long walk into this whole build will take time picking all the parts only because life is busy and very frustrating infuriating actually but that is whole different story.

 

Thank you for dropping by and the the advice this is what I was hoping for. So thank you to everyone who has replied for that I am very appreciative I truly am as without this encouragement it would be difficult to continue and go it alone.

 

 

[trurl]

2 hours ago, Hrridly said:

long slog figuring it all out

The Overview is a good place to start. 

 

https://wiki.unraid.net/Manual/Overview

[nasforthemass]

Well, I'm glad some of that info helped you out. And yes, there are many options and tricks to learn...but really, it's more important to understand your individual needs and how to execute them. There is a FANTASTIC youtube channel you need to follow/watch to expand your knowledge!
https://www.youtube.com/channel/UCZDfnUn74N0WeAPvMqTOrtA/videos  (this guy is a legend!!!!!)

Regarding building your own setup:
There is a builds section in this forum and there are specific forum areas just for hardware
https://forums.unraid.net/forum/9-hardware/
https://forums.unraid.net/forum/15-motherboards-and-cpus/
https://forums.unraid.net/forum/33-storage-devices-and-controllers/
^^this last one, "storage devices and controllers, will be pretty helpful for you regarding the HDD connections.

If you would like a little info from a different source, I used this website/group a TON, because they are more interested in used datacenter and small office server parts vs new and off the shelf consumer parts. I know you have a decent starting budget, but the reason this is helpful, is because server parts are usually built to last longer, have more customizations and built for large scale setups. 
https://forums.serverbuilds.net/c/builds/18
https://forums.serverbuilds.net/t/guide-anniversary-2-0-starter-kit/1075/8   (lots of helpful info like this)
Example: My motherboard can handle 14x Sata/Drive connections directly from the motherboard, no HBA/LSI card required. 

Here's your homework before buying your components:
-Watch a ton of SpaceInvaders videos, especially the ones that sound a little more basic. you will learn many things even if its repetitive.
-Have a "build plan". Watch the video below, 320TB video editing server. This is a real world scenario with a game plan.
-Focus on your personal needs: how much space do you have, how many drives will you need to fit in the case, do you need to be energy efficient, do you need it to be super quiet, is heat an issue??? All of this will change the components you will use.

-Will your UnRaid box be for storage only, or will you try and use the VMS and Docker functions? big difference in components.
-RESEARCH your Motherboard/cpu combo!!!!!! this will be the limiting factor to your set-up scale/potential. Example: My current motherboard has 2x 10GB ethernet built-in, 14x sata connections, LOTS of ram upgradeability, Lots of PCIE lanes, etc.. (I wanted 10GB ethernet on-board, I wanted to avoid an LSI card so that I had more PCIE slots for GPUs and PCIE storage AND I wanted to be able to have lots of CPU options)

PS: I think you will be fine with a 10GB ethernet connection. It doesn't sound like your use case will need more than that and 10GB = LESS HEAT than 40GB. (if you are doing big transfers from drives that already have info on them, you can add/mount those drives inside the server as a separate "unassigned device" and copy that data internally) i did this when i first setup my raid and needed to copy 4x 4TB HDDs into the UnRaid volume, vs connecting them from my computer and transferring them via 10G ethernet/SMB file share.
 

 

Edited June 23, 2022 by nasforthemass

[nasforthemass]

I didn't have time to find my spreadsheet, but creating something like this will help you with your homework of "create a build plan". 

[mathomas3]

If pure speed is what your looking for, TrueNAS might be worth looking into... it's more of a enterprise setup with a number of limitations... The key one for me is the inability to add disks to the array...

 

I have been using UnRaid for around 10 years now and it has been a very solid system that has expanded a lot over time. My one tip for you is to consider a larger case then you currently need... I started with a 8 bay case and migrated that over to a system that could hold 18 drives and just recently decided to move to full-on enterprise (recycled) hardware using a DAS that can hold 28 drives... 

 

Something to consider

 

Oh and label your drives before putting them into your case... makes finding a failed drive ALOT easier!

Edited June 23, 2022 by mathomas3

[Hrridly]

Thank you guys for all the information that is some amount of homework I was not aware you could get a motherboard with 14 sata  ports that is something which I did not realise you could do I assumed it had to be through the HBA card but now I see that is only for instances when you run out of Sata ports makes more sense now see things are becoming much clearer the more I read into this though I was wanting to end up with the ability to be able to add 24 hard drives maybe I would never ever reach that amount maybe 10 to 14 max is being more realistic.

 

Thank you nasforthemass for that excellent post God bless you Sir you are surely doing everything you can to make this whole process as easy as possible for me.  

 

Ok I think I have the grasp of it all I will now read all this information and start planning my build.  If I get stuck I will come back and give a hollar.  Gees this is fantastic cant believe the help I have received much more than what I expected you are truly fantastic people on here.  For that I am feel very humbled.  Thank you.

 

 

[Hrridly]

1 hour ago, mathomas3 said:

If pure speed is what your looking for, TrueNAS might be worth looking into... it's more of a enterprise setup with a number of limitations... The key one for me is the inability to add disks to the array...

 

I have been using UnRaid for around 10 years now and it has been a very solid system that has expanded a lot over time. My one tip for you is to consider a larger case then you currently need... I started with a 8 bay case and migrated that over to a system that could hold 18 drives and just recently decided to move to full-on enterprise (recycled) hardware using a DAS that can hold 28 drives... 

 

Something to consider

 

Oh and label your drives before putting them into your case... makes finding a failed drive ALOT easier!

Yes I believe TrueNas Scale is supposed to remedy that scenario as they are mooting you will be able to add a disc to the  array to make it larger in size, though they do not know for sure how it will be implemented as it is a complicated process to go about it it is not point and click so to speak you have to have a degree of knowledge and know your way around the software is what is being talked about saw A video on You tube some place explaining that particular dilemma they said it would be a Synology and Qnap killer, that is how good it is.

 

Yes I looked into TrueNas but as you say you have to start out with all your hard drives suppose I could buy 14 hard drives as I have 5 now thats another 9 I could swing for another 9 hard drives costing £245 a pop as that is all I pay for 18 Nas server grade hard drives brand new with full warranty purchased a Western Digital pro red 18Tb for £255 they are selling at over £500 I love a bargain you see I spend days searching on e bay you do hit lucky sometimes purchased a Western Digital Gold 18Tb for £215. 

 

I doubt I would entertain TrueNas for the reasons already explained but it would give you the same speeds or writing and reading as a standard Qnap or Synology device. If they crack the being able to extend the array with a few clicks I think that will sound the death knell for nas manufacturers like of Synology or Qnap or they would have to become much more competitive all of a sudden. We shall see have to keep an eye on developments to see how it all transpires over time, that is all I know about TrueNas and would have no interest in using their software as it currently stands.