HDD array with SSD parity - large number small parallel writes

[Zeon]

Hi all,

I am interested to consider using UNRAID to store some very old data (that could be considered archive only). The main motivation is cost, we have an UNRAID server using older desktop drives (I.e. no TLER) and it is stored in our data room at the office rather than the datacentre - so cheaper power and practically no hosting costs.

 

The data is a series of small images and text files 10-100KB of which we have 10,000,000s. I was wondering that, lets say we had 24 drives in our enclosure, that each could be written to in parrelel? In such a case, if we switched the 2x parity drives to SSDs then it would seem to be logical that we wouldn't limit our parallel write speeds by the parallel drives. Let's say that each HDD could do 100 IOPS, if we times that by 24 we would get to 2,400 IOPS - substantially less than most SSDs. I don't known the standard block size of UNRAID filesystems but I am guessing that we should have at least some sequential block writes with say a 50KB file so maybe better than pure random IO too.

 

I saw this article but it seems to be a single write where there wouldn't be an advantage of SSD IOPS improvements vs my use case:

https://www.spxlabs.com/blog/2020/11/27/ssd-parity-vs-hdd-parity-vs-ssd-array-in-unraid

 

The data will be migrated from a range of ZFS NAS servers and a 7x server ceph cluster. Data is stored in a automatically generated folder structure with paths stored in an indexed database with file metadata.

 

Comments welcome.

 

 

[trurl]

Just go without parity until archive done then add and build HDD parity after 

[Zeon]

11 hours ago, trurl said:

Just go without parity until archive done then add and build HDD parity after 

Thanks for the suggestion, this is an option. While the data is archival data we would prefer not to lose it and have some unease about it sitting only on a single unprotected disk since our scripts that will move it in will delete from the source location as soon as its moved. This means that without parity there would be no protection for potentially weeks as all the data is moved in.

I guess my question is, will my objective of all drives being able to write at their max IOPs be feasible if the parity drives are SSD?

[JorgeB]

It might be, depending on the SSDs users, fast NVMe devices would be better.

[trurl]

14 hours ago, Zeon said:

our scripts that will move it in will delete from the source location as soon as its moved.

Parity is not a backup. You must always have another copy of anything important and irreplaceable.