Jump to content

Using unRAID for primary file storage server - recommended?


Loosley

Recommended Posts

I read through the 'how do you use unRAID' thread, and it seems like most people are using it for a media streaming server.  I'm new to unRAID, and I'm planning on using my server as my primary media storage (photos and HD videos of family) as well as for file storage for work documents.  The unRAID server will be backed up to another PC (but unRAID will be the 'primary' source for the files).

 

I am currently using unRAID 5.0beta13 since the Intel 82579 LAN controller on my mobo (Asus P8Z68-V PRO) does not work on v4.7, but I have a concern with using a non-stable release for storing important files.  I guess my concern is: what if unRAID (silently) corrupts my files?  My backups of the unRAID server would consequently be corrupted as well (since the unRAID server is the primary source of the data).  Is this a valid or likely concern?  Has this happened in the past?

 

When is a stable 5.0 release expected?

Link to comment

I read through the 'how do you use unRAID' thread, and it seems like most people are using it for a media streaming server.  I'm new to unRAID, and I'm planning on using my server as my primary media storage (photos and HD videos of family) as well as for file storage for work documents.  The unRAID server will be backed up to another PC (but unRAID will be the 'primary' source for the files).

 

I am currently using unRAID 5.0beta13 since the Intel 82579 LAN controller on my mobo (Asus P8Z68-V PRO) does not work on v4.7, but I have a concern with using a non-stable release for storing important files.  I guess my concern is: what if unRAID (silently) corrupts my files?  My backups of the unRAID server would consequently be corrupted as well (since the unRAID server is the primary source of the data).  Is this a valid or likely concern?  Has this happened in the past?

There is nothing wrong with that use.

While I tend to use my unRAIDs mostely for media storage (media storage can be friggin huge!!), long term storage (files i don't use often like installers) and also backups...

 

There is no reason to not use unraid like a giant NAS. After all, it is a file server is it not?

with the added value of parity protection, it would much better then some off the store shelf single drive NAS appliance.

 

Your concern of file corruption is valid. While i have never had a file go bad while it was in my protected array yet (that i am aware of).

I HAVE HAD file corruption happen on my unRAID from a bad cache drive (once the bad drive was replaced, I never had that happen again).

For mission critical files, I would bypass the cache drive entirely and write straight to the array. I would also verify file copies with a program like "Beyond Compare" at first until you are comfortable.

 

As far as backups, and backing up corrupted data, however unlikely as it is, it can happen.

 

If your photos are that important. i would recommend copying them to a USB drive directly from your desktop and storing the USB drive (offsite is best).

also look into some sort of online storage for the data you covet most, perhaps crashplan or similar.

more then 1 copy is a must this day and age.. all hard drives DO fail. my work files and photos are in at least 4 places... and all copies are done automated. I don't have to think about it, i just know they are there.

 

 

When is a stable 5.0 release expected?

It's ready when ts ready....

it was on the fast path to being close to RC1 when some kernel/driver issues popped up.

 

 

 

EDIT:

one thing you could do if you plan to use a share more like the "My Documents" folder, a 7200 RPM parity and 1 or 2 data drives dedicated just to that share might help with any possible speed issues.

Link to comment

There is nothing wrong with that use.

Your concern of file corruption is valid. While i have never had a file go bad while it was in my protected array yet (that i am aware of).

I HAVE HAD file corruption happen on my unRAID from a bad cache drive (once the bad drive was replaced, I never had that happen again).

For mission critical files, I would bypass the cache drive entirely and write straight to the array. I would also verify file copies with a program like "Beyond Compare" at first until you are comfortable.

 

As far as backups, and backing up corrupted data, however unlikely as it is, it can happen.

 

If your photos are that important. i would recommend copying them to a USB drive directly from your desktop and storing the USB drive (offsite is best).

also look into some sort of online storage for the data you covet most, perhaps crashplan or similar.

more then 1 copy is a must this day and age.. all hard drives DO fail. my work files and photos are in at least 4 places... and all copies are done automated. I don't have to think about it, i just know they are there.

 

EDIT:

one thing you could do if you plan to use a share more like the "My Documents" folder, a 7200 RPM parity and 1 or 2 data drives dedicated just to that share might help with any possible speed issues.

 

Thanks so much for the detailed response John, and also for the heads-up about the cache drive.  I will forego the cache drive.  My data integrity is my primary concern, and so I can live with slightly slower speeds if it means better robustness.

 

Here is my intended use for my unRAID server.  I currently have two PCs which need access to the same files, and so instead of having local copies on each PC, I was going to put the files on the unRAID server.  Most of these files are photos and videos.  For backup purposes, I was planning on having local copies to a second unRAID server, plus remote external copies via CrashPlan or maybe USB drives that I physically transport offsite.  Does this seem reasonable to you?

 

For file comparison, I am currently using md5deep to create and verify MD5 checksums to verify that files were backed up successfully.  I haven't used Beyond Compare; I'll have to check it out.  I wonder which is faster, md5deep or Beyond Compare?  BC seems to be more oriented to code comparison...

Link to comment

sounds like a plan. keep it simple, yet useful.

 

I am not saying a cache drive is bad. I have an SSD cache drive on both of my unraid boxes now. just know there is a potential for problems.

 

I always say there are two types of people out there; those with backups, and those that wish they had one.

 

Beyond Compare does have robust code and document comparison now. it started out as a file copy program for syncing things like servers with a binary comparison option. it has since become a bit of a multi headed beast.

 

While BC has MD5 option like your MD5Deep, it can also do a binary compare so you know every 1 and 0 is correct.

I would assume BC is slower.

Link to comment

I personally use my unRAID as the central hub to my house.

Photos

Music

Movies

Wifes Scanned Recipes

Data files

 

You name it I have it on there. I do however keep backups of my photos offline as well just because they are a few of the un-replaceable items if there was something to go wrong.

Link to comment

UnRaid is my primary (Only?) storage at the moment. I am on 4.7, mainly because i have seen people having issues with the HBA(s) i use.

Hopefully Tom will find the cause (or maybe have to wait for the Linux Kernal to be updated again :( )

 

TBH, i've been surprised by how robust UnRaid is. I've had what i think is a faulty SATA cable and had 1 drive reporting 1000's of errors and dropping out of the array.

Changed the cable, ran the parity check and all is good :)

 

No matter how good or bad UnRaid is, if your data is important to you it should be backed up more than once.

ie, UnRaid, External HDD(s), online storage, offsite storage

 

Currently have ~4.5TB of data on my 11TB Array and have not had UnRaid corrupt anything that i know about.

Link to comment

If silent corruption on the filesystem is a top concern for you might be worth looking at alternatives that proactively address that (zfs being the obvious one).

 

Whilst unraid won't attempt to corrupt your data on purpose, it doesn't do anything specific to avoid it either in the way zfs / something that does checksums at the fs layer does.

 

However if you can build a workflow that works for you using md5 then at least you'll know when a file is corrupted. Though that doesn't necessarily help you with fixing it...

Link to comment

I use  beta13 (with two 3TB hard drives)  unraid server as my main storage area (my docs ,my videos,my pictures all point to the unraid server)

 

Been running it on version 5 for 4months.

Prior to that version 4 for 3 months.

 

No issues to report (other than if the tower is switched off, your machine on booting up takes ages ,as if its trying to connect my docs to the tower)

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...