So I am just beginning to build my first unraid server.  I was wondering if a person should wait a certain amount of time or run any utilities to ensure that you have a stable server?

So I am just beginning to build my first unraid server.  I was wondering if a person should wait a certain amount of time or run any utilities to ensure that you have a stable server?

You do not have to, but it is not a bad idea.

 

Run an overnight memory test, expect no memory errors.

Pre-clear all the disks, to eliminate those that fail early in their lives.

Run several parity checks... expect no errors in the syslog from the disks.

Perform md5 checksums on the files initially transferred, to determine if network errors or anything else is corrupting your files.

I agree with everything Joe L. said.  However, you can also use TeraCopy to transfer your files and let it run CRC checks on all your files after the copies complete.  This is essentially the same as MD5, just a bit easier.  Either will work, though.  And if you are truly paranoid you can run both

I like to copy files from one drive to another too. Of course I'm not talking a few meg at a time either. I like to rack up the gigs and let it go.

 

I used to have an old server that would often spit out errors and I later found out it was a controller card not a hard drive, but thats another story all together. I found it by shuffling files around....

I keep reading about pre clear but I keep wondering if it is a addon or built into unraid?  I just got it built and am letting it run memtest tonight 

Preclear is an add-on script written by Joe L.  You can find more information about it in the wiki.

 

Preclearing a disk is optional, though highly recommended.  It helps to minimize server downtime as well as catch defective drives before you trust your data to them.

So I am just beginning to build my first unraid server.  I was wondering if a person should wait a certain amount of time or run any utilities to ensure that you have a stable server?

You do not have to, but it is not a bad idea.

 

Run an overnight memory test, expect no memory errors.

Pre-clear all the disks, to eliminate those that fail early in their lives.

Run several parity checks... expect no errors in the syslog from the disks.

Perform md5 checksums on the files initially transferred, to determine if network errors or anything else is corrupting your files.

 

Server assembled  

Memtest ran overnight, no errors  

Purchased Pro license 

preclearing the harddisks now

 

Just wondering what your thoughts are:

Is it ok to have say 3 2tb drives with 1 2tb drive for parity then use a 1.5tb disk for a cache drive?

Just wondering what your thoughts are:

Is it ok to have say 3 2tb drives with 1 2tb drive for parity then use a 1.5tb disk for a cache drive?

yes perfectly fine, or if you have another smaller drive you can use that for cache  say an old drive under a TB works great.

 

-=Jason=-

So I have been watching the temps of the drives.  One drive hit 40C but currently they are all floating between 34C-37C.  Preclear has been running for around 5 hrs.  The drives are in a 5 in 3 icydock.  Do you think this is ok so far?

Yep, that sounds perfectly normal.  40C isn't anything to worry about during a preclear, as that's about the most activity you can expect from your server.  During day-to-day use I expect you'll see temps around 35C.

The disks are getting closer to finishing preclear (impatient).  All disks seem normal but there is one that seems to be a concern.  It took longer to perform step 2 (it was still on step 2 while the others were on the final reading stage) than all the other disks and is running hotter than the other disks.  43C as compared to 34-38C.  Could this be a sign of a potentially bad disk?  I am using putty sessions to perform preclear on 4 2tb disks simultaneously.  I am wondering is screenshot the only way to save the results for posting on the forum or does putty have some type of copy and paste to retreive the info?

 

 

Thanks

I am wondering is screenshot the only way to save the results for posting on the forum or does putty have some type of copy and paste to retreive the info?

No, the preclear results are saved in your syslog file (along with other important system info).  If you are running unRAID 4.5 or later you can access it at //tower/log/syslog.  Of course you'll need to put in a different name for //tower if you changed it from the default.  If you install unMenu it makes your life a little easier.  There is a page built in which pulls up your syslog and a feature that downloads the entire log as a text file.

Too early to tell if that's a bad disk or not, you have to look at the syslog once the preclear is finished.

The disks are getting closer to finishing preclear (impatient).  All disks seem normal but there is one that seems to be a concern.  It took longer to perform step 2 (it was still on step 2 while the others were on the final reading stage) than all the other disks and is running hotter than the other disks.  43C as compared to 34-38C.  Could this be a sign of a potentially bad disk?  I am using putty sessions to perform preclear on 4 2tb disks simultaneously.  I am wondering is screenshot the only way to save the results for posting on the forum or does putty have some type of copy and paste to retreive the info?

 

Thanks

Highlight area of interest, left click mouse to copy to clipboard.

right-click mouse in putty window to paste what is in clipboard.

 

 

Here is one disk sdc

 

 

============================================================================

S.M.A.R.T. error count differences detected after pre-clear

note, some 'raw' values may change, but not be an indication of a problem

63c63

< 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      25

---

> 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      26

============================================================================

 

another  sda

 

============================================================================

==

== Disk /dev/sda has been successfully precleared

==

============================================================================

S.M.A.R.T. error count differences detected after pre-clear

note, some 'raw' values may change, but not be an indication of a problem

54c54

<  1 Raw_Read_Error_Rate    0x002f  100  253  051    Pre-fail  Always      -      0

---

>  1 Raw_Read_Error_Rate    0x002f  200  200  051    Pre-fail  Always      -      0

63c63

< 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      24

---

> 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      25

============================================================================

 

another sdd

 

== Disk /dev/sdd has been successfully precleared

==

============================================================================

S.M.A.R.T. error count differences detected after pre-clear

note, some 'raw' values may change, but not be an indication of a problem

54c54

<  1 Raw_Read_Error_Rate    0x002f  100  253  051    Pre-fail  Always      -      0

---

>  1 Raw_Read_Error_Rate    0x002f  200  200  051    Pre-fail  Always      -      0

63c63

< 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      16

---

> 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      17

67c67

< 199 UDMA_CRC_Error_Count    0x0032  200  253  000    Old_age  Always      -      0

---

> 199 UDMA_CRC_Error_Count    0x0032  200  200  000    Old_age  Always      -      0

============================================================================

 

I have one more running.  Do you see anything wrong with these disks.  The one I am concerned about is still running.

 

Thanks

Bill

Those all look perfectly normal to me.

here is the final disk

 

== Disk /dev/sdb has been successfully precleared

==

============================================================================

S.M.A.R.T. error count differences detected after pre-clear

note, some 'raw' values may change, but not be an indication of a problem

63c63

< 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      24

---

> 193 Load_Cycle_Count        0x0032  200  200  000    Old_age  Always      -      25

============================================================================

 

Looks like you've got 4 good disks there.

In all three, the disk heads were loaded onto the disk platters once.

 

Typically, that is a good thing, because if the disk heads stayed parked it would be hard to read or write the disks.

Thank you so much for helping me setup my first unraid server.  Now I am moving all my movies onto the server using teracopy and I enabled the "test" (which I assume is to verify the that it is an exact copy, am I correct).

 

I setup the first share on the server.  It is of  "Movies".  The file structure is

 

Movies-folder with movie contained within for example I have the movie Avatar

 

Movies-Avatar-actual movie files.

 

So I names the share Movies

allocation method=most free

split level=1

export smb=read/write

I did not set anything else.  I see when it is copying file to the server the parity disk is active.  If I don't exclude the parity disk will it place files on the parity drive (my assumption was no but I figured I would ask).  Would you change/add anything to this share.

 

My next goal will be to figure out user security.  The idea would be to allow anonymous users to have read only to the movies share.  Then to write/delete to the movies share I want to assign permissions.  I have been reading on the assigning read/write permissions but it does seem some what confusing and it did not sound like the setting up anonymous read only was the most simple.  So would you mind assisting me with this next task?

 

Thanks again

I have been keeping an eye on the copying of the media files, I think every thing is going ok (takes a long time).  After doing more reading I changed the allocation to high-water.  I was not sure if it was working, because I thought it was only copying to 1 drive (drive 1) but it started copying to drive 2.  So now I am wondering if I need to specify "min space free".  I have a total of 4 tb disks, 1 for parity, 3 for data.  I don't want alot of unusable free space.  The media that is being copied s a mixture of dvd's and bluray's.  Any suggestions? 

Teracopy runs CRC checks on both the source data and the destination data.  If CRC matches, you can be confident that the data was copied to the server without corruption.  MD5 does a similar thing.  I trust CRC alone, but there's no harm in running MD5 as well if you want to be extra sure.  You can run MD5 once the transfers complete.

 

I support your decision to change the disk allocation method to high water, I feel it is the best option for most users.  Also, definitely don't use 'fill up' because there is a bug with it at the current time, a bug that causes unRAID to ignore the split level set for that share.  High water is great, though, just stick with that.  Remember that changing the disk allocation method will only affect future writes, it won't shift your data around for you.  I don't know if the change will kick in mid-transfer, or if it will only apply to future transfers.

 

I believe that you'll actually want a split level of 2 or higher.  A split level of 1 means that the files within the 'Avatar' folder can be split across different disks, which I expect you want to avoid (this can cause lags in video playback from a multi-file video, such as a Video_TS folder).  I believe I have my Movies split level set to 6, which is probably higher than I need.  However, some of my movies have multiple nested folders, such as:

Movies

-Ocean's Eleven

--Ocean's Eleven [1960]

---video file

--Ocean's Eleven [2001]

---video file

--Extras

---Behind the scenes, etc.

 

So I would rather have a split level that is sufficiently high to ensure that no single movie folder will ever split across disks.

 

How do you play your movies?  If you are using an HTPC, you may want to set your Movies share to export as smb: read only.  This means that you can't accidentally delete a movie from your HTPC (though perhaps you want that ability, I don't know).  You can then enable user level security, add a new user account with a password, and set an exception on that share for that user.  For example, I have two user accounts:

guest (no password)

admin (passworded)

 

My Movies share is set as smb: read only with an exception for the admin account.

 

My HTPC is set to always log in as guest.  I can play any movie, but not rename or delete it.  My desktop is set to always log in as admin.  From there, I can do all my file management (delete crappy movies, rename files, etc.).

 

This also gives me the confidence that anyone logged into my server as 'guest' can't mess with my files.  So if my roommate wants to watch some TV off the server on his laptop he can do so, but he can't accidentally delete any of my files.  I've also set some of my more personal shares (backups, pictures, documents, etc) so that only the admin account can access them.

 

You seem to have the wrong idea about how the parity disk works.  Files will never be written to the parity disk.  Instead, the parity drive is updated whenever you write a file to any of the data disks.  This is what allows you to recover from a single disk failure.  When you are reading the file back, the parity drive should not be involved as long as your array is healthy (no missing or disabled disks).

 

I believe I've answered your user level security question above, but feel free to ask for clarification or further examples.

 

You should specify your min free space to be the same size or larger than your largest individual file.  So if you largest file in your movies share is a 8 GB 1080p mkv file, then set your min free space to be 8 GB (8,000,000 kb).  Alternatively, if you largest file is a 60 GB BluRay ISO, then set your min free space to be 60 GB (60,000,000 kb).  I know this seems like a lot of wasted space, but there's a serious benefit to doing this - it allows the Reiser File System (which is what unRAID uses) to replay journaled events and potentially recover corrupted data on it's own.  Certain catastrophic events such as a sudden power loss can cause data corruption, and ReiserFS is able to recover from that situation on it's own sometimes (without even involving parity).

 

Note that the min free space advice above is optional - your server will still work even if you fill all your data disks to the brim.  However, it is highly recommended.  Think of it as a second layer of data protection built into your server.

I believe that you'll actually want a split level of 2 or higher.  A split level of 1 means that the files within the 'Avatar' folder can be split across different disks, which I expect you want to avoid (this can cause lags in video playback from a multi-file video, such as a Video_TS folder).

I don't think this is correct.  If you have a Movies share then setting a split level of 1 will make certain that the contents of any folder placed inside of the Movies share will all be on the same disk.  If you set the split level at 2 then the contents of the folder placed inside the Movies share could be spread across all the disks in the Movie share.  Check the unRAID manual for the explanation of split levels.  The important point is that no matter how you have your shares folders structured you want to set the split level high enough to be sure that you are not spreading the files within an individual movie folder (i.e. Avatar, Gladiator, Transformers, etc.) across multiple disks.  Let me see if I can show an example.

 

Split Level 1

- Movies Share (includes disks 1, 2, & 3)

- Avatar (disk 1) - everything in this folder must be on disk 1

  - VIDEO_TS.BUP (disk 1)

  - VIDEO_TS.IFO (disk 1)

  - VTS_01_0.BUP (disk 1)

  - VTS_01_0.IFO (disk 1)

  - VTS_01_1.VOB (disk 1)

  - VTS_01_2.VOB (disk 1)

  - VTS_01_3.VOB (disk 1)

  - VTS_01_4.VOB (disk 1)

- Gladiator (disk 2) - everything in this folder must be on disk 2

  - VIDEO_TS.BUP (disk 2)

  - VIDEO_TS.IFO (disk 2)

  - VTS_01_0.BUP (disk 2)

  - VTS_01_0.IFO (disk 2)

  - VTS_01_1.VOB (disk 2)

  - VTS_01_2.VOB (disk 2)

  - VTS_01_3.VOB (disk 2)

  - VTS_01_4.VOB (disk 2)

 

Split Level 2

Movies Share (includes disks 1, 2, & 3)

- Avatar (disk 1)

  - VIDEO_TS.BUP (disk 1)

  - VIDEO_TS.IFO (disk 2)

  - VTS_01_0.BUP (disk 3)

  - VTS_01_0.IFO (disk 3)

  - VTS_01_1.VOB (disk 2)

  - VTS_01_2.VOB (disk 1)

  - VTS_01_3.VOB (disk 1)

  - VTS_01_4.VOB (disk 1)

- Gladiator (disk 2)

  - VIDEO_TS.BUP (disk 2)

  - VIDEO_TS.IFO (disk 1)

  - VTS_01_0.BUP (disk 2)

  - VTS_01_0.IFO (disk 3)

  - VTS_01_1.VOB (disk 2)

  - VTS_01_2.VOB (disk 1)

  - VTS_01_3.VOB (disk 1)

  - VTS_01_4.VOB (disk 3)

 

Hope that makes sense.  Perhaps I have interpreted the unRAID manual incorrectly but I have my shares setup exactly as hunter69 proposed with a split level 1 and none of my individual movies are spread across multiple disks.

You are right, wsume99.  I always forget that the Share directory doesn't factor into the split level.  So a split level of 1 is correct for both of you.

 

I need to stop attempting to answer split level questions, I always mess them up