Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


Recommended Posts

I recently had a unclean shutdown and I thought it might have had something to do with UD.

 

What happened was I had mounted a drive to copy data off of, and when the transfer was done I forgot to unmount it before I pulled it.  I plugged another drive into the same spot, got nothing, moved it a slot over, and continued transferring data from the next drive.  After the transfer started I had that "...damn I'm a idiot" moment and released what I did.  So after the second transfer finished, I unmounted that drive and then hit "power down".  This is when the unlcean shutdown happened.

 

I'm not sure if it had anything to do with UD but its the only weird thing that happened during that session that I can think of.  I posted on general support and the pointed me this way so here I am.  I'm too stupid to figure out how to dig through the diag zip it gave me, and I can't seem to figure out exactly why I had the bad shutdown.

Link to comment
5 hours ago, FrostyWolf said:

I recently had a unclean shutdown and I thought it might have had something to do with UD.

 

What happened was I had mounted a drive to copy data off of, and when the transfer was done I forgot to unmount it before I pulled it.  I plugged another drive into the same spot, got nothing, moved it a slot over, and continued transferring data from the next drive.  After the transfer started I had that "...damn I'm a idiot" moment and released what I did.  So after the second transfer finished, I unmounted that drive and then hit "power down".  This is when the unlcean shutdown happened.

 

I'm not sure if it had anything to do with UD but its the only weird thing that happened during that session that I can think of.  I posted on general support and the pointed me this way so here I am.  I'm too stupid to figure out how to dig through the diag zip it gave me, and I can't seem to figure out exactly why I had the bad shutdown.

UD will unmount all devices when unRAID is shut down.  You wouldn't have to unmount your device before you shut down unRAID.  What probably happened is that the device you pulled and didn't unmount hung the shutdown.

 

When unRAID is shutting down there is a timer that runs that will force a shutdown if things are hanging.  When that happens, unRAID will attempt to write diagnostics to the /boot/log/ folder.  See it the diagnostics were written there and post them and we can see more of what happened.

Link to comment
6 hours ago, dlandon said:

UD will unmount all devices when unRAID is shut down.  You wouldn't have to unmount your device before you shut down unRAID.  What probably happened is that the device you pulled and didn't unmount hung the shutdown.

 

When unRAID is shutting down there is a timer that runs that will force a shutdown if things are hanging.  When that happens, unRAID will attempt to write diagnostics to the /boot/log/ folder.  See it the diagnostics were written there and post them and we can see more of what happened.

 

These?

wolflix-diagnostics-20180304-1040.zip

Link to comment
2 hours ago, dlandon said:

The best I can tell from the log is that the disk you unplugged without un-mounting caused the problem.

Thank you!  I haven't had it happen again and I was pretty sure that it was my fault that it happened, but thank you for clarifying for me.

 

While I am here in the Unassigned Devices thread, I plan on having a disk off-array for downloading/processing before moving the final files to the array.  Is there a particular format for my disk that I should use for that? 

Link to comment
4 minutes ago, FrostyWolf said:

Thank you!  I haven't had it happen again and I was pretty sure that it was my fault that it happened, but thank you for clarifying for me.

 

While I am here in the Unassigned Devices thread, I plan on having a disk off-array for downloading/processing before moving the final files to the array.  Is there a particular format for my disk that I should use for that? 

xfs would probably be best.

  • Upvote 1
Link to comment

I typically use this plugin to backup key data to hard disks for offsite storage. It works very well for the most part, but the challenge comes in recognizing whether adequate space remains on the backup disks as the data on the server grows. So I've been tying to figure out how to use the plugin's AVAIL variable in my scripts to both log and notify the webgui as to the amount of space available when the disk is mounted and how much remains after the backup is completed. My scripting skills/knowledge are pretty minimal and so far I haven't been successful. Advice would be very much appreciated!

Edited by lewcass
Link to comment

Not sure if anyone has run into this; but my symptoms were the following: 

 

500GB SSD formatted xfs via unassigned devices, when performing LARGE constant file transfers TO the drive (writes) the performance would be great for the first 80GB of a constant transfer but shortly after 80GB the performance would plummet and the drive would almost become not useable.  

 

Thought it was the firmware as this is an 840EVO; so updated that, tried again; same issue.

 

One of my co-workers remembered a long time ago that SSD's needed a certain alignment for the best performance.  I read through some white papers from IBM on theirs as well; and it was recommended to check alignment via "parted" command 

align-check opt n

Where n is the partition.  I kept getting that it was not aligned.  

 

I pulled the SSD and stuck it in another box, booted a live gparted OS and redid the partitions while aligning them over to 1MB and verifiied via "align-check opt n" while still mounted in the gparted live OS.  Pulled the drive, put it in the unraid box, booted and mounted it via unassigned.  Ran the align-check against it and it still said aligned.  

 

Performed the same tests; copying two large files totalling 130GB to the now aligned SSD; and have no issues.  Performance never drops off and all is well.

 

So; not sure why the alignment wasn't right when done through unassigned; but figured I'd let everyone know my experience and the fix in case they ran into this.  

 

 

Link to comment
16 minutes ago, dnoyeb said:

Not sure if anyone has run into this; but my symptoms were the following: 

 

500GB SSD formatted xfs via unassigned devices, when performing LARGE constant file transfers TO the drive (writes) the performance would be great for the first 80GB of a constant transfer but shortly after 80GB the performance would plummet and the drive would almost become not useable.  

 

Thought it was the firmware as this is an 840EVO; so updated that, tried again; same issue.

 

One of my co-workers remembered a long time ago that SSD's needed a certain alignment for the best performance.  I read through some white papers from IBM on theirs as well; and it was recommended to check alignment via "parted" command 


align-check opt n

Where n is the partition.  I kept getting that it was not aligned.  

 

I pulled the SSD and stuck it in another box, booted a live gparted OS and redid the partitions while aligning them over to 1MB and verifiied via "align-check opt n" while still mounted in the gparted live OS.  Pulled the drive, put it in the unraid box, booted and mounted it via unassigned.  Ran the align-check against it and it still said aligned.  

 

Performed the same tests; copying two large files totalling 130GB to the now aligned SSD; and have no issues.  Performance never drops off and all is well.

So; not sure why the alignment wasn't right when done through unassigned; but figured I'd let everyone know my experience and the fix in case they ran into this.  

 

 

Did you format the SSD with UD?

Link to comment

The first time around, yes.  When I put it into the other box and used gparted; I repartitioned and formatted it xfs via gparted.  

 

Pretty easy for anyone to check and see if their SSD's are aligned or not using the parted command; will be curious to see what other's are getting results wise.

 

 

Edited by dnoyeb
Link to comment
Not sure if anyone has run into this; but my symptoms were the following: 

 

UD formatted devices (same as unRAID formatted devices) are 4K aligned already, for advanced format devices and SSDs the partition needs to be aligned on a multiple of 4KB, they start at sector 64, so 64 x 512 bytes = 32 768 which is divisible by 4096, parted align-check is probably looking for a partition stating on sector 2048 (which would also be aligned).

 

Besides,  if your problem was an unaligned partition, you'd see the performance impact immediately, not just after an 80GB transfer, that doesn't make any sense, were you regularly trimming your SSD?

 

 

Link to comment

So when you run the align-check and it comes back no, you're saying it really is... interesting.

 

 I'm going to throw another drive in and do the same test today for the fun of it.

 

I think there is more to this honestly; I wouldn't be so quick to dismiss it.   One other item of note that I should have mentioned that could be why it takes a while to manifest itself; the system has 256gb of ram in it.

 

The testing was done multiple times on the same drive; after multiple reformats, different file systems (btrfs and xfs).  

 

This first picture shows the SSD writes on the bottom prior to the alignment; notice the bursts to write; then it starts to do smaller constant levels.

 

normal.thumb.png.3a767d8a816aa19a6f290a8639b83114.png

 

This second picture is on the timeline to the right, as you'll see the transfers stay consistently slow.  After the reads complete, the system will crap out after a while.

 

5aa28da511947_whereitstartsfailing.thumb.png.bd218c2acc0c1ce988072f585dd5d234.png

 

This last picture is post alignment, notice a much smoother pattern of writes:

 

after-alignment.thumb.png.25c3b2562d565dd2f04d3f8fe87eab30.png

 

The writes stayed looking like that until the entire 130GB was completed (two files, 70gb and 60gb in size back to back, the jump in read speed was as it moved from file to file).

 

 

Anyways, I am going to find another SSD and throw it in and let UD format it and verify if the alignment is off per the align-check and if so; will do the same tests.

 

 

 

Link to comment
1 minute ago, dnoyeb said:

 One other item of note that I should have mentioned that could be why it takes a while to manifest itself; the system has 256gb of ram in it.

Any unused RAM will be I/O buffer for Linux, so any initial burst of speed is going to be due to RAM.

Link to comment
7 minutes ago, dnoyeb said:

the system has 256gb of ram in it

So much RAM will make a large impact on testing as by default 20% are used for write cache, that's about 50GB, before testing change the defaults to minimum, by typing on the console:

 

sysctl vm.dirty_ratio=1
sysctl vm.dirty_background_ratio=1

 

Rebooting will change back to defaults or use 20 and 10 respectively.

 

 

Link to comment

I'll do the testing leaving everything the same at first; I was simply pointing out that the ram is probably part of why the issue doesn't manifest itself for most others (if this is actually an issue; trying to find another SSD now to test with).

 

 

Link to comment

in the interest of testing; I don't want to change more than one variable at a time.  I may play around with those settings after testing to see if they help level out writes however.

 

One of my co-workers is letting me borrow a couple SSD's later today; one of them is another 840 samsung, and one is another brand.  Will be an interesting test to see if I can reproduce the same outcomes.

Edited by dnoyeb
Link to comment

So, got some more SSD's; an OCZ and a Samsung 840.  The Samsung exhibits the same exact issue as the other Samsung; the OCZ does not.  I've done some reading and the Samsung 840 and 850 have a different NAND erase block size compared to most SSD manufactures: 1536kb and NAND page size: 8kb.  So, starting off at 32k seems to cause issues for the Samsung drives.  

 

Before alignment:

 

5aa485294ffbf_beforealignment-840.thumb.png.9a4fc065ee52e419325b8370df0f55d5.png

 

After alignment:

after-alignment-840.png.be186ab4d193a92e0abb882be27370e9.png

 

 

 

Link to comment
10 hours ago, dnoyeb said:

I've done some reading and the Samsung 840 and 850 have a different NAND erase block size compared to most SSD manufactures: 1536kb and NAND page size: 8kb.  So, starting off at 32k seems to cause issues for the Samsung drives.  

 

OK, that could explain what you're seeing, since as I mentioned partition is already 4k aligned, still I also have some 120GB 840 EVOs and did some testing with the partition starting on sector 64 and 2048 and results are practically identical.

 

copying a 50GB file from another SSD:

sector 64 - 159.86MB/s

sector 2048 - 159.88MB/s

 

writing a 20GB file with zeros:

sector 64 - 169MB/s

sector 2048 - 168MB/s

 

So not sure there's any value using a different starting sector even for Samsung SSDs, but if it works best for you keep it like that.

  • Upvote 1
Link to comment
16 minutes ago, johnnie.black said:

So not sure there's any value using a different starting sector even for Samsung SSDs, but if it works best for you keep it like that.

UD formats disks so they can be included into the array with partitions starting on sector 64.  The only file formats that are done this way are xfs and btrfs.  All other file formats use a partitioning that is optimal for the device.

 

You have several choices:

  • Use a file format other than xfs and btrfs using UD to format the drive.
  • Partition and format the disk on another system partitioned how you prefer.  Just keep in mind the disk cannot be put in the array without re-formatting.
Link to comment

Good morning @dlandon

 

As requested, I'm bringing my issue to this topic.

 

"Did anyone ever got this problem below?

 

I tried updating to the latest version, rebooting and reinstalling the plugin. This error appears only when I mount any drive.

 

Thoughts?

 

Thank you!"

 

1.PNG

Link to comment
38 minutes ago, Sanches said:

Good morning @dlandon

 

As requested, I'm bringing my issue to this topic.

 

"Did anyone ever got this problem below?

 

I tried updating to the latest version, rebooting and reinstalling the plugin. This error appears only when I mount any drive.

 

Thoughts?

 

Thank you!"

 

1.PNG

Remove the prelcear plugin, reboot, and then post diagnostics if you continue to see the error.

  • Upvote 1
Link to comment
Remove the prelcear plugin, reboot, and then post diagnostics if you continue to see the error.
@dlandon Just had a thought. To mitigate the issues with preclear plugin, might be an idea to put into your install routine to delete/move the plg off flash if it's found so it doesn't install on next reboot. I'd suggest uninstalling it as well if it hadn't spread crap far and wide across the whole of the system.....

Desperate measures and all that.....

Sent from my LG-H815 using Tapatalk

Link to comment
@dlandon Just had a thought. To mitigate the issues with preclear plugin, might be an idea to put into your install routine to delete/move the plg off flash if it's found so it doesn't install on next reboot. I'd suggest uninstalling it as well if it hadn't spread crap far and wide across the whole of the system.....

Desperate measures and all that.....

Sent from my LG-H815 using Tapatalk

As much as I'm in the no need to preclear camp, I don't agree with that.

I think it's time for you to change your avatar to the one I've suggested.
Link to comment
  • trurl pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.