Roscoe62

Hi there. After contacting support they ensured a new email came out with the new key. I queried them to see if they could find out why the original email hadn't turned up, but they weren't able to find out why. Seems they may still have a hole in their process somewhere.

I'm finally circling back around to this. I've now completed the Disk 1 upgrade and it's all gone well. I also got to re-run the test that caused the initial issue that lead me to raise this thread in the first place....I transferred a (roughly) 25GB file from my PC to the server. This time the file transferred without any issues. It did slow down a little - which was expected given I'm not using a cache drive (yet), but the transfer speed never dropped below 50MB/s, which is WAY faster than it ever went before. So, I'll mark this thread as solved. I believe - based on many of the discussions that have gone into this thread - that the cause of this issue is one of, or a combination of, these 3 things... The problem share used the 'most space available' allocation option. I'm now using the 'high-water' option My server was running a bit low on storage space. I've updated one of the drives the share was using (from 4TB to 12TB) My server still uses the 'reiserfs' file system. The drive I updated now uses 'xfs' instead Thank you to all who contributed to this conversation, helped me to learn a lot, and finally resolve the issue. I'm grateful.

Thanks very much. This is good to know!

I'm in the process of updating/expanding all the data disks (I have 8 of them) over the next 6 months or so. I'm part-way through the first one. While a parity rebuild was going on, I noticed an amber error message relating to disk 7, but it disappeared before I could read it - and the parity rebuild completed without error. I finally found something related to the error on the dashboard.... For the moment please ignore the temperature warning on disk 4. The disk temperature is often a little higher during a parity check or rebuild. You'll also note the utilization is quite high - especially on disks 2 & 3. The disks are all scheduled to be replaced/increased - disk 1 was the first (after the parity drive). It's the SMART error on Disk 7 I was interested in. If I click on it to get more information I see the following... This disk, like all the others, will be replaced/expanded over the next six months. What I'm asking is - does the UDMA CRC error count reported above make that disk a candidate to be replaced sooner rather than later? Should this disk be prioritized as the next disk to be replaced? I don't know enough to know how serious an issue this is. Thoughts?

Hi JorgeB, Thank you so much for this. I can see this working, and it ensures all 4 of my goals are achieved. There's a few specifics I don't know how to do yet (using UD to mount a disk & copying the data from old disk 1 over to new disk 1 (now in xfs format)), but I'm hopeful this will become more obvious once I actually have an unassigned device attached. I'm going to start this process today. Thanks again!

Actually, I'm going to try this from a different angle. Please bear with me, and forgive me if I tread on toes.... I'm here posting in this forum because I need help. I know what I want to do (see the goals listed at the top of my procedure document)....actually, I'll list them here as well. Goal 1 - Replace Disk 1 in my array with a larger, new disk Goal 2 - Ensure the new disk is formatted as xfs (all data disks in my array are currently Reiserfs) Goal 3 - Remove/retire old Disk 1 from the array Goal 4 - Ensure the new Disk 1 is re-positioned to occupy the same physical location in the server as the old Disk 1 That's WHAT I want to do. What I DON'T currently know is HOW to accomplish these goals as simply as possible, with as little risk as possible. Can someone please give me a list of steps showing me how to proceed? JorgeB gave me a pretty good high-level way of doing most of this in his post above, but I need this broken down a little further as I lack both knowledge and experience. THIS is what I want. I'm probably guilty of not being direct enough with my questions. I'm trying to fix that now. If someone is kind enough to provide the steps I will be very grateful. If I need more detail, I'll ask.

You make it sound so simple when you put it that way....😉 The simple answer is - because I don't know HOW. Since building the unRAID server about 7-odd years ago (actually, REbuilding is more accurate, but I won't elaborate) I've been using it pretty much as a "plug and play' device. Do a parity check each month, when wanting more space, swap the disks for larger ones starting with the parity drive, if a disk goes faulty, shut down the server and replace the faulty disk with a new one. Using plug-in's is almost completely foreign to me, so it's all stuff I need to come up to speed with now that I'm dealing with issues outside of what I've known before. So, to answer your question - I simply don't know HOW to use UD (I even had to look it up to find out that 'UD' refers to the 'Unassigned Devices' plug-in) to do the things you're suggesting. It's something I'll have to learn. What you suggested in your last post sounds WAY easier than my process document, and it's also likely to contain a lot less unnecessary duplication. I'd much rather do it your way, but I'm going to have to figure out HOW to use UD to format the new drive to xfs. Then I'm going to have to figure out HOW to use UD to copy all the data from disk 1 to the new drive. Then I guess I can use the steps in my document to set up the new config (leaving the old disk out of the picture completely). And after parity has been rebuilt - based on the new config - will doing my final set of steps work to allow me to physically relocate the new disk 1 to the old disk 1 location without screwing anything up? Your summary seems to have excluded that. In the meantime, I'm browsing through the UD support thread in an attempt to learn how to use UD to format the new drive, and then how to use UD to copy all the data from the old drive to the new one. Yes, your high level process sounds way simpler than what I'd documented, and I'd much rather use it, than mine. But I'm paranoid about doing something wrong and losing data in the process. I'm sorry if this post sounds "snippy" - it's just the frustration of not being able to move forward because of my own lack of knowledge.

OK, I've followed your advice and documented ALL the steps to upgrade ONE disk (increase in disk capacity and changing to use XFS file system) and I've attached the resulting document to this post. I've tried to be as detailed as I could be - to reduce the risk of misunderstanding/misinterpretation. I've included, at the end, a section of steps to reposition the new drive into the position occupied by the old drive. Please bear in mind I've never done any of this before, so some of the things that seemed logical to me may NOT be correct. I've had to take a guess at a lot of these steps - hopefully I got them right. I would be VERY grateful if someone who has done this kind of thing before, or who is confident they know the steps required to do them, could take a look at the document and see if the steps are correct or not. Like, any unRAID user, I'd hate to get part-way through this procedure only to find some steps are missing, or are incorrect. There are 48 steps in total covering 6 separate high level tasks, and accomplishing 4 goals (listed at the top). Don't worry though - it's only a 2 page document. Again, thank you for your guidance, time, and advice! UnRAID Server - Disk Replacement steps_v1.docx

Yes, I guess so, and it was worded that way in the original instructions as well. Got this one covered. During the copy process I will be the only one using the unRAID, there are no dockers or VM's running, and the ONE job I had running (to back up my main PC) I've deactivated for the time being. The system will be completely static. Yes, I think this is where some confusion has crept in - is the process step referring to the physical disk, or the disk slot? I do currently have a disk map document, which lists the physical disk data (make/model/serial number) for each disk and (currently) which slot it's in. That's a really good idea. Using that - along with the disk map - should help me to keep track of my progress. Many thanks!

Hi itimpi, Thanks for pointing that out. It hadn't occurred to me that there might be an updated (simpler?) way of doing this. I'm waiting for my final parity check to complete before starting this, so there's still time to change the process. Having said that - there are still a number of steps that remain common and, reading through the Storage Management document you linked to - I couldn't find equivalent steps anywhere, so maybe they haven't changed? Here's a high-level task list of the things I need to do... 1. Install new drive, assign it to 'Disk 9' (the next free disk number in my array) 2. Before starting array ensure disk9 is excluded in Global Share Settings 3. Start array - ensure unRAID clears Disk 9 4. When Disk 9 clear has completed, format Disk 9 to XFS format 5. Once Disk 9 formatting is complete, use Dynamix File Manager to copy all files from Disk 1 (my first 'source' candidate) to Disk 9 - ensuring, if possible, all file/folder permissions/settings have also been copied across 6. Stop the array This next section is crucial to ensure UnRAID handles the new drive correctly, and I couldn't find an updated process for this - maybe because it's so specific. 7. In Tools/New Config ensure Preserve Current Assignments is set to ALL. Click Apply and Done 8. On Main page select dropdown for Disk 9 and unassign it 9. Click on the dropdown for Disk 1 and reassign it to disk 9 10. Go back to the dropdown for Disk 9 and assign it to disk 1 11. Click on drive name of disk 1 & change file system from ReiserFS to XFS, click on drive name of disk 9 & change file system from XFS to ReiserFS Not 100% sure what to do in this step. For me, I'll be retiring disk 9 (the old disk 1 in Reiser format). 12. All array disks should be blue with warning "All data on the parity drive will be erased when the array is started". Make sure to click the check box "Parity is already valid". Then click the Start button to start the array. Do the above steps seem reasonable to accomplish the task? Is anything missing or outdated? Looking back at step 2 where Disk 9 was excluded from the share, does this still stand, or does it need to be fixed? I must admit I lost track among the reassignments. Sorry! So many questions!

Sorry it's taken so long to return to this thread, but I have been busy! I changed my main share (the one I'm having an issue with) to use the highwater allocation method. I've also switched out my 1 parity drive to a new lager one - went from a 4TB drive to a 12 TB drive. UnRAID rebuilt the parity data on the new drive and found no issues...so far, so good. I'm expecting the first of my new 12TB data drives to turn up later today. I'm going to use the "mirror method" as described here... https://legacy.wiki.unraid.net/index.php/File_System_Conversion I've read through this method several times now, and I'm satisfied I understand what each of the steps does. In my case it's a little easier in that each of my drives will be replaced with a new, larger drive and once the new drive (in xfs format) becomes the new drive, the old one will just be retired, as opposed to reformatting to xfs and starting on the next drive. Changing all my data drives over will likely take a good 6-8 months (money for drives is limited). I've still got 4 questions about the procedure I'm not sure of, and would appreciate it if someone could confirm the answers before I dive into this next step... 1. Once I've physically installed the new 12TB 'swap drive', and I've powered up the server, does unRAID automatically clear the drive? If so, at what point does that happen, or what extra step do I need to do to clear the drive?. If I've understood the procedure correctly, clearing the new disk should immediately precede formatting it to xfs, correct? 2. The main "tool" the procedure author uses is rsync. Is 'rsync' a part of the standard unRAID software package - so I can just run the command lines he states as part of the procedure, or do I need to add it in first? If so, where can I find it? 3. Can I run the rsync command lines using a terminal session from within the webgui, or should they be run from a keyboard attached to the server, or does it not matter? 4. Once I've completed the mirroring procedure for this first new data drive, and I've switched the drive assignments, there should be no need to physically swap the drives over - doing the re-assignment steps should take care of this. However, what if I wanted the new larger disk to be in the same 'slot' as the older smaller drive? Is it simply a matter of completing the procedure as written, powering down the server, swapping the drive slot, powering the server up, and ensuring the new drive is still allocated correctly before starting the array again? I currently have all the drives mapped in a document, and I'd like to leave the drives in the same positions in between upgrading each of the drives. I could just update my document after each drive upgrade, but if keeping the drive numbering and slot positioning the same is easy enough to do...that would be my preference.

First off, thank you for your responses. This is a BIG help! This is what I suspected. Thanks for confirming. One more question on this...Once I've changed a data drive over from reiserfs to xfs, does the parity for that drive need to be re-calculated? I guess I'm asking whether there's anything in changing the file system that would necessitate a change in the parity data held? Okay - will definitely incorporate the file system change with the migration to larger drives. I knew unRAID was moving off reiserfs, but I didn't realize it was so long ago! Will definitely start the migration asap.

Some further things... 1. Is the file system used applicable to the parity drive at all? I see on the main page of the webGUI a file system is listed for each drive except the parity drive. 2. Is it possible/advisable to have 2 file systems on the server for any length of time? i.e. some disks on Reiser & some on xfs? My new larger drives are pricey, so it could take months to replace them all. 2. The conversion to xfs looks to be something requiring a lot of planning to avoid data loss. I'm a little wary of changing the file system at the same time as replacing with larger drives. On the other hand, moving to xfs could fix my file transfer issue. <sigh!> I value your opinion of the best way forward.

Thanks JorgeB, Yes, I have been seeing the "ReiserFS is deprecated..." messages appearing and knew I'd have to do something about that fairly soon too. The list continues to grow! I guess I can switch the 'media share' allocation method to highwater as a first option - as that's very easy to do. However, for a next step, would I be better to replace the parity and data drives to newer, larger drives first, or look at changing over to xfs file system first? I've currently got 8 data drives & 1 parity drive - all currently 4TB in size, so I imagine changing over to xfs might take a while. I did a quick search on posts about moving from reiserFS to xfs. I also found the fs conversion process on the legacy unRAID wiki... https://legacy.wiki.unraid.net/index.php/File_System_Conversion ...so I'm going to read through that. In my case all the disks are old enough to warrant replacement, so I won't be reusing any of them....and the first of my new larger disks just arrived at my door 10 minutes ago. I'll read through the above procedure and hopefully I'll find something to match what I'm trying to do.

Hi JorgeB, Yes - by far the biggest share on the server is the 'Media Share' share, which is currently set to use the MOST FREE allocation method. This IS the share I'm having the issue with. Are you saying I should change the allocation method on this share to high-water? Also - going forward - I'm pretty much sold on the idea of installing an SSD in the last drive space on the server to act as a cache drive. I really didn't like seeing 3 of my data drives have now gone past the 90% allocation threshold, so I'm replacing a number of my drives with newer higher capacity drives - starting with the parity drive. Once that drive, along with those over the 90% full have been replaced, I'm keen to get the cache drive up & running. These steps are not a guaranteed solution to my file transfer issue, but it's certainly not going to hurt.

Thanks very much. I did some investigation, and all shares were set to use the highwater allocation method, except for the 'media share' share - which is by far the biggest share on the server, and the 'backups' share - which are really the only shares we use regularly. Both of those are currently set to the MOST FREE allocation method. I've attached latest diagnostics. server-diagnostics-20240322-1106.zip

I thought I knew this, but can't find where the setting is for this. I thought unRAID wrote new files to the disk that had the most spare space left, but I was a bit shocked to log into the unRAIID GUI yesterday to find that my drives 1, 2 & 3 (the drives with the least amount of storage space left) have now passed the 90% allocation mark, and were presenting me with warnings about it. It was always my intention to replace some of the unRAID server disks with larger ones (they're all currently 4TB in size), but thought I had a bit more time. Could someone please inform me of where to find the disk write preference setting? Thanks heaps!

Going to have to shelve this issue for a little while. Need to get my hands on another PC I can use to set up a mock unRAID trial, but haven't much spare cash right now. Will resurrect this thread when I'm able to.

Okay..... I'm now going through these suggestions one at a time. I'm currently 40 hours through a 48 hour memory test run. So far the server has successfully completed 26 passes of the memory test and, unless something changes dramatically, it's likely to complete the full 48 hours without showing any errors. The next suggestion involves setting up another computer to run an instance of unRAID and try to transfer a larger file through the network to it. I've got some questions as to how to view the results, and then some further questions on how to set this test up specifically. Here's the scenario followed by some questions on what the outcomes mean. Please let me know if I've misunderstood anything here... As I stated earlier, I don't have a spare computer with which to run this test, so I'm going to have to purchase a second-hand PC and some new HDD's to mimick setting up a new unRAID instance. I believe the purpose of this test is to attempt to identify whether the file transfer issue is being caused by unRAID, or something else on the network. Is that correct? The high level conclusions that can be drawn from this test are as follows: If I can successfully transfer a number of larger files over the network to the new (trial?) instance of unRAID, this would infer the fault lies within my existing unRAID system. Correct? Alternatively, if I cannot successfully transfer a number of larger files over the network to the new (trial) instance of unRAID, this would infer the fault lies elsewhere in my network. Are these outcomes correct? Are they a fair representation of what this test is trying to show? And now for some specific questions on how to properly set this test up.... What is the minimum number of drives I'll need to have in the test PC to run unRAID? I'm hoping it's 2. Can someone confirm? Am I able to set up a trial version of unRAID on a spare USB stick to run this test? I'm not keen to pay for another unRAID licence JUST to run this test. Is this correct? I may have more specific questions later, but this should be enough to get the ball rolling.

Hi JorgeB, you are not wrong! This is causing endless frustration for me. I hear you but, honestly, I think I've had a pretty good shot at eliminating 'the other things' in my earlier testing. The below post is a decent summary of what I've tried up to this point... In addition to the above, at the direction from @dlandon, I've tried forcing the server to use eth1 only (and eth0 only), and now upgraded the server's BIOS to the latest version. So far none of these things have uncovered the cause of the underlying issue. If you can see a test I've missed please let me know. As awesome as this forum has been through trying to resolve this issue it's beginning to feel like I'm close to exhausting the knowledge on offer. I'm now starting to consider whether paying for some 'one-on-one' unRAID support might be the only option left for me to try. It's an expensive option but it's gotten to the point where I can't put any new content on the server - not anything over 15Gbs or so...which is MOST of it. For me this defeats the purpose of having a server in the first place...so it's fast becoming a matter of either paying for professional help to find and correct the issue, or abandoning the concept of having all content on a central server altogether. Anyone with any further ideas of things to try (not already listed above), now is the time.

Unfortunately not. However, earlier in this thread I did a large file transfer from my wife's laptop to the unRAID server. Her laptop is in the same room as the unRAID server - so the file is sent out the laptop network cable into the same switch the unRAID server is plugged into, and through the LAN cable to the unRAID box, and the same issue was observed. Also, I've already swapped out both the switch and the unRAID LAN cable with new ones. Didn't make any difference. Based on the results from these tests, I don't believe it's a LAN issue.

Sorry it took so long to get this done, but BIOS has now been updated to latest version (BIOS dated 9 June 2021). I must admit I was nervous updating the BIOS - lots of stories about turning the mothboard into a brick - but it went smoothly enough, and unRAID came up again afterwards. Unfortunately, I still have the issue with larger file transfers pausing several times, and ultimately failing.

OK. Will take me a little while to organise this. I'll post back when it's done.

Sorry, it took a bit longer to action these tasks...sometimes it takes a while before I'm able to delete the failed file from unRAID. I unplugged the eth0 cable and tried the file transfer. It eventually paused, and then failed. I checked the network errors in the dashboard and it appears exactly as it did in the above screenshot - with the same count of eth0 drops and overruns. this outcome confused me a little, so I reversed the cables - unplugging eth1 and reconnecting eth0, and ran the test again. Again, a few pauses and the network error message. Oddly, when I checked the network errors section of the dashboard, the count of eth0 drops and overruns hadn't changed at all...and the other counters still remained at zero. I have NO clue what's going on. To answer some of your other questions... - No, the motherboard is not new. It's been used successfully for at least the last 5 years. - With motherboard firmware updates, do you mean BIOS update, or BMC Firmware updates? Drivers and utilities are also available, but they're based on OS (Microsoft, Redhat or SuSE). I'm not sure exactly what you had in mind.

Okay, I went to the Dashboard to view network errors. This is before doing anything else here's what I found... My switch is a 16 port unmanaged switch, so there's no interface with which to see network errors. Then I enabled bridging in unRAID, and tried a file transfer again. There were a couple of "slowdowns", but I think this is normal behavior because I don't use a cache drive. Also, a "slowdown" behaves differently in that I can still see the Windows file transfer speed constantly changing. During what I'm calling a "pause" the Windows file transfer speed freezes - showing whatever the speed value was prior to the pause. This time I thought to capture a screenshot of the file transfer during the pause.... Just FWIW, it may also be worth mentioning that during a file transfer "pause" condition, in the unRAID main window I see the number in the Writes column for both the destination drive and the parity drive continue to climb. The numbers cease to climb when the network error message is displayed. Finally, a screenshot of the network errors from the unRAID dashboard taken straight after the above file transfer failure... Hopefully there's some clue here into what might be causing the issue.