JohnO
-
Posts
123 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by JohnO
-
-
Any one else have /bzroot take forever to load on startup?
When switching from unRAID 5.0.5 to unRAID 6-Beta15 I noticed that the loading /bzroot takes about 4 minutes, whereas the v5 boot was probably under a minute. I'd guess the file is much larger now, and it needs to be uncompressed as part of the startup process. Thankfully, I don't restart the unRAID VM that often. This is using PLOP.
-
So, in the end, you wind up with both disk1 and disk2 as 2TB XFS. All of the original disk1 and disk2 data is on disk2 and disk1 is empty waiting for you to put other data onto from perhaps other disks so you can repeat the process.
I won't go into the details of how you actually do the copy/move. I did mine in mc. Others preferred rsync with verify. I think some may have even done it over the network with PC software.
Perhaps so many words it will only serve to further confuse.
Wow. No. I get it now! Thanks to you and gundamguy for taking the time to re-pour the words back into one ear, because clearly, they had gone in one ear, and drained out the other!
I see now it'll be a bit longer process than I had thought, but that's OK.
I really appreciate it. Thanks, guys.
John
-
Hi John,
I sincerely apologize here ..... after getting frustrated i decided to reboot the network .... and systems that are on it ...
leaving all the settings how i changed them it all appears to be working
I'm glad to hear its working! Now be sure to test to see you are getting the performance you would like to see.
John
-
What i have also discovered is if i have the servers MTU set 9014 and set system 2 jumbo packets to disabled i can connect to the server fine and achieve transfer rates of over 100MB/s......... Which has confused me slitly as when the server is at default values i get between 10 / 20 MB/s
Any ideas ?
What I believe is happening (it's been a few years since this was my daily work focus) is that when one system is set for jumbo, and the other is not, the network switch in between is fragmenting the packets for you, as it knows that one system can't handle the jumbo frames.
What type of network equipment is between the two servers? Can you confirm the jumbo frame size on that switch/switches?
If you assume the switch is set to 9000+buffer, then I'd set both servers to a jumbo frame size of 9000 and test. If that doesn't work, set each server to a jumbo frame size of 8968 and test.
Either way, you should have a lot fewer packet disassembly and reassembly activities that you would with a standard Ethernet frame size of 1500 bytes + buffer overhead.
John
-
You've got to different issues, replacing a disk with a bigger disk, and converting your file systems.
The thing to keep in mind is that if you rebuild from parity your array will rebuild that disks with the current filesystem, you can't switch file systems and rebuild at the same time. So what you will need to do is either convert the disk to XFS first then swap for the bigger disk, or swap for the bigger disk then convert to XFS.
If you've got space issues I think the second path makes more sense...
I think this is finally starting to sink in...
Your process does make sense, and will work, but as Trurl explained when you remove a data disk from the array you are going to invalidate parity and be prompted to rebuild your parity. So you might have more downtime as you do parity rebuilds during this process. Also since you plan to replace two disks that means you'll be prompted to rebuild parity twice. Not sure if there is a better way to sequence this, but you can view this as two seperate tasks, 1) Converting from RFS to XFS 2) Replacing to hard drives.
If I were trying to do what you want to do, I would first replace the drives and then convert from XFS to RFS, but you can sequence this as well so that you replace drives and convert in alternating steps.
So you are suggesting that I should just stop the array, and replace both 1 TB drives for 2 TB drives, and have the unRAID parity system "magically" rebuild them on to the new 2 TB drives. These would then be RFS drives at this point.
If I take this approach, I'm not sure the best way to get to XFS. I only have 4 slots on this controller card, and all are full (three data drives, one parity drive), thus the discussion of consolidating data from three data drives down to two as my first step.
Sorry for the basic questions, but as these are fairly lengthy processes, I figure I'd better get it right to try to minimize down time.
Thanks,
John
-
i edited the 'boot/config/network.cfg added MTU=9014' which all my systems are set to...
all my machines are Intel® PRO/1000 Server Adapter NIC So they suport jumbo frames
You need to make sure the frame size on your hosts is smaller than the frame size on the switches and routers. So, if the switches and routers are set to a jumbo frame size of 9014, you need to make the host frame size small enough to allow the jumbo frame header to be attached. At least for testing, you can decrease the frame size a bit on the hosts to ensure the jumbo packet is making it through the network infrastructure.
https://en.wikipedia.org/wiki/Jumbo_frame#Super_jumbo_frames
http://www.mylesgray.com/hardware/test-jumbo-frames-working/
John
-
@johnO
I replace my parity and added two drive and what gundamguy describe happens to me. Two new drives are xfs and the other rfs which makes parity rfs
So I'm left with what he suggested if moving data to a different disk and converting and moving back.
Just to clarify -- maybe I should not use the word "replace" but instead, I'll be add new drives to the array, and remove old drives from the array (with one removed/re-added drive being the same physical mechanism).
-
You've got to different issues, replacing a disk with a bigger disk, and converting your file systems.
The thing to keep in mind is that if you rebuild from parity your array will rebuild that disks with the current filesystem, you can't switch file systems and rebuild at the same time. So what you will need to do is either convert the disk to XFS first then swap for the bigger disk, or swap for the bigger disk then convert to XFS.
If you've got space issues I think the second path makes more sense...
Hmm... Maybe I'm missing something. I was not planning on rebuilding a drive from parity. My thought was to empty out one drive by moving it's contents to another existing drive, remove the empty drive and replace it with a larger drive, pre-clear the new larger drive, format it with XFS, and then move data to it from another existing drive. Once that is done, I would continue a similar process until the data has been migrated to drives formatted with XFS. In particular here is the layout:
Current state: Future State:
Parity: 2 TB (keep drive) Parity: 2 TB
Disk1: 1 TB RFS (60% full) (replace drive) Disk1: 2 TB XFS
Disk2: 1 TB RFS (40% full) (replace drive) Disk2: 2 TB XFS
Disk3: 2 TB RFS (20% full) (keep drive) Disk3: 2 TB XFS
My thought process is as follows:
Move all data from Disk1 to Disk3, remove current Disk1.
Install replacement for Disk 1
Move all data from Disk2 to new Disk 1, remove Disk2.
Install replacement for Disk 2
Move all data from Disk3 to new Disk 2, re-format Disk3.
Move some data from Disks 1 and 2 to Disk 3 to even out the disks somewhat.
Does that make sense?
Thanks,
John
-
10 - Now is the good time to move the files in the "t" directory to the root on [dest]. I do this with cut and paste from Windows explorer.
11 - Stop the array (no need to delete anything from the [source])
12 - Go back to step 2. Note that this isn't a race - you can do it at your leisure of the course of days, weeks, or months. I do one or two a week or so.
Greetings,
I've been reading and re-reading this thread over the last couple of weeks. I have two questions:
1) Before I can add a new, larger drive, I need to move data around so that I can remove a drive. I figure it's about about 600 GB of data. Would it be best to shut down the array to move that data? I assume in that case I'm safe to just telnet to the unRAID server and use the following rsync commands to move the data into the existing hierarchy, without having to create a temporary subdirectory that isn't part of the user share, correct?
rsync -av --progress --remove-source-files /mnt/diskX/ /mnt/diskY/
At that point, I should be able to start following the guide, I think.
2) Where does pre-clear of the new drive get done? Is it before the steps in bjp999's guide? It is mentioned, but it is above the steps, so i wasn't sure.
3) Also - in the steps above, as the array is running, are you at risk of duplicate files, and confusing things between steps 10 and 11?
Thanks,
John
-
Disclaimer: This is beta software. While every effort has been made to ensure no data loss, use at your own risk!
I just upgraded my NAS from unRAID 5.0.5 to 6.0-beta15 following the upgrade guide. The process went very smoothly. I ran a parity check over night, and re-enabled and tested services this morning. All seems to be working well. The new webGUI is really nice!
FYI - my unRAID environment is itself a VM on VMware ESXi 5.5 with a passed-through disk controller. I'm not running any dockers or VM from within unRAID.
John
-
I really don't know where to put this, so for now it's here, in the Forum Feedback board, seems sort of related. I needed a thread for anyone to place corrections and suggestions for the new guide to upgrading from v5 to v6.
I want to pass along my thanks for your guide. I upgraded my 1 year old unRAID 5.0.5 NAS (which is a VM in a VMWare ESXi 5.5 server with a passed-through disk controller). The process went very smoothly. I did not have to modify the PLOP settings at all -- I just followed your guide to re-format the USB boot drive and update with software and configs as you indicated.
One question/comment. In following the guide to copy over the files from the version 5 config directory, I note that it copies my older cache-dirs file. Should that copy be excluded from the copy, or deleted afterwards? It appears it is not active, but it might cause confusion.
I run a very bare-bones system, with no plugins. I was configured with 1 GB of RAM, but since I had been reading in other places to go with 2 GB RAM, I did that. Of course, in a VM environment, the RAM allocated is somewhat smoke and mirrors, as you can over allocate.
John
-
I also don't understand the virtualization part? Are users just whipping up unraid into a VM and running that VM on the server where the drives are? If so, then how are the other VM's using/seeing that hardware if they are all dedicated to UNraid anyway? I understand VM's, I have Esxi running with a couple VM's on it now, but can't picture it with unraid. Maybe someone can explain it, cause I do want to understand it to see if I can benefit from it in the future.
It's unclear if you are interested in running Unraid as a guest in a different VM Host OS, or use the VM Hosting features of Unraid V6 as a Host. If you -are- interested in running Unraid as a guest, there is a whole section here to discuss it.
http://lime-technology.com/forum/index.php?board=55.0
I've been running v5 for over a year as a guest VM on ESXi 5.x with no issues.
John
-
Done! Thanks! Don't know why I didn't think of that. Bought a 3rd edition, published in 2000, regrettable, but hard to pass at $4.50.
Cool. The 3rd edition is the most recent (previous) edition, so once you read through that, go visit your local Barnes and Noble and thumb through the newer version to see what changed -- Or grab the free table of contents from Amazon.
-
I was very interested to examine this, then discovered you have to buy the book, then saw the price! It's a little out of my reach just now, but I do appreciate the recommendation, and hopefully will acquire a copy some day. I know I need it.
Older versions are available used for a substantial savings. I see books in the $5.00 range on eBay. It'll still give you the 80% value for the architecture - some of the specifics have changed, but once you have the foundations, the rest you can search for on the Internet. DNS, syslog, DHCP, IP addressing, have been around a long, long time, and at the lower levels, really haven't changed all that much over time.
-
So rather than bother everyone and post every red line in my syslog, is there a toolset or code book or established technique somewhere that can help noobs learn to interpret what the error codes in the syslog mean - and how to resolve them? Or is this just a handed-from-one-generation-to-the-next thing...
This is probably a lot deeper than you want to go, but to understand what is in the syslog, you need to understand the components that send messages to syslog -- that pretty much means you need to understand Unix/Linux at a deep, under the hood level.
This book is (one of) the true bibles. It's been around (and revised) for ages. Reading the whole book really gives you a good understanding of the various technologies and how they all work together.
John
-
However, its not active. I noticed during boot that the activity and link lights are on for the first part of the boot (noting its connected at 1gb) but about halfway through boot, I assume when the card driver is loaded, the lights go out.
Do you have another chassis where you can test this card to rule out a bad card, or a conflict of some sort?
Do you have (or can you set up) a different, bootable OS from a DVD or USB memory stick so you can see if the UnRaid O/S is the issue?
-
@JohnO - So I got the Intel Pro/1000 PT Dual Port 10/100/1000 PCIe D50868-003 EXPI9402PTBLK today. Plugged it in, Unraid is not seeing it. Is there a driver that has to be loaded, or is the driver in the build? How did you configure the adapter. This was a pull from a server, so if the card is configurable, I'm sure its holding some type of configuration (remote boot?). How did you configure your adapter? Is there support software I need to use to configure the card?
That was fast! As I mentioned above, mine is in a box running VMware ESXi -- the drivers for that card are included in ESXi. What ever shows up to the Unraid guest OS is probably a VMware interface (which, I think emulates the Intel Pro 1000).
Hopefully someone else with that board who is using it natively to boot Unraid can chime in.
John
-
@JohnO - will dual nics on an Unraid server only offer redundancy, or can you bond the nics together to get faster throughput? I have a netgear GS108EV1 switch (currently) about to replace with a Rosewill RGS-1024 switch.
I'm not sure. I'm using mine as two separate ports, and my base OS on this box is VMware ESXi, so even if my Unraid Guest OS saw the two interfaces, it would probably be seeing the VMware network interface layer.
-
@koyaanisqatsi - I'm not against getting another NIC. In fact I was considering a two port NIC to improve throughput using bonding. Is that worth the effort in a home media environment with 4 or 5 active devices?
I added this 2-port card to my home server. Seems well supported. I run VMWare ESXi on this server (where Unraid is a guest OS).
-
So the answer, for the purposes of this issue is no, dual boot is not enabled.
Hmm... Along the same lines, do you have another network interface you could try, even if only temporarily?
-
That's a great question. I ran the system without the ethernet cable attached. Ran for a long time without crashing.
I also ran the latest 5x version. That ran for many hours without crashing either.
So it sounds connected to the ethernet somehow? Driver problem? I can access HTTP://Tower just fine until it crashes...
Do you have this system set up for dual-boot by any chance? I had a somewhat similar problem where if I was running Windows, then rebooted to Linux, the network interface would work for about 5 minutes, then die.
I had power-cycled the system from the "soft" power switch on the front on the machine, with no change in behavior. I unplugged the system from AC power and let it sit for 5 minutes. After that I booted straight into Linux, and the system has been fine for a year, including power cycling. I just can't boot to Windows and back to Linux without disconnecting from AC power.
-
1 GB of ram assigned to my 5.05 VM guest running unRAID. Swap hasn't been touched, and the system is not breathing hard (I've got two main directories, with 3-8 subdirectories each -- a very small set up.
unRAID Server Release 6.0-rc3-x86_64 Available
in Announcements
Posted
This seems troubling. I wonder if the new kernel doesn't play well as a Guest OS to VMware ESXi -- or maybe needs a kernel flag set.
I've got b15 (working fine) as a VMware ESXi 5.5 guest. I probably won't get a chance to try booting to the unRAID RC until next weekend.
John