-
[Solved] Extremely poor write performance to NVME Pool
Thanks I found it, writing to a disk share did indeed resolve the issue, thanks!
-
[Solved] Extremely poor write performance to NVME Pool
Can you point me towards a guide that illustrates how to set a disk share up? It is really not clear and google is failing me. Thank you.
-
Excypher started following [Solved] Extremely poor write performance to NVME Pool
-
[Solved] Extremely poor write performance to NVME Pool
I've a pool I've set up which is a pair of NVME drives in Btrfs, it doesn't act as a cache and merely as a mirror pool to act as fast storage between my machines, all of which are hard wired together via a 10gb switch. I get good numbers when writing sequential files to the drives over the network and can saturate the 10gb connection until the cache of the drives fills up. Also transferring files to and fro with windows explorer also performs acceptably. However, when I have a program writing directly to the nas, it becomes prohibitively slow. Reaper exporting an Mp3 for example will run at 2-3x realtime when writing to the nas, as opposed to 30-50x when writing to a local drive (taking 15+ minutes to export a 20mb mp3) Rokoko will slow to a crawl attempting to export similar, small files, blender will go from a physics bake taking 20 minutes to 12.5 hours. It's obviously unusable for it's purpose, and I am at a loss as to how to proceed, at the moment I just have to fragment where I'm saving my data and that is obviously not the solution I'm trying to achieve here. Some people advise this is just unraid being bad and not handling NVME drives well, is this the case? Or can something be done about it?
-
Some questions about first array build
Yes I see why you felt the need to clarify, seems there's potential for some problems if you mix the two by the looks of it. May I ask whilst I have your attention, my plan involves putting my entire storage onto a single disk temporarily, is it easy enough to have it distributed once all the disks are in place and the full array is built? Is there any appropriate reading for that operation? Thanks!
-
Some questions about first array build
Again, what these articles are really good at showing is what user shares are and how they operate, they don't show the difference between them and disk shares. When you say 'disk share' within the context of unraid it sounds much more like it's a method or definition within the unraid ecosystem, it leads you to think it's something special or another method of writing to an array rather than you using what might be the common sense definition, that being simply the disk itself. In this case, honestly, it was simply a definition problem, I said disk as In my application when I do the initial data load the array would consist of a single disk. I'm not at that point talking about an unraid definition (at that point I didn't even know disk shares were a thing!) When it came to setting up the array I would have created a user share because that's the default behaviour and what all the 'getting stated' guides indicate you should do, when you said disk share Initially and expressed concern my immediate thought was you were advising against any kind of sharing on the network at all, hence the confusion.
-
Some questions about first array build
No, that particular page of the wiki did not lead me to that conclusion, it was the exterior summary that pointed out user shares allow unraids management that make it make sense to me. You say: not sure you know the difference between these two shares, here read this, article says, there's two types of shares, here's what this one does, and here's what the other one is called, but doesn't explain why it's different directly to a disk share. To be fair, that's maybe not the purpose of that particular page you linked but that's the source of my confusion. Anyways, all is clear now so, thanks for the help
-
Some questions about first array build
OK I see the confusion now. I thought he was referring to the act of sharing the space at all over the network was the issue, rather then the definition of disk shares VS user shares. I gave the link a read (albeit skimming a lot of what I considered to be irrelivent to this discussion, IE the allocation method sections), and honestly, I couldn't really make much sense of the distinction until I just googled 'unraid difference between user and disk shares' where the following statement finally gave clarity: The article itself even says: Which doesn't suggest there's much of a concern or reason to use either. So I guess if you are looking for feedback on that article, it doesn't really give a clear definition between disk and user shares that at least, made sense to me. But to clarify the original concern I quoted, no I do not intend on using disk shares.
-
Some questions about first array build
Would you clarify your statement here please? My understanding, as loose as it is, is that a share is effectively unraids method to map what is recognised by other systems as disk to a physical space on the disks themselves? IE this share equals this space on these disks, and you then map that drive onto the user systems (in this case windows, so mapping a network drive) My plan is to build the unraid server on a new machine, and keep my existing array in an old machine temporarily, to allow the data to be transferred onto the unraid pool, then add the old array disks to the new unraid pool once this is populated. Or is this a definition thing, where a share or network share is different to a user share? Thanks!
-
Some questions about first array build
Gotcha so: set up array with 1x 18TB disk and the SSD cache load data from old array onto this share enable cache add parity add remaining disks? The machine is going to exist solely as a network share... so unless I'm missing something, I shouldn't need Docker or VM's
-
Some questions about first array build
So in effect, the best bet would be to copy from the existing drives onto one of the 18TB drives that is effectively alone in the share, then build the parity, then add the original source disks, then add the cache?
-
Excypher joined the community
-
Some questions about first array build
Hi everyone, I have a couple questions about the setup before going ahead with my unraid build, just to clear up a couple potential concerns. Currently I have 4x4TB drives in raid 1 as my storage, I have about 5TB used of the 7.25TB usable. I am planning on putting those 4x4tb, plus an additional 4TB into the unraid array, In addition I have 2x 1TB SSD's I plan to Mirror as a cache volume. Obviously since I'm going to have to store the data in the mean time, I have 2x new 18TB drives, one of which I'm going to use to copy the array to before building it, and the other is going to be the parity drive, once the array is built I'm going to add the other 18TB that I've used for temporary storage to the array. So, My question is: since the cache is going to be 1TB, and I'll have 5TB to copy to the array once completed, what steps need I take, if any, to ensure this transfer onto the array goes cleanly? As my understanding of the cache setup is that you schedule a worker task to move the data from the cache onto the actual redundant storage overnight during normal use, does this mean I'm going to have to copy 1TB ish worth, run the worker, rinse and repeat or is there a way to just transfer the whole shebang without interruptions and just leave it to it overnight? My other question is a general usability question, My understanding of how unraid works is it only spins up the drives required if you access them, basically, does this apply to metadata too? As in, if I have the share mapped as a network drive, if I go to click on it to browse the file structure, but not necessarily access files, am I going to expect some spin up time for the drives when this happens if I have them configured to spin down? Or is it smart enough to save that file structure elsewhere? Thanks all!
Excypher
Members
-
Joined
-
Last visited