Jerky_san
-
Posts
449 -
Joined
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Report Comments posted by Jerky_san
-
-
6 hours ago, trurl said:
When using Fill-up allocation, or even just when you have disks that are very full, it is especially important to understand the Minimum Free setting.
Unraid has no way to know how large a file will become when it chooses a disk for it. If a disk has more than Minimum, the disk can be chosen, even if the file won't fit.
You should set Minimum Free to larger than the largest file you expect to write to the share.
I've tried setting it to a min I feel comfortable with but do you happen to know what happens if it does try to write and it can't fit it? Does it retry or something or can it be catastrophic?
-
6 hours ago, Squid said:
It never does 2 moves at the same time. As already explained, it's simply the write-caching of the system as a whole. But, the reason for the delay is that when it also has to switch between drives, then read operations also take effect (to gather directory information, etc). At this point, the system also switches from reconstruct write down to read/modify/write mode (by design) and that impacts your speeds. It takes a bit for it to switch back to reconstruct write mode, and depending upon file sizes, how much gets cached to ram, speed of the drives and how often the system actually has to switch from drive to drive, you may see a huge impact in transfer speeds.
As your screenshot shows, the system is currently in read/modify/write mode.
Is there a document or something that talks about things like this? Until yesterday I had no idea Unraid can switch between reconstruct write and read/modify/write mode without me explicitly telling it to. Do you know if there are other settings that can turn on/off when the system deems it required? Anyways would be nice to know of anything else so I can remember it if I'm ever trying to debug an issue.
-
4 hours ago, Frank1940 said:
It is probably happening because of the RAM buffer for writes. As soon as the last of the current file is written to that buffer, the client will start the transfer of the next file. This will start the file allocation process which requires disk access (both read and write operations) on the other disk which (of course) is available because there are no pending operations for it.
PS --- I see this all the time when I use ImgBurn to write an Bluray .iso to the server. The data transfer to the server will stop about thirty seconds before ImgBurn receives back the message that the file write to the physical device is completed.
I do have 32 gigs so it most definitely could read at least two if not three in. I am going to test one last thing because I think I realize whats going on. Only reason it matters to me is because I've had my array near packed before but I never remember this occurring in any other time. The two drives are nearly exactly the same amount of space free 1.96TB. So I wonder if it is doing what you said but only because they are the same fullness so it opens the gate to allow it.
Edit:
So tested it using unbalance to make the two drives different free space sizes. It does indeed work like I always remembered it doing. Where it would write to one drive at a time. I guess somehow I used mover to move files that made the two drives the exact same free space which I guess makes the mover initiate two moves instead of just one move at a time. The files I guess were similar size or so close in size that it would bounce between the two drives since I have enough ram to easily cache multiple files from the cache drive. Once the drives were different enough free space though it appears(greater than 10gb difference) it appears it will just write to one till it reaches the same free space again. I am going to do what @johnnie.black recommended and do fill-up instead but that explains why I never saw it in previous versions. It must of just happened in the past week sometime.
Edit 2:
Thank you @trurl @TexasUnraid @johnnie.black @Frank1940 for your assistance in figuring it out
-
9 minutes ago, johnnie.black said:
Correct.
It does appear to have fixed it. It's very strange though. The files were each 10gb or so. I'd of expected it to take time to write it before switching to the other drive. Anyways thank you for your explanation and helping me.
-
13 minutes ago, johnnie.black said:
Like @trurlmentioned this is the result of having shares set to "most free" allocation method, parity writes will overlap making the performance considerably slower, note also that since v6.8.x "turbo write" will be disable once multiple array disk activity is detected making performance even worse, you'll see the same behavior with v6.8, I never recommend using most free as an allocation method precisely because of this performance issue, it can be a little better if used with some split level that avoids the constant disk switching for any new file.
I'll set it to fill-up once it gets done rebooting and test. I guess it bases it off of free space and not % of free space? Also thank you for explaining..
-
3 minutes ago, trurl said:
Have you tried it with turbo write as suggested in that thread?
Yes I always use reconstruct write.
-
Just now, TexasUnraid said:
Try installing netdata docker, you can use that to see what is really going on in real time with the indivdual drives. Really helps figure issues like this out.
It's booted in safe mode with no dockers or plugins running. I specifically did this to make sure nothing was interfering with the tests
-
3 hours ago, trurl said:
The screen isn't refreshed sufficiently fast for you to claim "exact same time". And there is buffering to consider also. If allocation method is causing it to choose first one disk then another then writing to more than one disk during a brief interval is exactly what I would expect.
Do the test if you don't remember.
I mean this can run for hours and still be the same speed. It isn't a refresh issue.
There was even a thread specifically asking for this and johnnie.black stated exactly what i'm seeing. Slow write speeds.
-
1 minute ago, trurl said:
Not clear why you think it shouldn't. In fact, you have a share with allocation set to Most Free, and many of your drives are very full. It shouldn't be surprising that it constantly switches between disks when it is moving that share.
So.. your saying that the mover writing to two data drives at the exact same time as seen in the picture is considered normal? It drags the write speed down to a crawl doing this. I don't remember it doing it in previous versions either?
-
On 7/5/2020 at 3:07 PM, limetech said:
This is a 'beta' topic - that means more information is needed than simply "anyone notice ...".
At least needs diagnostics.zip, maybe an explanation of what's happening.
I noticed when I start my mover process it literally writes to two data drives at once. The reason I stated that I was going to downgrade was to do more research but was just throwing it out there wondering if anyone had observed a similar thing happening to them and as stated I'd get a diagnostic before I did it.
All I do is start the array, click mover, and it beings writing to two data disks at once. If I tell the mover to stop it will eventually. At the very end as the mover is stopping it will suddenly only write to one drive again. Running unbalance makes it only write to a single data drive as well. Diagnostics were taken during a safe mode run. All dockers halted as well. The only correlation between the two drives that i could make is they are both not as full as the others and both have encryption.
-
I'm going to downgrade soon after I collect a diagnostic but I was wondering if any of you have experienced the mover attempting to write to two drives at the same time? I also noticed it's the only drives that are encrypted in my array so far.
-
14 hours ago, Squid said:
Probably not the issue, and no way of knowing at this point what it was without the diagnostics, but one of your plugins installed has been deprecated and superseded by another. (Which FCP is telling you about)
Apr 3 07:59:02 Tower root: Fix Common Problems: Warning: Deprecated plugin dark.theme.plgYeah saw that this morning and removed it. Thanks for pointing it out though.
I switched out my USB stick today as well since I don't need it failing on me. Thanks again.
-
https://github.com/electrified/asus-wmi-sensors <- this would be nice to for 6.9 for ASUS peoples. It directly reads the WMI interface that ASUS has moved to and displays all sensors properly.(Supposedly)
-
9
-
-
Things are looking good so far for me.
-
o-o Thank you @limetech installing now
-
On 1/29/2020 at 10:57 AM, testdasi said:
Soon™ 😂
Following up on this.. any detailed release plan coming up? Really really want the improved kernel stuff. Honestly debating downgrading at this point and applying the vulnerability patches.
Btw
Not directed at you testdasi =0
-
1
-
-
15 minutes ago, eagle470 said:
Any idea on when we will see 6.9-RC1? There have been enough updates/bug fixes and security issues that I don't feel comfortable rolling back to 6.8-RC7, but I really need the new kernel version.
Yeah I'm excited for the improved monitoring for temps and stuff.
-
1
-
-
3 hours ago, limetech said:
Has to do with how POSIX-compliant we want to be. Here are the issues:
If 2 dirents (directory entries) refer to the same file, then if you 'stat' either dirent it should return:
a) 'st_nlink' will be set to 2 in this case, and
b) the same inode number in 'st_ino'.
Prior to 6.8 release a) was correct, but b) was not (it returns an internal FUSE inode number associated with dirents). This is incorrect behavior and can confuse programs such as 'rsync', but fixes NFS stale file handle issue.
To fix this, you can tell FUSE to pass along the actual st_ino of the underlying file instead of it's own FUSE inode number. This works except for 2 problems:
1. If the file is physically moved to a different file system, the st_ino field changes. This causes NFS stale file handles.
2. There is still a FUSE delay because it caches stat data (default for 1 second). For example, if kernel asks for stat data for a file (or directory), FUSE will ask user-space filesystem to provide it. Then if kernel asks for stat data again for same object, if time hasn't expired FUSE will just return the value it read last time. If timeout expired, then FUSE will again ask user-space filesystem to provide it. Hence in our example above, one could remove one of the dirents for a file and then immediately 'stat' the other dirent, and that stat data will not reflect fact that 'st_nlink' is now 1 - it will still say 2. Obviously whether this is an issue depends entirely on timing (the worse kind of bugs).
In the FUSE example code there is this comment in regards to hard link support:
static void *xmp_init(struct fuse_conn_info *conn, struct fuse_config *cfg) { (void) conn; cfg->use_ino = 1; cfg->nullpath_ok = 1; /* Pick up changes from lower filesystem right away. This is also necessary for better hardlink support. When the kernel calls the unlink() handler, it does not know the inode of the to-be-removed entry and can therefore not invalidate the cache of the associated inode - resulting in an incorrect st_nlink value being reported for any remaining hardlinks to this inode. */ cfg->entry_timeout = 0; cfg->attr_timeout = 0; cfg->negative_timeout = 0; return NULL; }
But the problem is the kernel is very "chatty" when it comes to directory listings. Basically it re-'stat's the entire parent directory tree each time it wants to 'stat' a file returned by READDIR. If we have the 'attr_timeout' set to 0, then each one of those 'stat's results in a round trip from kernel space to user space (and processing done by user-space filesystem). I have set it up so that if you enable hard link support, those timeouts are as above and hence you see huge slowdown because of all the overhead.
I could remove that code that sets the timeouts to 0, but as I mentioned, not sure what "bugs" this might cause for other users - our policy is, better to be slow than to be wrong.
So this is kinda where it stands. We have ideas for fixing but will involve modifying FUSE which is not a small project.
Thank you very much for the thorough explanation.
-
On 1/20/2020 at 2:49 PM, limetech said:
Turn off hard link support, as I already suggested in this topic, and see if this makes a difference.
It did really help with my ftp transfers. Program no longer locks up. Do you believe it will eventually be able to run with hard links enabled or is that kind of up in the air at this time? Also sorry for not completely reading the thread before responding.
-
I experience this as well. When I'm doing SFTPs to these folders it can cause my whole vm to lag in an attempt to list the folders. It can take a few seconds to bring up all the folders but honestly I have the folders as condensed as I can without plex hating me.
-
7 hours ago, jbartlett said:
I'll take this off-thread for any further notes and share it on my thread for this MB.
Just to throw it out there.. I upgraded to RC5 and my VM wouldn't make it to the desktop. QC35 2990wx. Just saw your thread so didn't try turning off PDO but downgraded to RC4 and it worked properly again. I had downgraded the version to 4.0 in the XML but it would just crap out when it was starting.
-
On my 2990wx my upgraded to the latest QEMU version and lost a ton of performance.. Trying to figure out what is going on but so far can't see anything wrong. Went from 400 points on CPUZ single threaded to barely 250..
Also this happened on 6.7.2 as well but when I boot in UEFI mode it takes forever to ever get a webpage.. talking like nearly 5 minutes at times.
-
39 minutes ago, MisterWolfe said:
I had the same issue using the NUT plugin. I fixed it by uninstalling and reinstalling the NUT plugin. I now have the UPS watt usage display on the dashboard.
I think i am just using the stock ups thing that comes with unraid.
-
The UPS plugin on the dashboard no longer shows the wattage consumption only % load. It would be nice if it could still show that.
-
1
-
Unraid OS version 6.9.0-beta22 available
-
-
-
-
-
in Prereleases
Posted
Interesting thank you