egtrev Posted December 21, 2016 Share Posted December 21, 2016 Hi, I have a folder within a share in UNRAID that is mirrored with a local folder in Windows. When I check the properties in Windows of both folders, the folder on UNRAID shows a massive difference between Size (150GB) and Size on Disk (200GB) the same mirrored folder locally on the Windows machine does not. Is this normal? Something to worry about? Why is the Size on Disk so high for the folder on UNRAID? Thanks Pics of properties attached below. Quote Link to comment
gubbgnutten Posted December 21, 2016 Share Posted December 21, 2016 Normal and nothing to worry about. Windows is simply misinformed about allocation sizes of the networked drive so the displayed value can be way off. Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 I just realized my system is showing the same thing ... a few examples: Size: 147GB => Size on Disk: 204GB Size: 90KB => Size on Disk: 18.0MB Size: 327GB => Size on Disk: 517GB I could, of course, delete one of the entire backup folders (e.g. the 327GB) ... look at the UnRAID used/free stats; then re-copy it and look again to see the REAL amount of space it's using => but is there a Linux command I can simply run on the server that will show this? I'm really surprised I'd never noticed this before -- I guess I'd not done a "Properties" of the folders before ... at least none that had this big discrepancy. Not all folders have this problem -- any idea why?? [ Quote Link to comment
gubbgnutten Posted January 2, 2017 Share Posted January 2, 2017 I just realized my system is showing the same thing ... a few examples: Size: 147GB => Size on Disk: 204GB Size: 90KB => Size on Disk: 18.0MB Size: 327GB => Size on Disk: 517GB I could, of course, delete one of the entire backup folders (e.g. the 327GB) ... look at the UnRAID used/free stats; then re-copy it and look again to see the REAL amount of space it's using => but is there a Linux command I can simply run on the server that will show this? I'm really surprised I'd never noticed this before -- I guess I'd not done a "Properties" of the folders before ... at least none that had this big discrepancy. Not all folders have this problem -- any idea why?? [ Pretty sure all folders have the "problem", just that it is not as visible depending on the file sizes and amount of files. I assume there are 18 files in the 90KB folder? Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 Yes, there are 18 files in the 90kb folder -- clearly Wndows "thinks" there's a 1MB min allocation size. HOWEVER ... I was "fiddling" a bit and discovered that this is NOT a problem with Windows 7 -- only with Windows 10. Not only does Windows 7 show the correct results ... but it completes a "Properties" window [i.e. right-click on a folder and select Properties] DRAMATICALLY faster than Windows 10 does. For example, the two results below show the same folder -- one done in Windows 10; on in Windows 7. Windows 7 completed the Properties window in ~ 1 minute; Windows 10 took nearly 10 minutes. And as you can see, the Windows 7 result is correct. Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 And just for grins, here's the tiny backup that only has 18 files ... Quote Link to comment
egtrev Posted January 2, 2017 Author Share Posted January 2, 2017 HOWEVER ... I was "fiddling" a bit and discovered that this is NOT a problem with Windows 7 -- only with Windows 10. Not only does Windows 7 show the correct results ... but it completes a "Properties" window [i.e. right-click on a folder and select Properties] DRAMATICALLY faster than Windows 10 does. Yes, I've just tried Windows 7 and it's shows correct and completes way quicker as you have mentioned compared to Windows 10. Strange! Quote Link to comment
Squid Posted January 2, 2017 Share Posted January 2, 2017 The difference between size and size on disk becomes far, far worse if there are hardlinks involved in the folder in question, because Windows calculates the size of each link thinking that its a separate file. In the case of my Appdata share, Windows believes that the size on disk is ~1.8TB, which isn't too bad considering that my entire cache drive is a whopping 240GB Quote Link to comment
limetech Posted January 2, 2017 Share Posted January 2, 2017 This value is being reported by Samba wrong and because of it, if you check the properties of directory it can get wildly invalid, eg, 40MB of small files shows up as 1.5GB on disk. The problem stems from this Samba parameter: allocation roundup size (S) This parameter allows an administrator to tune the allocation size reported to Windows clients. The default size of 1Mb generally results in improved Windows client performance. However, rounding the allocation size may cause difficulties for some applications, e.g. MS Visual Studio. If the MS Visual Studio compiler starts to crash with an internal error, set this parameter to zero for this share. The integer parameter specifies the roundup size in bytes. Default: allocation roundup size = 1048576 Example: allocation roundup size = 0 # (to disable roundups) Often there are doc errors in Samba so if we set this thing to 4K it may or may not result less "improved Windows client performance." Some of you please add this line to your smb-extra.conf file (Settins/SMB Settings/SMB Extras): allocation roundup size = 4096 or even allocation roundup size = 0 and let me know if you notice any performance differences. 1 Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 Tom -- thanks for looking at this. I tried changing the SMB allocation parameter -- set to both 4096 and then 0 with very similar results. There's still a HUGE performance difference between Windows 7 and Windows 10, but at least the massive difference in "size on disk" is gone ... although interestingly it's still not shown as the same number in the two OS's. The following results are from both Windows 7 and Windows 10 after changing the allocation parameter to 0. Note that the huge difference is gone, but the results are still not the same in the two OS's. Also, the time to generate the Properties window was about 15 seconds in Win7, and nearly 3 minutes in Win10 !! Definitely STRANGE !! Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 Tom, One other question: How can I test this change in v5? Do I just add the "allocation roundup size = 0" line to the network.cfg file on the flash drive? Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 ... and yet another thought: Changing the allocation roundup seems to effectively "fix" the "size on disk" issue. But I have to wonder why there's such a huge difference in performance between '7 and '10. Any ideas? [Different SMB version perhaps?] Any thoughts on a network configuration change in '10 that might help? This is not, of course, a real problem => and changing the allocation roundup parameter effectively "fixes" the massive glitch in "size on disk" (although it's still strange that '7 and '10 give different results) ... but the performance difference between '7 and '10 is perplexing. I have no idea why it takes over 10 times as long for '10 to compute the Properties window. Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 ... In the case of my Appdata share, Windows believes that the size on disk is ~1.8TB, which isn't too bad considering that my entire cache drive is a whopping 240GB Just for grins, does changing the allocation roundup to 0 resolve this for you? Definitely interesting to see a 1.8TB "size on disk" on a 240GB disk Quote Link to comment
limetech Posted January 2, 2017 Share Posted January 2, 2017 The "correct" setting for this should be: allocation roundup size = 4096 because all our file systems used a 4K block size. When set to 4096 do you see a big disparity between win7/win10? I'm not so much interested in how long it takes to populate the Properties for a directory, instead I want to know what impact this setting might have on transfer rates. Also, the way storage space is calculated is just how we do it on the Shares page (and how 'du' command does it) - by using 'stat' to fetch the size of each and every file in the directory. This is going to produce a great deal of network traffic and can be affected by whether the inode for those files exists already in RAM on the server or not. 1 Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 I changed the setting to 4096 with the following results ... "Size on Disk" is exactly the same as with a 0 setting -- i.e. it's fine. But there's still a difference in the reported numbers for '7 and '10 ... definitely STRANGE. The actual Size of the folder I used to test this is 158,161,341,405 bytes. The "Size on Disk" is shown as 158,199,072,768 bytes in Windows 7, and 158,320,988,160 bytes in Windows 10. Not significant, of course, but still strange that they'd be different. Performance wise, there's still a BIG difference. Windows 7 took 16 seconds to show the finished Properties page. Windows 10 took 2:46 (166 seconds) ... or just over ten times as long. This folder is on a v6.2.4 dual parity server. I have one server still on v5.0.6 that has the same issue -- does changing the allocation rounding just require adding the "allocation roundup size = 4096" line to the network.cfg file on the flash drive? I presume making this change there will also resolve the issue. Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 ... I'm not so much interested in how long it takes to populate the Properties for a directory, instead I want to know what impact this setting might have on transfer rates. I copied a 16GB file from the server to see if changing this setting had any impact ... and it doesn't seem to have changed anything. Copied at ~ 110MB/s, which is what I typically see (basically network limited). I agree how long it takes to populate the Properties window doesn't really matter -- it's a bit frustrating KNOWING that it's so much longer than it needs to be ... but not something I really EVER do, so it doesn't matter. It IS nice to have the size disparity resolved (but also strange that the results are different in '7 and '10). Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 Just for grins (with the allocation rounding set to 4096) ... Vista populates the Properties window in the same time as Win7, and has exactly the same result. XP actually populates it a bit quicker (about 12 seconds) ... and has exactly the same result as '7 and Vista. Windows 98 takes 40 seconds to populate the Properties window, and has yet-another result for "on disk" !! (158,179,085,312) Windows ME takes 45 seconds to populate the window, and has the same result as '98 Windows 2000 Pro takes 12 seconds, and has the same result as XP, Vista, and '7 Windows 95 doesn't work correctly with a folder that large -- displays the # of files and folders okay, but totally hoses the size of the folder (and doesn't show a "size on disk"). So ... excluding the ancient Win 9x OS's, all Windows OS's from 2000 - Windows 7 work very nicely and have identical results; Windows 10 gives slightly different results and takes ten times as long to produce them. Quote Link to comment
CHBMB Posted January 2, 2017 Share Posted January 2, 2017 Windows 10 gives slightly different results and takes ten times as long to produce them. Which is why I now use Ubuntu.... Quote Link to comment
garycase Posted January 2, 2017 Share Posted January 2, 2017 ... I have one server still on v5.0.6 that has the same issue -- does changing the allocation rounding just require adding the "allocation roundup size = 4096" line to the network.cfg file on the flash drive? Answered my own question by just trying it -- this does NOT change the results, so I assume there's some other way to change the parameter on v5. So ... what do I have to change ?? Quote Link to comment
limetech Posted January 3, 2017 Share Posted January 3, 2017 ... I have one server still on v5.0.6 that has the same issue -- does changing the allocation rounding just require adding the "allocation roundup size = 4096" line to the network.cfg file on the flash drive? Answered my own question by just trying it -- this does NOT change the results, so I assume there's some other way to change the parameter on v5. So ... what do I have to change ?? Wow thanks for the detailed analysis! Probably there's a simple setting somewhere to speed up win10. For unRaid-5 you can type 'smbd -v' to get the samba version number and then try to find the smb.conf man page for that version; could be they didn't support that setting then... Quote Link to comment
Lev Posted July 30, 2017 Share Posted July 30, 2017 On 1/2/2017 at 1:32 PM, limetech said: The "correct" setting for this should be: allocation roundup size = 4096 Confirmed, this makes Windows 10 report as expected now 4K size (it was previously using 1MB). UnRaid is version 6.4rc7a, smbd v4.6.6 Quote Link to comment
Meph Posted December 27, 2017 Share Posted December 27, 2017 (edited) I can confirm this change works! This should be a setting that is made in unRAID when it is installed. Because default: allocation roundup size = 1048576 creates problems in shares when trying to copy files and reports wrong size used in Windows. I tried to copy 800GB of data to my unRAID server but got the message that the data would not fit even though i had 2TB free.. This made me find this thread... The correct setting should be: allocation roundup size = 4096 Setting this was a little unclear cause another guy talked about changing in a file on the USB... Here is the correct way to change it (It's often not explained in this forum i have noticed): In the unRAID web GUI (v6.3.5) click Settings -> SMB (under Network Services) then you add "allocation roundup size = 4096" (without the " ") in the text box "Samba extra configuration".. After that im not 100% sure if the settings will take effect or if you have to stop and then start the array.. I ended up with rebooting unraid after that it is working correctly! Finally! Maybe someone could report this as a bug (or change request)? Or I will when i get the time in a few weeks or so... Below you can see the that the allocation size has changed.. Size on disk: now shows 4KiB instead of 1MiB! Edited December 27, 2017 by Meph Quote Link to comment
Inshakoor Posted February 17, 2018 Share Posted February 17, 2018 This is fantastic! I noticed this when I first setup unRAID a few years ago but searches at that time didn't provide a solution. I can confirm that the solution provided using Meph's method of applying it works like a champ. The only other thing I had to do was stop the array and then start it again. I did not have to reboot. Thanks for this! Been driving me nuts :). Quote Link to comment
L0rdRaiden Posted March 31, 2018 Share Posted March 31, 2018 On 1/3/2017 at 1:19 AM, limetech said: Wow thanks for the detailed analysis! Probably there's a simple setting somewhere to speed up win10. For unRaid-5 you can type 'smbd -v' to get the samba version number and then try to find the smb.conf man page for that version; could be they didn't support that setting then... I'm having a similar issue here What has the solution at the end? it was fixed in unraid? Quote Link to comment
limetech Posted March 31, 2018 Share Posted March 31, 2018 5 hours ago, L0rdRaiden said: What has the solution at the end? it was fixed in unraid? Fell through cracks. Added to prelease bug list. 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.