January 27, 201115 yr Hi I think du is taking a long time on my unraid (I also think that solving this will make my backup much faster using rsync) Here's the results of a test, on a folder that contains 6000 files, totaling 1735MB On my main computer (not unraid) it takes 0.3s. On unraid it takes 31 seconds Main computer (not unraid) Coruscant:~ matthew$ time du -sm Backgrounds.aplibrary/ 1735 ./Backgrounds.aplibrary/ real 0m0.284s user 0m0.013s sys 0m0.183s unraid root@centax:/mnt/disk2/matthew/Pictures/Libraries# time du -sm Backgrounds.aplibrary/ 1737 Backgrounds.aplibrary/ real 0m31.785s user 0m0.070s sys 0m0.160s Is that expected? Or is there something wrong with my box? Cheers!
January 27, 201115 yr What happens when you run it on unraid immediately afterwards? You don't say anything about the disk configuration on your workstation. This is all about metadata so is down to IOP capacity of your disks and caching that's in the way. I'd bet that your workstation has the data cached which is why it's returning so quickly. Unraid probably hasn't. Rerun it immediately on unraid, see if you get a similar result to your workstation.
January 27, 201115 yr What happens when you run it on unraid immediately afterwards? You don't say anything about the disk configuration on your workstation. This is all about metadata so is down to IOP capacity of your disks and caching that's in the way. I'd bet that your workstation has the data cached which is why it's returning so quickly. Unraid probably hasn't. Rerun it immediately on unraid, see if you get a similar result to your workstation. The time will also depend on the disk itself. If it is sleeping it must first spin up to speed. That alone could take a fair amount of the time in your test.
January 27, 201115 yr Good point - and probably makes more sense. That's a long time for such a small number of files even without caching. If you add a rough disk spin up time in there though it starts to look much more feasible.
January 27, 201115 yr Author The test isn't good evidence, but I've found du -sm to be consistently slow on my unraid for the last month or so. I'm not sure about before then. It can take several hours on a large folder (e.g. ~300GB of varying size files) but I've never found it to take that long on similar size folders on my main computer. My unraid is running on Atom, but it's hardly hitting the CPU/memory at all when doing these operations.
January 27, 201115 yr What happens if you re-run your du command again on unraid immediately after the initial slow one? How much memory does your system have? What does the directory hierarchy / structure look like? Is the disk spun up when you issue the request? What does the output of free -m look like?
January 27, 201115 yr Author Thanks for taking the time to help me Repeating the command is quick after the initial slow one. All disks are spun up before I run the command. System has 1GB memory. root@centax:~# free -m total used free shared buffers cached Mem: 978 925 52 0 179 455 -/+ buffers/cache: 291 686 Swap: 0 0 0 I have 1 parity (2TB) drive, and two data drives (1TB, 1.5TB) I have two user shares, but both are 'basic' - no includes/excludes. High water. As far as I can tell, my disks are in good condition and happy. However, both data drives are about 90% full. Here's a couple more tests, which I'd expect to execute in a few seconds. root@centax:/mnt/user/matthew# time du -sm Music 710521 Music real 7m44.589s user 0m0.250s sys 0m1.560s root@centax:/mnt/user/matthew# find Music/ | wc -l 47311 root@centax:/mnt/user/matthew# time du -sm LivingRoom/ 77825 LivingRoom/ real 1m48.380s user 0m0.060s sys 0m0.470s root@centax:/mnt/user/matthew# find LivingRoom/ | wc -l 11930
January 27, 201115 yr Repeating the command is quick after the initial slow one. All disks are spun up before I run the command. That's saying to me the original pass just wasn't cached. root@centax:/mnt/user/matthew# time du -sm Music 710521 Music real 7m44.589s user 0m0.250s sys 0m1.560s root@centax:/mnt/user/matthew# find Music/ | wc -l 47311 root@centax:/mnt/user/matthew# time du -sm LivingRoom/ 77825 LivingRoom/ real 1m48.380s user 0m0.060s sys 0m0.470s root@centax:/mnt/user/matthew# find LivingRoom/ | wc -l 11930 What are the hierarchies like? Do you have subdirectories under Music or are all your files just directly under it? Are you running the cache_dirs script? If not try it and see what happens.
January 27, 201115 yr Author Even uncached, I don't think it should take that long. Music contains an iTunes library (including movies and tv shows), so the majority of files are 4 or 5 directory levels deep. I don't have the cache_dirs scripts. I'll look in to it.
Archived
This topic is now archived and is closed to further replies.