Compute Share Sizes Asynchronously

[lankanmon]

I made a post requesting help a few weeks ago about this issue that had no responses. However, I think this feature is beneficial. 

 

Something that has been bugging me for a while is the fact that hitting the Compute (or Compute All) button on the shares page will require the server to calculate the storage usage for that share. From my experience, this locks up the web UI while the size of the share on each disk is calculated. This tends to take a long time depending on how big the share is and will make the WebUI unresponsive. This is a massive productivity hit because I am usually only doing a compute of space if I am working on the settings of my server. This forces me to take a break and wait for the server to become responsive again.

 

Request:

If it is possible, please change the way that shares are computed, such that the calculation is done in the background (Asynchronously) and the results visible whenever they are done (maybe with a notification to let you know when it is ready).

 

If Implemented:

I do realize that calculations being made asynchronously have the potential for inaccuracy as more data may be written to disk in the meantime. Maybe have an Icon to show if the size computed is no longer valid (per drive in the results) for information. I think it would use a similar implementation that is currently used to show the orange triangle for unprotected drives.

 

That way, we can hit computer or computer all without having to worry about the delay that it will cause my workflow.

 

Thank you!

[pwm]

32 minutes ago, lankanmon said:

I do realize that calculations being made asynchronously have the potential for inaccuracy as more data may be written to disk in the meantime. Maybe have an Icon to show if the size computed is no longer valid (per drive in the results) for information.

Background computations would be nice and optionally a field showing when the last result was computed so the user can decide if it's good enough or if it would be meaningfull to request a recomputation.

 

I don't think the current computations will show better results if there are ongoing file updates - any code that traverses a directory tree will miss changes performed in the parts of the tree that has already been traversed. The only way to catch all is to capture all file write operations to compute how that changes the total. When there is only one share on a partition, then some file systems allows the disk use value to be picked up and used for free.

[1812]

You could just open a new browser window and continue with what you were doing.

 

But with that said, I would like having it not lock up the window I'm currently in.

[bonienl]

The use of Dynamix Cache Directories plugin would also accelerate the computations.

 

[lankanmon]

7 minutes ago, pwm said:

I don't think the current computations will show better results if there are ongoing file updates - any code that traverses a directory tree will miss changes performed in the parts of the tree that has already been traversed. The only way to catch all is to capture all file write operations to compute how that changes the total. When there is only one share on a partition, then some file systems allows the disk use value to be picked up and used for free.

1

 

Interesting... I am sure the Devs will be able to find the best way to handle this. It is a bit beyond my knowledge of how unraid tracks file changes. I just noticed that it is able to tell you that that the parity is invalid when the contents are updated outside of parity.

 

5 minutes ago, 1812 said:

You could just open a new browser window and continue with what you were doing.

 

But with that said, I would like having it not lock up the window I'm currently in.

 

For me, this does not work. Even opening up a new tab still waits for the operation to complete before that page loads (waiting for a connection).

 

2 minutes ago, bonienl said:

The use of Dynamix Cache Directories plugin would also accelerate the computations.

 

I will check that out... Thanks!

[StatMatt]

I hit the "Compute All" on the Shares tab literally 90 MINUTES AGO and it's still calculating. How could it possibly take that long? Could it cache the most recent result and display that with a timestamp and give the user the option to update that if it's not current enough?  If I navigate away from the Shares tab, it'll have to start over.  I'm trying to get the information to work on my file organization and the wait is extremely frustrating.

 

Reminds me of another semi-related UI cache issue that I might submit as it's own feature request. Whenever I get a notification from Fix Common Problems that I have plugin that needs to be updated, I go to the Plugins tab and then have to wait (what feel like) 5 minutes while it's "checking for updates" for every plugin.  I've never actually timed it, but it's bad enough that I leave the tab open and go work on something else.  Somewhere the system is checking that information often enough that the results are known to the Fix Common Problems plugin, so why isn't a cached result displayed on the UI?  There are a lot of things to like about UnRAID, but the UI can be maddeningly slow!