Hey, I am a new user just starting out. I have had this array on and it has been working fine for a couple of weeks now but I just notice that some of my data appears to be missing. It is on the default high water so it fills the drive to half then it starts on the next drive. As you can see on disk 2 it is used to 1.07Tb it was filled to 1.5Tb. Is there anyway to tell what was removed from the disk to change the fill level? The only thing I deleted was a VM, could this be the cause? I created several others (VMs) but they did not seem to have an effect on the disk usage. A parity check preformed earlier today shows no errors.

 

I I remember/recall correctly copies of VM's barely use any space since all that needs to be saved is library and database files and config files and folders...besides the changes between copied VM's.
It reads from all the same files and saves changes seperately.
Requiring minimum space and optimizing efficiency.

Don't quote me on this...I could be way off.
Many changes if saved can be found in logs here and there....
Which would be the question I would like to ask.

Sent from my GT-I9505 using Tapatalk

20 minutes ago, JimDijkstra86NL said:

I I remember/recall correctly copies of VM's barely use any space since all that needs to be saved is library and database files and config files and folders...besides the changes between copied VM's.
It reads from all the same files and saves changes seperately.
Requiring minimum space and optimizing efficiency.

This is what I found as well they take a couple of gigs at most. And the storage is dynamic i think, where it take up data as needed. And the VM I deleted was set up but unused.

Call me noob and unkowligable.....
Nowadays I am once again.
Other then logs what comes to mind is the spread of the IOPS/load over several pieces of hardware.
I mean, why place an extra drive if it is not gonna be used...thus I assume the devs implemented spreading of load over several if not all pieces of hardware.
Thus leveling of storage space somewhat to get the highest performance and the least of wear and tear of hardware over time.
(Hardware is getting SMART'er over the generations and versions of interfaces and connectors especially NAS/storage specific devices thus automated behavioral patterns where devices migrate data for efficient, performance and lifespan extension)

Hope that helps a little....I ain't the brainiac I once was...tho still trying to kick ass onto answering everything IT like I once used to.
Greet Jimmy ;-D

Anyone else with ideas and suggestions?
Please DO correct me, I have a craving and somewhat a need to learn it all over again.
(Will in near future own a qnap 873/673...first true and proper NAS

Sent from my GT-I9505 using Tapatalk

12 hours ago, JimDijkstra86NL said:

Other then logs what comes to mind is the spread of the IOPS/load over several pieces of hardware.
I mean, why place an extra drive if it is not gonna be used...thus I assume the devs implemented spreading of load over several if not all pieces of hardware.
Thus leveling of storage space somewhat to get the highest performance and the least of wear and tear of hardware over time.
(Hardware is getting SMART'er over the generations and versions of interfaces and connectors especially NAS/storage specific devices thus automated behavioral patterns where devices migrate data for efficient, performance and lifespan extension)

I think you have some assumptions here that don't really apply to unRAID. The OP already said he had the default High Water allocation, which is one of the settings that controls which drives unRAID decides to use and when for any particular User Share. Once unRAID writes a file to the array, it never does anything to automatically move things to balance storage space.

13 hours ago, greencode said:

As you can see on disk 2 it is used to 1.07Tb it was filled to 1.5Tb. Is there anyway to tell what was removed from the disk to change the fill level? The only thing I deleted was a VM, could this be the cause?

I see how much is used on disk2 from your screenshot, but I don't have any evidence it was ever filled to 1.5TB.

 

Was your VM stored on the array or on cache? People usually put their dockers and VMs on cache for performance reasons, but it looks like your cache is practically unused. unRAID will not use your cache disk by default, you have to configure things to use it.

 

If you post your diagnostics (Tools - Diagnostics, post complete zip) we would know more than we do now.

 

I/O errors are logged, but adding and deleting files isn't logged, except for what mover logs when moving files from (to) cache to (from) array.

 

Do you actually think you are missing any data?

4 hours ago, trurl said:

I see how much is used on disk2 from your screenshot, but I don't have any evidence it was ever filled to 1.5TB.

I don't have a screen shot to prove it, the only proof I can provide is that the 3rd disk is starting to be filled which only happened after the 2nd reach the high water mark. 

4 hours ago, trurl said:

Was your VM stored on the array or on cache? People usually put their dockers and VMs on cache for performance reasons, but it looks like your cache is practically unused. unRAID will not use your cache disk by default, you have to configure things to use it.

It was left at default. So it used the array.

4 hours ago, trurl said:

Do you actually think you are missing any data?

I have no idea, I have been going though the data and comparing it to my back ups and so far have not found a major discrepancy. However I only have gone though a few hundred gigs. Most of the data I don't really care about but my main concern is that I did something wrong or it is a symptom of a larger problem and would like to get it nailed down before I put critical data on it. I attached the log however I don't know how helpful it will be. I did restart the server on the hope it would be fixed.

tower-diagnostics-20170419-1704.zip

If I pulled the drive and scanned it with a data recovery program would it reveal missing data? For example recuva. 

Edited April 20, 20179 yr by greencode

Actually, there are some "default" user shares on a new install that will use cache, appdata and system. These are normally set to cache-prefer but it looks like you have them set to cache-yes for some reason. Your syslog doesn't show anything getting moved, at least not since the last reboot.

 

See here for an explanation of the different options for caching user shares:

 

1 hour ago, greencode said:

f I pulled the drive and scanned it with a data recovery program would it reveal missing data? For example recuva

I don't know if that particular utility will even work with XFS, I doubt it.

 

It is usually a bad idea to pull a drive from the parity array and let something else work on it, since if anything is changed on the drive it would invalidate parity.

 

If you have good backups then just doing the comparison you've been doing is probably simpler.