_Shorty

There is a difference between dedicated network storage and local storage that you are actively using. You need a certain amount of free workspace on your local machine to ensure it continues to operate properly in all conditions, as it might need some space for temporary files or extra pagefile space at any given time, but you do not have that limitation on a network machine dedicated to storing files. Filling such drives until they are full is not a concern, and is making things as efficient as possible. Empty space is wasted space in this instance. I can't think of any reason for maintaining a large amount of free space on such drives. As for performance falling off, well, that's how hard drives have always been. More sectors in outer cylinders than there are in inner cylinders dictate this. It's (usually) rotating at a constant speed, so more sectors passing the read heads in the outer cylinders translates directly to more speed, and as you work your way in the number of sectors reduces and so does the transfer speed. Actively going out of your way to avoid that is a bit much. Why not simply by a 10 TB drive and only use the first half then? I mean, you're then multiplying the cost of your storage space by two, since you are ignoring half of it, but hey, at least the performance drop off is gone, right? The unRAID system does a pretty nice job of using up the space you give it. I wouldn't be so concerned with micromanaging it, as your return on that time investment is rather small. Both for you thinking about how to manage it, and for the time/energy used to constantly move files around. Consolidating a few things here and there in order to avoid spinning up more drives can certainly be helpful, and save you having to wait a bit longer each time you access that stuff, but beyond that I wouldn't worry about how things are stored. Just doesn't seem worth it.

Full drives are slower? Not really. The graph shared above is typical of any hard drive. They're fastest at the start of the disk because there are more sectors per cylinder in the outer cylinders, and then as you move to the inner cylinders there are less sectors per cylinder, so they're a little slower there. But this has nothing to do with the drives being full. It just has to do with the location that you're reading/writing at the time. You could be reading/writing to the inner cylinders even when the drive is completely empty, for example. You'd still be seeing that relatively slower performance even though the drive is empty. You're will probably be wasting more time and energy moving files around than you'll save by just leaving the system alone to do its job. Consolidating some similar/related files can stop you having to spin up multiple drives and lower your time waiting around for drives to spin up, but other than that I don't really see any upside to micromanaging the files.

I'm curious. What would be the logic behind the operation? What do you think you would gain by spreading data out?

More than one of us has shared that we have another source of checksums to check against utilizing other checksum tools, and when this plugin has said there is a corrupted file we have checked against our own checksum with another checksum utility to find the file data has not actually changed at all. The plugin, for whatever reason(s), is returning bad checksum errors when in fact the file and checksum for it has not actually changed. One idea is that a file has changed and the linux change notification simply didn't trigger for some reason, so the plugin didn't get notificed of a change and consequently didn't update the checksum to reflect the new/changed data. And then when it is verifying the checksum it currently gets does not match the stored one and it reports a corrupted file. Unfortunately, with files that haven't actually changed and have been verified via the other checksum utility/utilities to still actually be the same as it was before, this indicates that something is not operating as intended with the plugin. But if the dev can't reproduce this it makes it difficult to track down and fix, and it is my understanding that this has not been something the dev has seen locally.

I don't think there is any upside to doing this. A downside is spreading out your data can mean more drives being spun up when they may not have been had you not spread the data out. Just add a new drive and let it do its thing.

superloopy1, this has been reported by multiple users. I'd say 99.999% chance you do not have any corrupt files at all. The tool isn't working properly. This thread is over 100 pages long, but just read over the last few pages and you'll see you're not the only one reporting this.

While it would be ideal if trim were working, it isn't really an issue worth worrying about here for this usage. Even if you're not using the default of "moving" your cached files every night, I wouldn't worry about it. Unless you've got pretty old SSDs, they're rather proactive now about doing their own garbage collection anyway. It was a lot more important with earlier drives, but these days the housekeeping is taken care of by the drive's firmware even if it isn't getting told to trim now and then.

Like I say, while great in theory, the plugin just doesn't seem to be that useful, given this issue. It randomly gives me "mismatches" on files that have not changed. It's not like it is missing a notification from the OS that a file has changed. Files haven't changed, and it spits out messages about files being corrupt that most definitely are still intact.

What makes this strange is that this is happening with files that have been static for a very long time, and nothing has updated them. In my particular case, while I have lots of backups on this particular unRAID machine that do indeed get updated from time to time, I also use it as my storage pool for TV shows and movies which is accessed by a home theatre PC running Kodi. And those files never change. Yet I would get "corrupted" messages on some of those files, too.

Is there something else responsible for notifying the plugin of that, or is that its own responsibility? Seems to not be happening often enough to be a concern, anyway.

I suspect this plugin can't be trusted. Whenever I would encounter "corrupted" files I could not find any issue when hashing with some other tool. When multiple hashing tools agree, and this plugin disagrees, you can deduce that there seems to be some issue with the plugin.

It's just buggy. Stop it and restart it and the CPU usage will be fine again. Or just stop using it, as I did. I can't be bothered to babysit it and restart it all the time.

This tool does nothing but report whether or not the hash for the file is still the same as the last time it checked the hash or created the hash. If the hash is now different the tool does nothing except inform you of it thinking the hash is now different. No, it does nothing to help you repair the file. If the file is truly corrupt because of some problem with your hard drive you might be able to cure things by rebuilding from parity, perhaps after replacing a faulty drive if that's really the cause. You could also use some kind of file-level parity utility if you wanted to add some more redundancy to safeguard against things that may happen in the future. QuickPAR comes to mind. It is similar in theory to a parity drive. You can build parity files of a selectable amount of redundancy, and rebuild/repair if anything ever gets corrupt. This has to be done before you have any suspect files, though. You build parity files when everything is fine, and then if there is ever a time where things are not fine then you have redundant info to hopefully help you out of a jam.

Also as stated earlier, there really isn't any benefit to spreading things out evenly among your disks. It's kind of counter-productive and wastes more electricity by requiring more drives to be spun up more of the time. You're really better off just letting the drives automatically fill up and move on to the next one as it sees fit.

It doesn't work as designed when it says the hashes don't match but all the other hash tools say the hashes match. Sorry if you don't like to hear that, or if you don't care to believe user reports, but it is occasionally giving errors stating bad hashes when the file still matches the original kept elsewhere. Being rude because it works on your machine is kinda humourous.

*shrug* I'm not the only one that has it not working properly. If it's working for you, that's great for you. Doesn't change the fact that it is broken for me and other people. Have you tested it versus other hashing tools, or are you just assuming it is working because it hasn't reported any errors to you on its own?

Jammer, no, I haven't even bothered looking. It would be a nice additional safety net, for sure. But with the Dynamix one acting flaky I just turned it off. Not much point when it isn't properly doing what it should be. I haven't looked to see if there are any alternatives that perform a similar job.

Not having trim isn't really a huge deal. It's made out to be a big deal, but it really isn't. At best, you'll never notice any difference because the controller might do some automatic maintenance at some point anyway. At worst, you'll see slightly degraded performance sometimes, but not likely anything worth worrying about, either. Ever since SSDs were a thing, and trim came to be, it has been made out to be something absolutely essential to have. It really isn't worth worrying about. edit: When they take care of the issue on their own without intervention from the OS it is referred to as garbage collection. When the OS triggers it, this is a trim command being sent. Both accomplish the same end. One just happens automatically at certain intervals without a command having to be sent from the OS. Really, nobody should be worried about this all that much unless they're actually in some situation where bleeding every single last bit of performance from a drive 24/7 is important.

It's pretty easy to move data where you'd like to have it with unBalance. Wouldn't it be better just to leave your current data where it is so only those two drives get spun up when that data is needed, rather than having the chance of spinning up all the drives at some point?

Frankly, I don't think this thing can be trusted to do what it is intended to do. If it's flagging even one file as being corrupt, as in my case, when other file hashing tools come back and say both copies actually have the same hash then there's a problem somewhere with this tool. I wouldn't necessarily put too much stock into it saying you've got 295 corrupted files. If you just replaced the drive and consequently things were just rebuilt from parity I would be more inclined to think your data is just fine.

I guess I should be happy mine isn't crashing when it produces incorrect hashes.

Hmm, a bit confused.

Keith, the web interface is just a remote way to control/start a process that is local to the server. Once you start it, the remote portion of the job is finished. Your client computer is no longer involved with what the server is doing. Close it and be on your way. Check later to make sure things finished if you wish.

*puts hand up* Same thing here. Old bookmark wasn't working, manually change it to https and it works again, at least after going through Chrome's roadblocks trying to get me to not fall for the website not actually being secure.

Thanks for the reply. I think maybe this was a false alarm. I was browsing from a Windows machine, and Explorer for some reason had gone into that mode where it tries to guess what types of files are in a "folder" and reads some amount of information from every file in an attempt to "help" you in some way. Resetting all "folders" fixed this and it began reading directories at normal speed again. Stupid Windows bug making it forget that I turned that crap off already, causing me to have to turn it off again. *sigh*