tazire Posted October 13, 2018 Share Posted October 13, 2018 Sorry just a quick question.. is it best practice to turn off dockers and vms when using this? just I am using it to encrypt a large existing array and as you can imagine it will take quiet some time. I have a few ppl who use the services regularly and am curious if having dockers running will cause any issues. Sorry if this is already answered I couldnt find it when i searched and googled it. Quote Link to comment
jbrodriguez Posted October 13, 2018 Author Share Posted October 13, 2018 Hi, Short Answer Use unbalance exclusively. No other dockers/vms/plugins/mover touching the disks. Long Answer The bottom line is if you're moving content outside unbalance when it's running, the assumptions it made may no longer be valid. That is, it thought you had 40Gb free on disk1, but a docker or mover dropped it to 35Gb. Maybe it will still work, but it's very possible that it won't (out of space, source not found, etc) If you know what you're doing, that you won't be touching the disks that unbalance is operating on, then keep running dockers/vms, but you want to be sure that doing so won't have side effects. 1 Quote Link to comment
tazire Posted October 15, 2018 Share Posted October 15, 2018 Thanks for the info. I have shut off everything that might add or move files on the array. Plex will stay active but it shouldnt change any drive. One more question. As i said earlier im using this to encrypt my drives. When im moving files to empty drives I have a few drives that are 100% full, save for about 500mb. Im not sure if this is over filling the hard drives if i wanted to defrag etc. or when im done is there a way to balance the data over the entirety of the array? Or is this totally fine to leave as is! Quote Link to comment
jbrodriguez Posted October 15, 2018 Author Share Posted October 15, 2018 The plugin reserves a minimum of 512Mb as a hard limit, but you can go higher in the settings. There are some forum members very knowledgeable on the hard disk subject matter and they prefer not to have disks "filled to the brim". On the other hand, if you use your hard disk as "read-only" (fill it with contents, start with a new disk, repeat), it shouldn't really be a problem, and you skip the fragmentation you mentioned. Quote Link to comment
tazire Posted October 15, 2018 Share Posted October 15, 2018 ah very good. I guess on the one hand the majority of the hard drives are going to be read only after the contents is added. Also i am formatting each drive so fragmentation shouldnt be an issue. Ill look into that hard limit when im done with the process. cheers for the response. Quote Link to comment
JonathanM Posted October 15, 2018 Share Posted October 15, 2018 22 minutes ago, jbrodriguez said: prefer not to have disks "filled to the brim". I'm not very knowledgeable on the subject, but I do know that you can run into issues with file system repair and other nasties if there isn't enough slack space for the file system to do its thing. Just recently there was an update in the XFS format, where disks formatted after a certain date reserved a much larger percentage of the total file system size, so if you had a disk filled to the brim with the old format, and tried to do a disk to disk copy to a newly formatted disk of the same size, you would be out of luck as there isn't as much free space available on the new format. I'd try to keep some portion free just to avoid headaches later. How much? Who knows. 512MB is probably safe, but I'd rather keep as much space free as my largest typical file. YMMV, IANAL, yada-yada, etc. 2 Quote Link to comment
jbrodriguez Posted October 16, 2018 Author Share Posted October 16, 2018 ...Just recently there was an update in the XFS format, where disks formatted after a certain date reserved a much larger percentage of the total file system size.... Thanks for presenting this scenario jonathanm. I was surprised by how much overhead xfs had, the last time I formatted a disk. I'm thinking about raising the hard limit to 1Gb, although one can always go above it in the settings, if so preferred. Sent from my iPhone using Tapatalk Quote Link to comment
m4f1050 Posted November 14, 2018 Share Posted November 14, 2018 Not sure if this has be addressed before, but I wanted to use unBALANCE today but the PLAN is showing moving everything from one drive to another, it's not splitting even though I have "most free" on my drives and shares. These are new drives and the only difference I see is that they do not have the folders created. Should I create my main share folders? Not sure how to solve this problem. Thanks! Quote Link to comment
jbrodriguez Posted November 15, 2018 Author Share Posted November 15, 2018 Hi, unBALANCE works at the disk level not share level, so it won't consider any share rules you set. I'd guess you're doing a scatter? The concept for scatter is to empy a disk by moving data to other disks you choose: you can choose one, some or all of them. It's up to you. Hope this helps. 1 Quote Link to comment
m4f1050 Posted November 17, 2018 Share Posted November 17, 2018 On 11/15/2018 at 5:21 AM, jbrodriguez said: Hi, unBALANCE works at the disk level not share level, so it won't consider any share rules you set. I'd guess you're doing a scatter? The concept for scatter is to empy a disk by moving data to other disks you choose: you can choose one, some or all of them. It's up to you. Hope this helps. Well, I have all my data on 1 drive (from another unRAID server) and I wanted to "balance" the data through all the drives, is that possible? Or could it be a future option? Quote Link to comment
BRiT Posted November 17, 2018 Share Posted November 17, 2018 16 minutes ago, m4f1050 said: Well, I have all my data on 1 drive (from another unRAID server) and I wanted to "balance" the data through all the drives, is that possible? Or could it be a future option? As said earlier, Select more than 1 DESTINATION drive. Quote Link to comment
jbrodriguez Posted November 17, 2018 Author Share Posted November 17, 2018 Right, as BRiT mentioned, choosing more drives will scatter the data across disks, but if the data you're trying to move fits one of the disks, it will put it all there.unbalance doesn't "balance"/fill disks evenly, that's more of an intrinsic feature of Unraid, by using share rules.Sent from my iPhone using Tapatalk 1 Quote Link to comment
_Shorty Posted November 18, 2018 Share Posted November 18, 2018 Also as stated earlier, there really isn't any benefit to spreading things out evenly among your disks. It's kind of counter-productive and wastes more electricity by requiring more drives to be spun up more of the time. You're really better off just letting the drives automatically fill up and move on to the next one as it sees fit. Quote Link to comment
jus7incase Posted December 1, 2018 Share Posted December 1, 2018 Dear all if I understand correctly the "Docker Safe New Perms" is installed by the plugin "Fix Common Problems", so I post my problem here. I have run Docker Safe New Perms on all disks except for the Cache disk (nothing to be moved). Then I wanted to use unBalance to move the content of one disk to another disk. During the planning phase unBalance would complain that it found directories with improper permissions and I shall run "Docker Safe New Perms". So how can this be since I did run "Docker Safe New Perms" in advance. It turns out that unBalance considers existing Set-U-ID oder Set-G-ID bits as problematic, and "Docker Safe New Perms" does not clear out these bits. I do not know whether these bits are actually a real problem for the unBalance procedure. But if they are, then "Docker Safe New Perms" should remove those bits, e.g. by setting the permissions to 00777. (Note the leading 2 zeroes! Yes this is correct, with just one zero it will NOT clear those bits!) This can be done manually as long as "Docker Safe New Perms" does not do it: find /mnt/disk?/ -type d -and -not -perm 0777 -exec chmod 00777 {} \; However, if those bits do not create a problem with the unBalane procedure, the unBalance should not report them as an issue. if you search like this, you will find the directories where SUID or SGID are set: find /mnt/disk?/ -type d -and -not -perm 0777 This is not what we want if we do not want to report those directories. Instead it should be used: find /mnt/disk?/ -type d -and -not -perm -0777 Note the "-" (minus) character in front of the 0777. Now additional bits being set will not be reported! It would be great to get feedback from the author of unBalance whether these bits should be cleared or whether the should rather not be reported. In the first case it would be good to get feedback from the author of the plugin "Fix Common Problems" whether clearing of the SUID SGID bit for directories can/will be implemented. I am going to cross-link both postings in both support threads. Fix Common Problems thread: Quote Link to comment
trurl Posted December 1, 2018 Share Posted December 1, 2018 45 minutes ago, jus7incase said: I am going to cross-link both postings in both support threads. Please don't do this. Which thread should we respond to? Should we all be expected to read both threads to see what others have said in response? In a more general case, you could have someone going to a lot of trouble to research a problem for you and write a long response in one thread, without being aware that someone else has already made a similar effort on the other thread. This is one reason why crossposting has been frowned upon on message boards since long before the World Wide Web. In future, if you really feel you must make the post in one thread and take responses there, then just post a link in the other thread. 1 Quote Link to comment
jbrodriguez Posted December 2, 2018 Author Share Posted December 2, 2018 Hi, I'm away from my development machine, so I can't look at the history.I'll take a look next week and report back.Sent from my iPhone using Tapatalk Quote Link to comment
ceyo14 Posted December 3, 2018 Share Posted December 3, 2018 Want to use this, especially now that a drive failed, but since I've installed this I can't uncheck the dry run option, I can only perform dry runs. Changed browser and computers and still can't uncheck it. what can I do to solve this? Quote Link to comment
jbrodriguez Posted December 4, 2018 Author Share Posted December 4, 2018 Want to use this, especially now that a drive failed, but since I've installed this I can't uncheck the dry run option, I can only perform dry runs. Changed browser and computers and still can't uncheck it. what can I do to solve this?I remember something like this happened before, not sure what was it.The check box is disabled until there's enough input for the operation, make sure about that.If you can try on Chrome, could you check the dev console for any errors?Last option, send me the log (/boot/logs/unbalance.log), to take a look.Sent from my iPhone using Tapatalk Quote Link to comment
jbrodriguez Posted December 7, 2018 Author Share Posted December 7, 2018 I took a look at the code with regards to SUID/SGID. It checks that permissions are "r--r--r--" or "rw-rw-rw-" (for files), "rwxrwxrwx" (for folders), so if SUID/SGID are set, it will report a file or folder permission issue. Not sure how to go about this, but SUID/SGID don't seem too common, I think ? Quote Link to comment
BRiT Posted December 7, 2018 Share Posted December 7, 2018 I had SUID/SGID initially on my array as it was setup from before and I think one of my downloaders had it set. You could use the following to get octal permissions and take the right-most 3 character of the first portion (not shown below): stat -c "%a %n" * Or more verbose to compare between Octal and Human Readable form: stat -c "%a %A %n" * Quote Link to comment
BRiT Posted December 7, 2018 Share Posted December 7, 2018 Very hackish, but it seems to work to for 3 and 4 character octal permission maps: stat -c "%a %n" * | awk '{print substr($1, length($1)-2, 3), $2}' Quote Link to comment
jbrodriguez Posted December 7, 2018 Author Share Posted December 7, 2018 Thanks BRiT ! I'll look into it. Quote Link to comment
publicENEMY Posted December 20, 2018 Share Posted December 20, 2018 I have 2 issues 1. When I scatter, the scatter process failed/not complete. I can see the target drive free space is reduced which means that some files have been moved. Unfortunately, the source drive space is not increasing which means that the moved files are not being deleted. 2. Later when I run plan and wait for it, I notice an error message pop up and then vanished with no way of retrieving the error message back. This are the error message (after several time quickly captured it). There are some permission issues with the folders/files you want to transfer 9 file(s)/folder(s) with an owner other than 'nobody' 9 file(s)/folder(s) with a group other than 'users' 9 folder(s) with a permission other than 'drwxrwxrwx' 0 files(s) with a permission other than '-rw-rw-rw-' or '-r--r--r--' You can find more details about which files have issues in the log file (/boot/logs/unbalance.log) At this point, you can transfer the folders/files if you want, but be advised that it can cause errors in the operation You are STRONGLY suggested to install the Fix Common Problems plugin, then run the Docker Safe New Permissions command How can I 1. Remove duplicate files generated by Unbalance? By the way, can you make it the important message not vanished? If its important why hide it? Thanks. Quote Link to comment
jbrodriguez Posted December 21, 2018 Author Share Posted December 21, 2018 10 hours ago, publicENEMY said: 1. Remove duplicate files generated by Unbalance? Move works by copying then deleting, but if a command fails, it doesn't delete the source by design, so your original files are intact. You can check the operation "log" in the History tab and check which command ('rsync') failed and act accordingly (delete target folder manually, retry the operation, etc.) Why did it fail ? What does the log say ? 10 hours ago, publicENEMY said: By the way, can you make it the important message not vanished? If its important why hide it? It's a fair point. Maybe I could just leave it open, you can always click to delete it. Quote Link to comment
maxse Posted December 25, 2018 Share Posted December 25, 2018 (edited) Can someone please help. I am having so many issues learning v6. I came from unrid v5 for 5 years without any issues... I am encrypting drives so I am first using the unbalance plugin to move files from one drive to another, then exncrypt the empty one, etc... For some reason everything has come to a crawl lately. I have disabled any parity and don't even have any parity drive assigned. I am getting max speeds of about 20MB/s! I just goan i5 8400 processor, LSI IBM m1015 controller. Never had issues with such slow speed before! Can someone help me out and let me know what's happening? I feel like this is slowing down my other dockers that I installed. I have 16 gigs of ram too. What's happening and why is everything so slow, even without parity! Edited December 25, 2018 by maxse Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.