MAM59 Posted August 7 Share Posted August 7 22 hours ago, jbrodriguez said: iirc, Unraid's web ui aggregates ? "english is not my natural language" 🙂 What I mean is that if you create a pool with more than one disk, UNRAID shows the pool name but also shows the single disks of the pool with names like "pool1", "pool2" and so on (in case it is named "pool"). But for copy purposes unbalanced should only offer "pool" (the total of all disks of that pool). Quote Link to comment
jbrodriguez Posted August 9 Author Share Posted August 9 On 8/7/2024 at 1:36 PM, MAM59 said: But for copy purposes unbalanced should only offer "pool" (the total of all disks of that pool). yes, that's kind of the issue Quote Link to comment
jbrodriguez Posted August 9 Author Share Posted August 9 hi, published 2024.08.09 - fixed an issue where disks with no items (gather) caused a crash (this was an issue reported on github) - updated packages - improved speed calculation (after rate change) (i thought i had published this already 🤷♂️) Quote Link to comment
MAM59 Posted August 10 Share Posted August 10 (edited) Hmm, it would be nice if you could focus on that "missing source pools" problem. it becomes a bit annoying over the years... And, why does the "forum entry for unbalanced" point to "diskmv" ? Edited August 10 by MAM59 Quote Link to comment
jbrodriguez Posted August 10 Author Share Posted August 10 (edited) 5 hours ago, MAM59 said: Hmm, it would be nice if you could focus on that "missing source pools" problem. it becomes a bit annoying over the years... i absolutely agree with you, it's just that i haven't found a workable solution to the issue, there's obviously a way to do it (the web ui can distinguish them), alas it has eluded me so far i remember reading the web ui code, but couldn't quite grasp it i'll take another stab, now that i updated a couple of things in the app 5 hours ago, MAM59 said: And, why does the "forum entry for unbalanced" point to "diskmv" ? some previous versions of unbalanced (unBALANCE then) used diskmv internally, since it was a robust solution to transfer files around via the command line i switched to rsync later on (diskmv also uses rsync internally), mainly to have more control over the rsync process (to show ui progress, etc) i left the reference in the app, as a thank you to Freddie (fellow unraid member, creator of diskmv) (maybe i have it pointing to the wrong url ? will check 😀) Edited August 10 by jbrodriguez Quote Link to comment
MAM59 Posted August 10 Share Posted August 10 2 hours ago, jbrodriguez said: i remember reading the web ui code, but couldn't quite grasp it i'll take another stab, now that i updated a couple of things in the app I would take a (hopefully helpful) look too. But this is PHP stuff? Not one of the languages I use and understand. Sorry. But maybe I take a small look for regular expression which I still prefer to blame for the bug. They should look alike in every language. Quote Link to comment
_Shorty Posted August 11 Share Posted August 11 On 8/10/2024 at 7:30 AM, jbrodriguez said: i absolutely agree with you, it's just that i haven't found a workable solution to the issue, there's obviously a way to do it (the web ui can distinguish them), alas it has eluded me so far i remember reading the web ui code, but couldn't quite grasp it i'll take another stab, now that i updated a couple of things in the app It might not hurt trying to have Chat-GPT explain the web ui code. You could also give it your code, and have it suggest a fix to yours to get it to behave in a similar manner. It might not be good to rely on Chat-GPT for large chunks of code, but it can certainly be helpful in overcoming a roadblock. Quote Link to comment
jbrodriguez Posted August 12 Author Share Posted August 12 2 hours ago, _Shorty said: It might not hurt trying to have Chat-GPT explain the web ui code. You could also give it your code, and have it suggest a fix to yours to get it to behave in a similar manner. It might not be good to rely on Chat-GPT for large chunks of code, but it can certainly be helpful in overcoming a roadblock. that's a great idea !! php is definitely not my forte, i spend more time, checking what functions do than actually following the code however, i asked for some help in the programming section here and itimpi gave me a great suggestion that i will be exploring, hopefully this will help. Quote Link to comment
jbrodriguez Posted August 13 Author Share Posted August 13 i just released 2024.08.13, which should solve the pool detection issue i don't have a zfs pool so couldn't test it, let me know how it goes 1 Quote Link to comment
MAM59 Posted August 14 Share Posted August 14 There they are! GREAT JOB! (No ZFS Stuff over here too, wait for somebody else to check) Thanks! 🥰 Quote Link to comment
MAM59 Posted August 14 Share Posted August 14 Now that the main problem is gone, I would like to present a minor glitch: Look a the "file" "FTP". In reality it is a folder. An empty one, but still a folder. Therefor the icon in front of "FTP" is debateble. I was confused and opened a shell to check the content of the "file", just to see, that is a directory. 1 Quote Link to comment
jbrodriguez Posted August 14 Author Share Posted August 14 16 hours ago, MAM59 said: An empty one, but still a folder. this should be fixed in 2024.08.14 1 Quote Link to comment
MAM59 Posted August 15 Share Posted August 15 (edited) great! tnx again. well done 😘 Edited August 15 by MAM59 Quote Link to comment
methanoid Posted August 15 Share Posted August 15 Was looking at the Gather function but it doesnt seem to do what I want... My "tv shows" files are in /mnt/data/media/tv/?????. Obviously my "tv" folders are on more than one drive I want to gather all of the episodes etc of a single show into folders on the various disks. But Gather seems to be to gather all the TV shows onto a single HDD. I have 200+ shows so dont want to have to select them one at a time. Any ideas?? Quote Link to comment
jbrodriguez Posted August 15 Author Share Posted August 15 2 hours ago, methanoid said: I want to gather all of the episodes etc of a single show into folders on the various disks a little too advanced for what unbalanced can support atm i really want to implement pluggable algorithms, that is, some lua code that defines selection of source, selection of targets, planning and transferring, while the ui is taken care of by unbalanced, gather/scatter would be the first 2 it's just an idea, hopefully some time down the line i can flesh it out a bit more 1 Quote Link to comment
malaki86 Posted August 16 Share Posted August 16 Suggestion: Add support for transferring to/from Unassigned drives 1 Quote Link to comment
beantje Posted August 17 Share Posted August 17 I have used unbalenced to move data from disk1 to other disks, its done but disk1 is not empty, and i have no space left as it duplicated the files? any idea how to fix this? Quote Link to comment
jbrodriguez Posted August 17 Author Share Posted August 17 3 minutes ago, beantje said: its done but disk1 is not empty, and i have no space left as it duplicated the files? if you go to the history (assuming you did a scatter / move), do you have errors in your transfer operation ? Quote Link to comment
beantje Posted August 18 Share Posted August 18 On 8/17/2024 at 7:39 PM, jbrodriguez said: if you go to the history (assuming you did a scatter / move), do you have errors in your transfer operation ? well i tried again, but stopped the transfer, so i don't see the logs from before im first trying to find out how to delete the duplicates now. no space to run again Quote Link to comment
jarken Posted August 20 Share Posted August 20 is safe "gather" a share to move the data from 2 disks to another diferent disk while that data is in use? Quote Link to comment
jbrodriguez Posted August 20 Author Share Posted August 20 3 hours ago, jarken said: is safe "gather" a share to move the data from 2 disks to another diferent disk while that data is in use? if in use means reading that data, it will work (although slower) if it means writing to it, the results will be unpredictable 1 Quote Link to comment
johnny.ink Posted August 21 Share Posted August 21 I found the updated version from this year and installed and removed the old version. The issue I have is I have tried 2 scatters whilst mover is not going and dockers are all shut down. I did a scatter from disk1 for movies and videos and told unblanaced to move them to disk8. According to the logs it looks like it was successful but it appears nothing has moved to disk 8 2024/08/21 11:28:23 disk(/mnt/disk5):fs(xfs):size(9998694297600):free(5354610081792):blocksTotal(2441087475):blocksFree(1307277852) 2024/08/21 11:28:23 disk(/mnt/disk6):fs(xfs):size(9998694297600):free(6613137870848):blocksTotal(2441087475):blocksFree(1614535613) 2024/08/21 11:28:23 disk(/mnt/disk7):fs(xfs):size(9998694297600):free(9838385901568):blocksTotal(2441087475):blocksFree(2401949683) 2024/08/21 11:28:23 disk(/mnt/disk8):fs(xfs):size(4998844026880):free(4963957579776):blocksTotal(1220420905):blocksFree(1211903706) 2024/08/21 11:28:23 disk(/mnt/cache):fs(xfs):size(499863638016):free(269321502720):blocksTotal(122037021):blocksFree(65752320) 2024/08/21 11:28:23 scanning:disk(/mnt/disk1):folder(Videos) 2024/08/21 11:28:24 issues:owner(98):group(0):folder(0):file(64) 2024/08/21 11:28:24 items:count(3):size(594.01 GB) 2024/08/21 11:28:24 scatterPlan:items(3) 2024/08/21 11:28:24 scatterPlan:found(/mnt/disk1/Videos/_Misc):size(583586594028) 2024/08/21 11:28:24 scatterPlan:found(/mnt/disk1/Videos/XMAS Specials):size(7630432909) 2024/08/21 11:28:24 scatterPlan:found(/mnt/disk1/Videos/Music Videos):size(2797862717) 2024/08/21 11:28:24 scatterPlan:issues:owner(98),group(0),folder(0),file(64) 2024/08/21 11:28:24 scatterPlan:Trying to allocate items to disk8 ... 2024/08/21 11:28:24 scatterPlan:ItemsLeft(3):ReservedSpace(1073741824) 2024/08/21 11:28:24 0s 2024/08/21 11:28:24 scatterPlan:3 items will be transferred. 2024/08/21 11:28:24 scatterPlan:willBeTransferred(Videos/_Misc) 2024/08/21 11:28:24 scatterPlan:willBeTransferred(Videos/XMAS Specials) 2024/08/21 11:28:24 scatterPlan:willBeTransferred(Videos/Music Videos) 2024/08/21 11:28:24 scatterPlan:ItemsLeft(0) 2024/08/21 11:28:24 scatterPlan:Listing (9) disks ... 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk1):no-items:currentFree(241.37 MB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk2):no-items:currentFree(283.43 GB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk3):no-items:currentFree(181.85 GB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk4):no-items:currentFree(460.79 GB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk5):no-items:currentFree(5.35 TB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk6):no-items:currentFree(6.61 TB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk7):no-items:currentFree(9.84 TB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/disk8):items(3)-(594.01 GB):currentFree(4.96 TB)-plannedFree(4.37 TB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 [583.59 GB] /mnt/disk1/Videos/./_Misc 2024/08/21 11:28:24 [7.63 GB] /mnt/disk1/Videos/./XMAS Specials 2024/08/21 11:28:24 [2.80 GB] /mnt/disk1/Videos/./Music Videos 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 disk(/mnt/cache):no-items:currentFree(269.32 GB) 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:24 2024/08/21 11:28:24 ========================================================= 2024/08/21 11:28:24 Bytes To Transfer: 594.01 GB 2024/08/21 11:28:24 --------------------------------------------------------- 2024/08/21 11:28:36 packet {ID: Topic:scatter:move Payload:map[bytesToTransfer:5.94014889654e+11 chosenFolders:[Videos] ended:2024-08-21T11:28:24.577388756-04:00 fileIssue:64 folderIssue:0 groupIssue:0 ownerIssue:98 started:2024-08-21T11:28:23.197378394-04:00 target: vdisks:map[/mnt/cache:map[bin:<nil> currentFree:2.6932150272e+11 dst:false path:/mnt/cache plannedFree:2.6932150272e+11 src:false] /mnt/disk1:map[bin:<nil> currentFree:2.41373184e+08 dst:false path:/mnt/disk1 plannedFree:5.94256262838e+11 src:true] /mnt/disk2:map[bin:<nil> currentFree:2.834269184e+11 dst:false path:/mnt/disk2 plannedFree:2.834269184e+11 src:false] /mnt/disk3:map[bin:<nil> currentFree:1.81853413376e+11 dst:false path:/mnt/disk3 plannedFree:1.81853413376e+11 src:false] /mnt/disk4:map[bin:<nil> currentFree:4.60792672256e+11 dst:false path:/mnt/disk4 plannedFree:4.60792672256e+11 src:false] /mnt/disk5:map[bin:<nil> currentFree:5.354610081792e+12 dst:false path:/mnt/disk5 plannedFree:5.354610081792e+12 src:false] /mnt/disk6:map[bin:<nil> currentFree:6.613137870848e+12 dst:false path:/mnt/disk6 plannedFree:6.613137870848e+12 src:false] /mnt/disk7:map[bin:<nil> currentFree:9.838385901568e+12 dst:false path:/mnt/disk7 plannedFree:9.838385901568e+12 src:false] /mnt/disk8:map[bin:map[blocksUsed:1.45023167e+08 items:[map[blocksUsed:1.42477196e+08 location:/mnt/disk1 name:/mnt/disk1/Videos/./_Misc path:Videos/_Misc size:5.83586594028e+11] map[blocksUsed:1.862899e+06 location:/mnt/disk1 name:/mnt/disk1/Videos/./XMAS Specials path:Videos/XMAS Specials size:7.630432909e+09] map[blocksUsed:683072 location:/mnt/disk1 name:/mnt/disk1/Videos/./Music Videos path:Videos/Music Videos size:2.797862717e+09]] size:5.94014889654e+11] currentFree:4.963957579776e+12 dst:true path:/mnt/disk8 plannedFree:4.369942690122e+12 src:false]]]} 2024/08/21 11:28:36 {Started:2024-08-21 11:28:23.197378394 -0400 EDT Ended:2024-08-21 11:28:24.577388756 -0400 EDT ChosenFolders:[Videos] OwnerIssue:98 GroupIssue:0 FolderIssue:0 FileIssue:64 VDisks:map[/mnt/cache:0xc0004b5f20 /mnt/disk1:0xc0004b5f50 /mnt/disk2:0xc000198090 /mnt/disk3:0xc0001980f0 /mnt/disk4:0xc000198150 /mnt/disk5:0xc000198db0 /mnt/disk6:0xc000198f30 /mnt/disk7:0xc000198fc0 /mnt/disk8:0xc000199050] Target: BytesToTransfer:594014889654} 2024/08/21 11:28:36 running Move operation ... 2024/08/21 11:28:36 command started: (src: /mnt/disk1) rsync -avPR -X --dry-run "Videos/_Misc" "/mnt/disk8/" 2024/08/21 11:28:37 command:retcode(0):exitcode(0) 2024/08/21 11:28:37 Command Finished 2024/08/21 11:28:37 Current progress: 98.24% done ~ 0s left (0.00 MB/s) 2024/08/21 11:28:37 command started: (src: /mnt/disk1) rsync -avPR -X --dry-run "Videos/XMAS Specials" "/mnt/disk8/" 2024/08/21 11:28:37 command:retcode(0):exitcode(0) 2024/08/21 11:28:37 Command Finished 2024/08/21 11:28:37 Current progress: 99.53% done ~ 0s left (0.00 MB/s) 2024/08/21 11:28:37 command started: (src: /mnt/disk1) rsync -avPR -X --dry-run "Videos/Music Videos" "/mnt/disk8/" 2024/08/21 11:28:38 command:retcode(0):exitcode(0) 2024/08/21 11:28:38 Command Finished 2024/08/21 11:28:38 Current progress: 100.00% done ~ 0s left (0.00 MB/s) 2024/08/21 11:28:38 unbalanced - MOVE operation completed Started: Aug 21, 2024 11:28:36 Ended: Aug 21, 2024 11:28:38 Elapsed: 1.504s Move operation has finished Transferred 594.01 GB at ~ 0.00 MB/s --------------- I'm sure I am being stupid but any help is appreciated. Quote Link to comment
johnny.ink Posted August 21 Share Posted August 21 8 minutes ago, johnny.ink said: I'm sure I am being stupid but any help is appreciated. I believe I found the fact that the "dry run" button was checked, it would appear that the historical jobs can't be rerun from the history tab to do actual move correct? 1 Quote Link to comment
Gaertner Posted August 24 Share Posted August 24 Hello, I've had a problem several times now. When moving several TB of data, the hard drive from which the data is being read suddenly reports CKSUM errors. The files with the CKSUM error then have a current date (changed) on the reading hard drive. Strangely, the file doesn't seem to be damaged. At the moment it was mkv files that were affected. I can open and play the file. I have to move the file so that ZFS reports everything OK again. It seems that something happened when the file was moved, because the file on the target hard drive also has the changed date. So what is the problem? Hardware? Unbalanced? So far, no use case has had CKSUM errors. Always unbalanced. Oh, the last time the CKSUM error occurred again, a parity check was done beforehand, with no errors. I recently replaced some hard drives with 8 TB. I had the same problems with the old hard drives. The hard drives don't report any errors. Do you have an explanation for my problem? Hardware: Supermicro X13SAE-F Intel i7-13700K Kingston 128GB 4800MT/s DDR5 ECC ASM1166 (6xHDD) Mainboard (8xHDD) 3x WDC WD120EDGZ-11 11x WDC WD80EDAZ-11T Log: command:end:error(exit status 23) command:retcode(0):exitcode(23) Command Finished skipping:deletion:(rsync command was flagged):(/mnt/disk7/******/******) Quote Link to comment
WobbleBobble2 Posted August 25 Share Posted August 25 (edited) UPDATE: I found a solution! I just had to keep running the planning step repeatedly and it seems to be caching the information each time it runs, even if it times out. So the last time I ran it (just now) it only took a few minutes to run and then I could successfully get to the Transfer step and start it! ORIGINAL POST: Hello, I am trying to move 10tb of photos, about 500k individual files, from one drive to another, but Unbalanced keeps timing out during the planning Plan phase. The plan phase will appear to complete and I can proceed to the Transfer step and start the file move. However the Transfer step then freezes without showing any progress and when I finally give up and reload the page I am taken back to step 1 (Select) as if I had done nothing. I am thinking this is related to how long it takes to scan so many tiny files and unbalanced is just timing out. Is there anything I can do to prevent the time outs? I tried turning on notifications in the hopes that I could click on the notification to get back to the completed Plan step, however, I get the same symptoms as above so no dice. Any help much appreciated! Edited August 25 by WobbleBobble2 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.