Optimbic
-
Posts
17 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by Optimbic
-
-
I'm using unmanic to remove unwanted audio streams and the issue is that it copies everything to the temp cache and then back to the array, regardless of whether any work needs to be done. So even for files that have no audio tracks to be removed. So if I let it rip through my library it will essentially copy-paste like 50TB of data which is obviously not desirable.
Did I misconfigure something or is this the expected behavior?
Is there some way we can analyse files without doing this copy+write action? Just read the files in their current location?
-
Quick question, is there a way to scan (no writing, just reading) video files and report the audio stream sizes by language?
I'm using unmanic to remove unwanted audio streams and the issue is that it copies everything to cache and then back to the array, regardless of whether any work needs to be done. So even for files that have no audio tracks to be removed. So if I let it rip through my library it will essentially copy-paste like 50TB of data which is obviously not desirable.
I haven't used tdar before so I'm just wondering if it's possible for it to scan the library (without copy+write) and report back to me the audio stream sizes so I can then tell it which files to process.
-
Do we need to restart the server version 2023.02.11 to work? Because I deleted some stuff and they are not showing up in the recycle bin. I'm not 100% sure I had upgraded to the latest version before deleting though so it might also be that. I don't have anything to delete right now I can't check.
-
when using the "Remove audio/subtitle streams by language" is there a way to tell it which languages to keep rather than which ones to remove? Essentially just invert the logic.
-
22 hours ago, alturismo said:
may try this as you use the GUI function (i never used it, only amc here ...
)
add another path to the docker like
/storage2 <> /mnt/user0/movies/
now in the web GUI set storage2 as target ?
or may play with this settings here, as you use manual webui processing anyway, test these functions ...
Thanks, will try that. The copy action should work, although then I have to go in and delete things manually again so it's the same amount of work as rename + mv operation from terminal.
When you use the AMC you don't have these kinds of problems? Or are your shares set up in such a way that it doesn't matter?
-
1 hour ago, Djoss said:
All you are discussing should not be a concern for FileBot or any applications working with /mnt/user/. The issue/behaviour you are raising seems to be related to unRAID internals.
For example, you mentioned that moving a file from a cache-enabled share to a cache-disabled shared does not produce the expected result. From FileBot's point of view, the file has been moved from one folder to the other. However, under the hood, unRAID did not place the file to the expected disks.
Yes exactly, it seems that everything is "working as intended", except that the files are physically in places that violate the rules of the share.
Although the fact that Filebot creates empty folders in the correct physical location, tells me that maybe something does try to move them but Unraid tells it not to, since they files are already on the share. At that point, as far as Unraid or Filebot is concerned, everything is fine.
So essentially the question is how do we get Filebot and unraid to work together so that the share rules are not violated?
-
19 hours ago, Optimbic said:
yeah I see what you mean about it being expected behavior, it's just not what I want it to do
I guess filebot creates the folders/files in the cache and then when it goes to move them it doesn't need to because they are already in the share. For Filebot and Unraid, the files are where they should be.
I'm not using the AMC script so I left those paths empty.
I did turn on the cache for the movie share and I'm using the mover until I figure a way to get around this.
I wonder if I could somehow tell filebot to move the files to user0/ shares, that should technically bypass the cache, but the files are already in cache so I guess it's not going to work.
So I've been thinking about this some more.
If filebot does a rename operation which changes the path of the file (which is the same as a move operation) in situ, it will always bypass the share's cache and allocation settings.
For example, let's imagine somebody has their mnt/user/downloads/movies/ share to include only disk 5, and the mnt/user/movies/ share's allocation setting is disks 1 to 4 high water. So disk 5 should never have a mnt/user/movies/ folder structure.
Filebot is instructed to rename a file from
mnt/user/downloads/movies/movie_folder/movie_file
to
mnt/user/movies/movie_folder/movie_file
Filebot will create the movies share on disk 5 and the renamed files will not be moved to the proper disk under the high water calculations. They will stay on disk 5 since filebot has created the movie share folder structure on disk 5.
However, I can't find anybody else discussing this issue in the threads so I suspect I am somehow misunderstanding this? Can someone let me know if I'm wrong? I'm probably doing something wrong aren't I? I mean, I can't be the only one that has encountered this issue?
It's not a showstopper or anything, for me personally at least since I have my downloads share to be on all disks high water and the cache issue can be bypassed by enabling cache on the destination share and then run mover.
However it's a bit of an interesting puzzle on how to optimize the operations so it's more efficient. Obviously we can just rename the files without changing the paths and then do a mv command from the terminal, but it's extra steps.
-
25 minutes ago, alturismo said:
well, i would say thats the expected and wanted behaviour most likely (well, even all i know off
)
may rather show also how you mounted the shares, you may can change this behaviour when u really split them.
this part here (not splitted here, i really really want to move instead copy ...
)
when i remember correctly and had it wrong setted up, i made two mounts for the docker or so ...
even easier ... just use cache yes for your movie share too and run the mover there too ... but i guess thats not a option cause otherwise you would run it like this already.
yeah I see what you mean about it being expected behavior, it's just not what I want it to do
I guess filebot creates the folders/files in the cache and then when it goes to move them it doesn't need to because they are already in the share. For Filebot and Unraid, the files are where they should be.
I'm not using the AMC script so I left those paths empty.
I did turn on the cache for the movie share and I'm using the mover until I figure a way to get around this.
I wonder if I could somehow tell filebot to move the files to user0/ shares, that should technically bypass the cache, but the files are already in cache so I guess it's not going to work.
-
On 12/18/2020 at 11:31 PM, Pixel5 said:
Also i had a strange thing happen when i used the GUI to change some names.
I set it up to sort the files into some folders and when it ran this on a different share which does not have this folder structure yet it did create it but on a share level.
So i have a share for TV Series which cache disabled and a downloads share that is cache only, when i ran my renaming i suddenly had the TV series structure on my cache drive and had to set it to cache yes and run mover to have the files moved of the drive.
I'm having the same issue with filebot not moving files (from GUI) when renaming. Did you ever resolve this issue?
I have my shares set up like this
mnt/user/downloads/movies (Cache:Yes)
mnt/user/movies (Cache:No)
When renaming files from the downloads share, I tell Filebot to rename and move them to the movies share.
This works fine only if the mnt/user/downloads/movies files are already off the cache.
If the files are on cache, it creates:
1) mnt/user/movies/"Movie_name (date)" an empty folder on the array
2) mnt/user/movies/"Movie_name (date)" a populated folder with the movie file in it, in the cache
This orphans the files there since mover will not touch them without changing the cache settings for the movie share.
I guess this is because filebot doesn't do a copy+delete operation, so for the move operation essentially filebot creates its own share folder structure in the cache which then gets stuck in there. Filebot and unraid think that the files are where they should be since they are in the share now.
How do we get around this?
-
1 hour ago, apandey said:
I solved this by writing a custom rsync script. It basically runs rsync in dry run mode to itemize the actions it would take, then ignore directory creation greedily and instead do tgst lazily when copying files. It also handles permissions as per unraid shares
You can see here if interested: https://github.com/ashishpandey/scaffolding/blob/master/roles/godaam/files/scripts/mirror.sh
This way, you can keep the split level that you want and things fill up properly
Nice, thank you very much!
-
9 minutes ago, itimpi said:
It is the Unraid OS. Rsync will not know which drive a file is on if you are writing to a User Share.
Note that Unraid never moves files between array drives. The Split level setting is applied to new files at the point the file is first created.
Thanks, that clears it up!
I have to import 25TB from external hard drives, so they are all going to be new files, so I guess it should work with the share/splitlevel/allocation setup I described above?
ED: Ah I see, it's because I said "always move stuff to another drive", yeah that was just badly worded, it's for a first time import.
-
1 hour ago, JorgeB said:
Yes.
Correct, it will leave empty folders on disk1, that's not a big deal and if you want there are easy ways to get rid of them once the transfer is complete.
Cool, thank you very much! Is it actually rsync that will create new folders in another drive? or something in the unraid OS ? I mean to say, how does rsync know that "this file should go in this folder that I already created on disk 1, but disk 1 is full, and the split level rules allow me to put this folder on another disk, so I'll create the same empty folder on disk 2 and copy the file there" ? Just curious how it works internally
Also, in theory if I set up my shares and split level correctly so it can always move stuff to another drive, I could do it with out setting split level to "all", right? So I don't have to go hunting files spread out across drives.
For example make two shares :
/media/movies/movie_name_date/ - split level 2
/media/series/series_name/season_number/ - split level 2 - allocation "most free"
is rsync agnostic of allocation rules? will it attempt to cram all series into a single disk and give up if it fills up? if it is then I guess I can set split level 3 and do some manual re-arrangement after the fact. Or copy the series into batches that I know will fit into a drive.
-
28 minutes ago, JorgeB said:
Set split level to all temporarily.
is "all" the "automatically split any directory as required"?
isn't that potentially going to make a mess of putting files everywhere?
and isn't rsync still going to put all the folders on the first drive since it doesn't know that the drive will get full once files start transferring?
sorry If I'm being dense, still new to this. thanks for the help
EDIT: Wait I think I got it, it will create the empty folders on the first drive but since they are not constrained by split level it will move them somewhere else once space starts filling up with the actual files? is that how it works?
-
I understand that rsync will first create empty folders and then start moving the files that live in those directories. Since empty folders don't take up space, this can cause an issue where the copy will eventually fail since the disk will run out of space.
Is there a way to get around this other than manually estimating the weight of files and copying in batches that will fit in the largest drive?
)
[Support] Djoss - FileBot
in Docker Containers
Posted
Thanks, I'll play around with AMC and see.
Yeah I also don't want the copy+write, I want it to move stuff.