[Support] bobbintb - rmlint

[bobbintb]

Support thread for rmlint Docker container

 

rmlint finds space waste and other broken things on your filesystem and offers to remove it.

Features:

Finds…

• …Duplicate Files and duplicate directories.
• …Nonstripped binaries (i.e. binaries with debug symbols)
• …Broken symbolic links.
• …Empty files and directories.
• …Files with broken user or/and group ID.

 

Differences to other duplicate finders:

• Extremely fast (no exaggeration, we promise!)
• Paranoia mode for those who do not trust hashsums.
• Many output formats.
• No interactivity.
• Search for files only newer than a certain mtime.
• Many ways to handle duplicates.
• Caching and replaying.
• btrfs support.

 

It's worth noting this Docker compiles rmlint from source. As such, it should always be the latest version of rmlint. The base image used for this Docker is based on Ubuntu and their repo has rmlint but it is several years old. The base image is linuxserver/docker-baseimage-guacgui which uses Guacamole to provide a webUI. By default the webui has no username and password set.

 

Application Name: rmlint

Application Site: https://rmlint.readthedocs.io/

Docker Hub: https://hub.docker.com/r/bobbintb/docker-rmlint-unraid

Github: https://github.com/sahib/rmlint

Post any questions or issues relating to this docker in this thread.

Edited February 15, 2021 by bobbintb

[Chunks]

I've tried getting this to run, and after downloading/installing the image, I am stuck.

 

When I start the container, the log gets filled with a TON of "Exec Format Errors". It looks like they're just looping after that.

 

These are the first few lines of the log, in case that's helpful. I can provide the rest of the log as well if it would help..

 

2021-04-11T11:34:39.048094872Z foreground: warning: unable to spawn /docker-mods: Exec format error
2021-04-11T11:34:39.078252944Z [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
2021-04-11T11:34:39.145482632Z [s6-init] ensuring user provided files have correct perms...exited 0.
2021-04-11T11:34:39.148697163Z [fix-attrs.d] applying ownership & permissions fixes...
2021-04-11T11:34:39.150926422Z [fix-attrs.d] done.
2021-04-11T11:34:39.152785844Z [cont-init.d] executing container initialization scripts...
2021-04-11T11:34:39.155464041Z [cont-init.d] 01-envfile: executing... 
2021-04-11T11:34:39.157618520Z foreground: warning: unable to spawn /var/run/s6/etc/cont-init.d/01-envfile: Exec format error

 

I googled a little, and maybe this is a problem with the shell command? I dunno.... I know just enough about linux to be a danger to myself

[bobbintb]

I'm not sure. I just did a fresh pull and can't recreate the issue. Can you post your template settings and the shell command it is using?

[Chunks]

I'm not sure I'm doing this correctly, but I grabbed the XML. I don't think there's anything sensitive in there, the only change I made was to the path.

 

Here's the command that runs (successfully as far as I can tell):

 

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker create --name='rmlint' --net='bridge' -e TZ="Europe/Berlin" -e HOST_OS="Unraid" -e 'GUAC_USER'='' -e 'GUAC_PASS'='' -e 'PUID'='99' -e 'PGID'='100' -p '8322:8080/tcp' -v '/mnt/user/Photos/':'/root':'rw' -v '/mnt/user/appdata/rmlint':'/config':'rw' 'bobbintb/docker-rmlint-unraid'

d86f95afa658ec3ac59acf94325c058d73b9afe1a140f01a0991049069b4e849

The command finished successfully!

 

---

I really hope none of this data is sensitive. I think it's not  Thanks for checking this out, I appreciate it. There's a chance it's a system issue. I dunno.

rmlint.xml

[bobbintb]

No, nothing should be sensitive. The command looks ok. Did you get the XML file from the flash drive? Are you using the webUI to edit the template or editing the XML directly?

 

Anyway, it looks like you manually edited the XML and I believe that's the issue. Manually editing it in Windows with a text editor will change the line ending and it can cause issues.

Edited April 13, 2021 by bobbintb

[Chunks]

Regarding the formatting, I wasn't sure how to grab the XML you asked for, (normally the Flash share is disabled) so I used the "Tools --> Config File Editor" to grab it from the flash drive, then pasted it into Notepad++ on windows. That explains the formatting.

 

But you made me wonder about the xml, so I deleted the docker and that, and started over - everything works.

 

Anyway, I have no idea what happened, or why the first time I removed/readded it, it didn't fix it. Sorry for taking your time but I do appreciate the help.

 

/Chunks

[bobbintb]

No problem. I'm glad it's working.

[Updog]

I know it's been a while since this was made available, so I'm not certain this project is still active, but I'm watching the container log at the first run attempt and saw this interesting entry that gave me pause...

(shredder:435): dconf-WARNING **: 14:38:58.020: failed to commit changes to dconf: Failed to execute child process “dbus-launch” (No such file or directory)

 

This appears only after I modify and change settings in the search parameters / what to do with duplicates screen and click "apply". They don't look like they're saved, the log implies they aren't, and it looks like the choices I made don't end up being passed to rmlint in the way I'd hoped. I tried to search only files >1MB, but that gets reset when you navigate away from the page. I wanted to use hardlinks to recreate the previously hardlinked structure that was disrupted when I copied to a new disk with rsync. The hardlink option in the gui appears to still be selected, but that handler value, like the filesize filter, is not recorded and passed along as expected.

 

 

In the end, instead of hardlinking my duplicates as I was expecting, it just removed them. I ran it on a relatively small directory for testing so the impact was really negligible, but for anything more substantial.

 

Despite Hardlink being chosen, -c sh:handler=remove is set instead of -c sh:handler=hardlink.

I've included a copy of the command that was ultimately run when turning off the dryrun toggle.
 

 

 

 

 

 

 

[bobbintb]

Sorry, I just noticed this. Try 

apt-get install dbus-x11

I'll have to update the docker because I guess they discontinued automatic builds.

Edited October 9, 2022 by bobbintb

[MrTyton]

Hi - I'm encountering the same thing as Updog. Any changes that I make in the "Settings" menu don't end up reflected in the actual command that gets generated

 

```

# Your command line was: rmlint --no-with-color -T duplicates --see-symlinks --partial-hidden -c sh:handler=remove --hardlinked --crossdev --algorithm blake2b --size 1-18446744073709551615 --max-depth 512 --replay /tmp/shredder-ljo8vu68/shredder.replay.json -o sh:/tmp/shredder-ljo8vu68/shredder.sh -o csv:/tmp/shredder-ljo8vu68/shredder.csv -o json:/tmp/shredder-ljo8vu68/shredder.json /root/Testing

```

 

 

I tried running the command that you posted, `apt-get install dbus-x11`, however  it fails with `unable to locate package dbuss-x11`.

 

Is there a fix for this? Thanks.

[bobbintb]

17 hours ago, MrTyton said:

Hi - I'm encountering the same thing as Updog. Any changes that I make in the "Settings" menu don't end up reflected in the actual command that gets generated

 

```

# Your command line was: rmlint --no-with-color -T duplicates --see-symlinks --partial-hidden -c sh:handler=remove --hardlinked --crossdev --algorithm blake2b --size 1-18446744073709551615 --max-depth 512 --replay /tmp/shredder-ljo8vu68/shredder.replay.json -o sh:/tmp/shredder-ljo8vu68/shredder.sh -o csv:/tmp/shredder-ljo8vu68/shredder.csv -o json:/tmp/shredder-ljo8vu68/shredder.json /root/Testing

```

 

 

I tried running the command that you posted, `apt-get install dbus-x11`, however  it fails with `unable to locate package dbuss-x11`.

 

Is there a fix for this? Thanks.

You have to run `apt-get update` first before you run `apt-get install dbus-x11`. That will update the list of what packages are available to install, which is why it can't find it. Sorry, it's kind of habitual and standard practice to do that first so I don't always remember to explicitly say that.

FYI, I'm not sure how much longer I will support this. For one, Docker Hub is requiring payment for features that used to be free, which means I may have to figure out an alternative and I'm not sure I want to spend time on it because I don't use rmlint anymore. It's only been pretty solid, as I've had only three support requests in over two years. Additionally, I've been working on a new plugin that will be a replacement for this. It's getting close to being good enough for a beta release so keep an eye out in you are interested.