T0a

Dear paperless-ng fellows, we now have a dedicated paperless-ng support thread. Come over and share your setup or get help with your paperless-ng instance. See you there.

Execute Management Utilities Paperless-ng offers a few management utilities that might come handy in some situations. You can either execute them from the command line or create custom scripts with the user scripts plugin from available via the CA. The latter allows you to schedule the task execution e.g. to invoke the document exporter for backups on a regular basis. The command is as follows: docker exec -it paperless-ng <command> <arguments> # Example: Execute the document exporter and remove files that do not belong to the current export document_exporter /usr/src/paperless/export -d Have a look at the documentation for the available commands and individual arguments.

Overview: Dedicated support thread for the Docker template paperless-ng provided via the selfhosters/unRAID-CA-templates repository. Project Page: https://github.com/jonaswinkler/paperless-ng Documentation: https://paperless-ng.readthedocs.io/en/latest/ Registry: https://hub.docker.com/r/jonaswinkler/paperless-ng/ Changelog: https://paperless-ng.readthedocs.io/en/latest/changelog.html This is the official paperless-ng Docker support thread. Feel free to ask questions, share your experience with paperless-ng or describe your paperless setup at home. I try to update this main post regularly based on your feedback. From here on, I will use the terms paperless and paperless-ng interchangeable. 1. What is paperless-ng and how does it differ from the original paperless? Paperless-ng is a fork of paperless, adding a new interface and many other changes under the hood. The original project paperless hasn't received a lot of updates and bug fixes in the past. Even pull requests are not merged for some time now (Update: 26.08.2021, The old paperless repository has been archived). Here are a few key features of paperless-ng: New front end build with Angular. It features full text search with scored and highlighted results, savable filters, a dashboard, and document uploading on the landing page Mobile support is also almost there. Some layouts don't work yet on small screens New mail consumer that supports multiple accounts and custom filters and actions. Fully tested! Paperless-ng trains a neural network on your documents and assigns tags and correspondents automatically, if you instruct it to do so Updated dependencies More tests of critical backend parts A proper task processing queue that can consume multiple documents in parallel. Consumption of many documents is now blazing fast on multi core system. Fixed much of the consumer code, so that it does not block the database during consumption, for instance 2. How to Install Download and install a Redis container from the community application store (CA) Download and configure the paperless-ng container from the CA Make sure you point the container to your Redis instance. Use your actual IP and not localhost, because the reference is resolved in the container. In case you need to pass a password to Redis, use the following connection string redis://:[PASSWORD]@[IP]:6379 instead. At the moment Redis doesn't support users and only provides authentication against a global password. You can pass anything as a username, including the empty string as in my example here. To configure a password for your Redis container, set 'redis-server --requirepass "your-secret"' as post arguments on the Redis docker container. Also make sure to not use any special characters. Otherwise, the connection string might not be readable by paperless. Create a user account after this container is created i.e. from Unraids Docker UI, click the paperless-ng icon and choose Console. Then enter the command "python manage.py createsuperuser" in the prompt and follow the instructions. Alternative, set 'PAPERLESS_ADMIN_USER' and 'PAPERLESS_ADMIN_PASSWORD' in your paperless-ng docker template. With the later approach, it might be easier to find your password to sensible documents stored in paperless. 3. My personal workflow I use the iOS app ScannerPro to scan my documents and upload them via the app to a web DAV target on my Unraid server. The web DAV target is then mounted in the container as consume directory. Further, I use the pre and post hooks to execute web hooks in order to check via Home Assistant whether the processing failed for a uploaded document. Home Assistant sends then notifications about the import status to my phone. This way I can throw away the physical document without worrying about it being not imported. How does your workflow look like? Feel free to share it in this thread. Here you can also find the official recommended workflow for managing your documents with paperless-ng. 4. FAQ 4.1 Why does the consumer not pick up my files? The consumer service uses `inotify` to detect new documents in the consume folder. This subsystem, however, does not support NFS shares. You can disable `inotify` and use a time-based polling mechanism instead (see `PAPERLESS_CONSUMER_POLLING`. If set to a value n greater than 0, inotify is disabled and the directory is polled every n seconds). 4.2 How to customize paperless-ng? Paperless-ng does support much more environment variables than the Unraid template offers. You can find them in the documentation here. 4.3 What scanner do you use for paperless-ng at home? A list of scanners used by our community: iPhone with ScannerPro app; one time purchase (@T0a) More will be added when you share your scanner Paperless-ng also maintains a list of recommended scanners. Feel free opening a pull request over there to add your recommended scanner to the documentation too. 4.4 Can I use paperless-ng on a mobile device? Mobile support in paperless-ng is also almost there. Some layouts don't work yet on small screens. There is also a mobile app in pretty early development stage. Though, it is only available on the Android store yet. 4.5 What is the future of the original paperless template in Unraid? At some point, I will probably remove the paperless template and close its support thread. 4.6 How to configure PostgreSQL as a database? See this post on how to configure PostgreSQL in the template. The official documentation gives further migration steps needed.

Are the web UI logs and the docker logs are both empty? Have you checked the javascript console of the browser? Do you find an issue there? Does the document contain sensitive information? I'm asking because the previous question aren't no help then you could open an issue in the paperless-ng repository.

Yes, see here. Make sure you create a backup first.

So problem solved?

I haven't monitored the whole conversation. But first let's make sure your configuration works and you can backup files. To do so, please open the console of the borgmatic container and execute a backup yourself via "/usr/bin/borgmatic prune create -v 1 --stats 2>&1" (without the ""). For testing purpose only include a small file in your backup via the `config.yaml`. Let me know how this goes. Just to make sure: Do you know how cron works and what the entries mean? The backup is only triggered when the cron expression evaluates to true. You may have to restart the container after making changes to the crontab.txt file.

Please post your issue to the paperless-ng support thread. See my post: "Issues with the Unraid setup go here". This thread here is meant for issues with the original paperless docker container. My first case would be that your document is password protected. Further diagnostics in the other thread then.

What does the borgmatic Docker log tell you? For me the crontab.txt in the right place is all I need for the automation (despite the config.yaml and keys for sure)

Yes, just in this moment. Here is what I did for my Linux VM: Create new connection with Protocol SSH Set parameter Hostname in the networks section to the IP address of the Linux machine Enter user credentials in the Authentication section Tick the checkbox "send WoL packet" Set the field "Mac address of the remote host" to the MAC address of your virtual machine (see vm template) Set the field "Broadcast address for WoL packet" to you routers IP address Set the time to wait to e.g. 35 Save connection Have fun.

Did you read the instructions? You need a separate Redis container running. Replace <IP> in the paperless template with the servers IP running Redis then. You can get Redis from the CA too. Regarding your second question: No, you just need the single Redis and paperless-ng container.

No, I totally get your point and your arguments seems reasonable to me. I run a docker container with a filesystem-based SQLite database that has no build-in database export like you mentioned. That's why I asked this question in the first place. I installed your container today and I really like it so far. Up until now, I did my offsite backups to Google drive via rclone. However, this solution didn't let me sleep well to be honest - especially the Google drive part. So, I ordered a storage box from Hetzner today and did my first backup to it with borgmatic! The last piece missing is stopping the docker container I mentioned above. The plan is to use "HA dockermon" from within the borgmatic container. Would you mind adding curl to the docker container for me? Then, I would be able to stop any container via: curl -v -X POST <ha_dockermon_ip>:8126/container/container_name --header 'content-type: application/octet-stream' --data '{"state": "stop"}' Thanks for bringing borgmatic to my attention

Sorry, for being late here. I'm glad you got it working! I remember how frustrating it was for me to figure it out and find a working solution. Enjoy your IGD inside your VM. Do you mind reporting what CPU and mainboard you are using?

Hello paperless users, unfortunately, paperless hasn't received a lot of updates and bug fixes in the past few month. Even pull requests are not merged for some time now. Though, paperless runs rocks solid and gets the job done! For some time now, there is a well-maintained fork of paperless out there. It's called paperless-ng and I'm happy to announce that paperless-ng is officially available via Unraids community application store (CA store). Let me briefly outline a few improvements over the existing solution: New front end build with Angular. It features full text search with scored and highlighted results, savable filters, a dashboard, and document uploading on the landing page. Mobile support is also almost there. Some layouts don't work yet on small screens. New mail consumer that supports multiple accounts and custom filters and actions. Fully tested! Paperless-ng trains a neural network on your documents and assigns tags and correspondents automatically, if you instruct it to do so. Updated dependencies. More tests of critical backend parts. A proper task processing queue that can consume multiple documents in parallel. Consumption of many documents is now blazing fast on multi core system. Fixed much of the consumer code, so that it does not block the database during consumption, for instance. Paperless-ng now uses OCRmyPDF to perform OCR on documents. It still uses tesseract under the hood, but the PDF parser of Paperless has changed considerably and will behave different for some documents (also see PAPERLESS_OCR_MODE @bigbangus @vakilando). Compatible with the paperless iOS and Android app There is even more. So, don't miss to check out the documentation too! Jonas, the maintainer of paperless-ng, is a highly motivated dev and currently he works towards the first stable release 1.0. Thus, the current version of paperless-ng is flagged as beta in the CA store. If you are interested in paperless-ng and want to support Jonas, please test paperless-ng and give him feedback. You can find a migration guide here too. Be warned, paperless-ng has received a lot of changes and you might encounter bugs. So, whatever you are doing get your backup right first! Also make sure to read the product vision of paperless-ng first, before submitting feature requests. Things like multi-user support are not in the scope of the project for example. That being said: Issues with paperless-ng go here Issues with the Unraid setup go here Happy testing and stay healthy!

Hi sdub, this Borg integration looks promising to me. Thanks for taking the time creating the container and making it available to the community. I will definitely check it out and may consider it as a replacement for my local rsync and remote rclone offsite backup. Will report back! How do you make sure that files are not getting written by your docker containers while the backup is running? The CA backup stops containers to prevent file corruption AFAIK. I cannot see such a mechanism in your solution. Technically, this would be possible with the before_backup and after_backup hooks. Not sure, if any further/similar steps needs to be taken into account for the flash drive. May be worth looking into the CA backup code to review the protection mechanisms.

@al3388 Would love to hear your feedback with Unraid running on the new ODroid H2+. This little device got my attention too.

I just updated it and tested the new release with my external USB drive. Sadly, a refresh on the main tab still wakes up the drive. The option "PASS THRU" is enabled. Found this in my logs when I refresh the tab: May 31 23:27:23 Zeus unassigned.devices: Error: shell_exec(/usr/bin/dd bs=446 count=1 if=/dev/sdb 2>/dev/null | /bin/sum | /bin/awk '{print }') took longer than 5s! Note that the device is formatted as ZFS and reported as "zfs_member" by UD. Can confirm that the command wakes up the drive: root@Zeus:~# /usr/bin/dd bs=446 count=1 if=/dev/sdb 2>/dev/null | /bin/sum | /bin/awk '{print }' 23591 1 Edit: Solved with 2020.05.31a. Big thanks @dlandon!

It does not 🙂

Let me know if I could assist you with testing e.g. I can grap the changes from a Github branch and modify my Unraid installation when you don't want to push a release yet.

I see, then I had the wrong suspect here, sry. Should have read the code in more depth to get a better grasp. Okay, any ideas what could be improved to prevent the wake up then? Do you think disabling of checking the size when PASS THRU is enabled would be a good solution? People might want to see the disk usage even if they pass-through the drive. Can you point me to the code part? Would it be an option to make it optional to show the "Unassigned Devices" section in the main tab? For the time being I removed the "Unassigned Devices" section from the main tab by editing the header of "UnassignedDevices.page". The change will be reverted when I update the plugin though.

I got an external USB hard drive (WD Elements). When I put it to sleep via 'hdparm -y /dev/sdb' and click on the main tab (or refresh it), the 'Unassigned Devices' section refreshes (takes a few seconds) and wakes up the external drive again. I can reproduce this, pinning the wake up of the drive exactly to the time refreshing the main tab. * The drive is formatted with ZFS * "PASS THRU" is enabled * Auto Mount USB Devices set to No Why does the plugin wake up the drive? After troubleshooting the issued rescan command on refresh here (/sbin/udevadm trigger --action=change 2>&1) wakes up the drive. Can I exclude the drive from the plugin without removing the plugin? I don't want the plugin to interact with this respective drive. Is it possible to disable rescan on page refresh? Can I remove the "Unassigned devices" section from the main tab only displaying it in the plugin section? Other solutions?

Playing around with ZFS in Unraid the for a few days now. Thanks for keeping the Plugin up-to-date! I created a single zpool on an external USB disk using the commands mentioned in the first post. The device name changed however from 'sdb' to 'sdg' and the pool was not loaded automatically any more. Thus, I exported the pool and imported the device via its unique UUID i.e (source) : root@server:~# zpool export extdrive root@server:~# zpool list -v no pools available root@server:~# zpool import -d /dev/disk/by-id extdrive root@server:~# zpool list -v NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT extdrive 928G 139G 789G - - 0% 15% 1.00x ONLINE - usb-WD_Elements_25A2_575833314142354837365654-0:0 928G 139G 789G - - 0% 15.0% - ONLINE root@server:~# This makes sure the pool is loaded, even if the device name changed. For me it looks like it is recommended to create the pool using the unique UUIDs rather than device names. What do you guys think? Edit: Seems like it does not work well. After reboot another device name is assigned and despite the fact that the pool is mounted via UUID, commands like 'zpool list -v' stuck Edit2: Looks like that the "stuck" behaviour occurs, when the device labels changes (e.g. re-plug USB drive) while the pool is still loaded. Thus, I ended up doing the following via UD plugin: #!/bin/bash PATH=/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin case $ACTION in 'ADD' ) ;; 'UNMOUNT' ) ;; 'REMOVE' ) ;; 'ERROR_MOUNT' ) DEST=/mnt/extdrive zpool import -d /dev/disk/by-id extdrive zfs mount -a if mountpoint -q $DEST; then rsync -a -v --delete a b 2>&1 [...] sync ud_backup_exit=$? if [ ${ud_backup_exit} -eq 0 ]; then echo "Completed UD backup" else echo "UD backup failed" fi else echo "Backup drive not mounted, exiting" exit 1 fi zfs umount /mnt/extdrive zpool export extdrive if mountpoint -q $MOUNTPOINT; then echo "Error while un-mounting ZFS drive" else echo "Device can be removed" fi ;; 'ERROR_UNMOUNT' ) ;; esac I assigned this script via the UD plugin and configured auto-mount for the device. Now I can plug in my ZFS USB device and remove it once the backup is finished. Using the "ERROR_MOUNT" state is kind of a hack. Would love to have the "ADD" state renamed to "MOUNTED". Then an additional state "ADD" would allow to just indicate the occurrence of new devices. Maybe also custom mount commands for the UD plugin would be cool for kind of scripting with ZFS drives. How do you guys handle such cases?

* Do you access your VM from another device via VNC? * Have you tried a wired connection? * Have you tested other remote accessing software such as nomachine, splashtop or parsec? * Do you have the same bad experience when you plug in a monitor directly to your MB port and use an attached keyboard and mouse? * Does Windows recognize your UHD 630 in the device manager with correct drivers loaded? * Have you tried to download the latest UHD 630 drivers from Intels Website? They usually perform better than the one shipped with Windows.

Thanks @testdasifor your input. I got my hands on a used Xeon-2146G for about 200 Euro on Ebay - waiting for it to arrive. For all that are interested in a comparison: Intel Xeon E-2146G Benchmarks (Idle is around 32W and maximum power consumption hits just under 109W) Intel Core i3-9100F Benchmarks (Idle Power (Performance Mode): 31W, STH 70% Load: 75W, STH 100% Load: 92W) Note that I ruled out the Core i7 option, because it does not support ECC RAM.