October 26, 20214 yr Hi I use Unraid to backup files for a web app. During some peaks, hundreds of files can be queued to be processed. The app downloads the file to a temp storage (unrelated to Unraid), then parses and analyses it, then copy it to the cloud for storage and to unraid for backup. 4 service workers actually work concurrently to process these queued files. These files are between 50MB to 150MB During the peaks, I have two problems - Copying the files from the service worker drive to the share seems to work, but the target file is actually empty (0 byte). That one problem doesn't even happen only during peaks. It happens frequently even when the web app traffic is slow and there are not that many files to process. - It gets worse and the share become unresponsive It happens multiple times a day, and I have many files which are 0 byte. They aren't originally 0, and the files stored in the primary location in the cloud are not empty. Only the unraid copies are empty. One episode happened recently between 3:13 and 3:25 today The thing is for the first empty file, the one named AC0DDDF... , the app workers log don't show an error : the program thinks the file has been correctly copied to its destination, the unraid share. For some of the other files, the worker crashed during the copy : After a few minutes the shares become responsive again. A few remarks : - I tried with and without using a cache pool. There are empty files on the cache pool drives if I use one. - 2 workers (of the 4 total) run on a VM hosted on the same Unraid instances. 2 other workers are on a different servers. The 4 workers have the same issues. - I have been having this issue for 2 years now sunraid1-diagnostics-20211025-2059.zip Edited October 26, 20214 yr by 20_100
October 26, 20214 yr Community Expert 31 minutes ago, 20_100 said: Copying the files from the service worker drive to the share Is this "service worker drive" part of the array or cache?
October 26, 20214 yr Author Hi , it's a RAM drive created by a VM hosted on the same unraid instance Edited October 26, 20214 yr by 20_100
October 26, 20214 yr Author Yes, there is 128gb of ram and a memtest takes 3 days but it pass. The files are never 0 MB on the ramdrive. Also the vm uses the ramdrive for some other heavier processing and it works flawlessly
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.