WishmastR Posted February 25, 2021 Share Posted February 25, 2021 On 2/22/2021 at 4:26 PM, wgstarks said: This seems to be related to scanning large files but from what google is showing me it was fixed in an earlier release. I’m wondering if there is a problem with the current release? I confirm having the same issue Quote Link to comment
wgstarks Posted February 25, 2021 Share Posted February 25, 2021 Is it possible to revert to the previous version? I don’t see prior releases listed on GitHub. Quote Link to comment
wgstarks Posted February 28, 2021 Share Posted February 28, 2021 On 2/21/2021 at 10:49 PM, wgstarks said: I have started having problems with this docker recently. Not sure exactly when they started but I noticed this error repeating in the log- LibClamAV Error: CRITICAL: fmap() failed I get this every time I attempt a scan. brunnhilde-diagnostics-20210221-2247.zip 173.6 kB · 1 download The worst thing about this bug is that I still get notifications that the scan completed successfully and no infections were found even though the scan is failing. Quote Link to comment
Aceriz Posted February 28, 2021 Share Posted February 28, 2021 5 minutes ago, wgstarks said: The worst thing about this bug is that I still get notifications that the scan completed successfully and no infections were found even though the scan is failing. I am also getting this error... any update that you find greatly appreciate sharing Quote Link to comment
TQ Posted March 2, 2021 Author Share Posted March 2, 2021 Sorry guys, life's priorities always win. Just updated the "Latest" tagged image (which the Unraid version is working from) Alpine: 3.13 ClamAV: 0.103.1-r0 I just scanned a decent chunk of files and did not see the behavior we all were seeing. 2021-03-01T03:45:47+0000 ClamAV process starting Updating ClamAV scan DB ClamAV update process started at Mon Mar 1 03:45:47 2021 daily database available for download (remote version: 26094) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-ded1d73233782def9a1cb364ec3fb77b.tmp-daily.cvd' ... Database test passed. daily.cvd updated (version: 26094, sigs: 3955353, f-level: 63, builder: raynman) main database available for download (remote version: 59) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-462eb68c40e1e0c4badfd63fb77fc22d.tmp-main.cvd' ... Database test passed. main.cvd updated (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr) bytecode database available for download (remote version: 332) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-27c26d1f3ba016bc7ba80a8909c019f2.tmp-bytecode.cvd' ... Database test passed. bytecode.cvd updated (version: 332, sigs: 93, f-level: 63, builder: awillia2) WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory Freshclam updated the DB ClamAV 0.103.1/26094/Sun Feb 28 12:14:26 2021 Scanning /scan ----------- SCAN SUMMARY ----------- Known viruses: 8505015 Engine version: 0.103.1 Scanned directories: 2867 Scanned files: 35955 Infected files: 0 Data scanned: 44733.52 MB Data read: 34502.84 MB (ratio 1.30:1) Time: 6768.326 sec (112 m 48 s) Start Date: 2021:03:01 03:46:10 End Date: 2021:03:01 05:38:58 2021-03-01T05:38:58+0000 ClamAV scanning finished Pull the latest and cross your fingers. Quote Link to comment
Aceriz Posted March 2, 2021 Share Posted March 2, 2021 3 hours ago, TQ said: Sorry guys, life's priorities always win. Just updated the "Latest" tagged image (which the Unraid version is working from) Alpine: 3.13 ClamAV: 0.103.1-r0 I just scanned a decent chunk of files and did not see the behavior we all were seeing. 2021-03-01T03:45:47+0000 ClamAV process starting Updating ClamAV scan DB ClamAV update process started at Mon Mar 1 03:45:47 2021 daily database available for download (remote version: 26094) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-ded1d73233782def9a1cb364ec3fb77b.tmp-daily.cvd' ... Database test passed. daily.cvd updated (version: 26094, sigs: 3955353, f-level: 63, builder: raynman) main database available for download (remote version: 59) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-462eb68c40e1e0c4badfd63fb77fc22d.tmp-main.cvd' ... Database test passed. main.cvd updated (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr) bytecode database available for download (remote version: 332) Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-27c26d1f3ba016bc7ba80a8909c019f2.tmp-bytecode.cvd' ... Database test passed. bytecode.cvd updated (version: 332, sigs: 93, f-level: 63, builder: awillia2) WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory Freshclam updated the DB ClamAV 0.103.1/26094/Sun Feb 28 12:14:26 2021 Scanning /scan ----------- SCAN SUMMARY ----------- Known viruses: 8505015 Engine version: 0.103.1 Scanned directories: 2867 Scanned files: 35955 Infected files: 0 Data scanned: 44733.52 MB Data read: 34502.84 MB (ratio 1.30:1) Time: 6768.326 sec (112 m 48 s) Start Date: 2021:03:01 03:46:10 End Date: 2021:03:01 05:38:58 2021-03-01T05:38:58+0000 ClamAV scanning finished Pull the latest and cross your fingers. So I just tried deleting the entire CLAMav docker removing the app data folder, and re installing But continue to get the following error .... 3 hours ago, TQ said: Pull the latest and cross your fing Did you change anything in the Docker edit page when pulling down the image? change the /scan 2021-03-02T04:10:19+0000 ClamAV process starting Updating ClamAV scan DB ClamAV update process started at Tue Mar 2 04:10:19 2021 daily.cvd database is up to date (version: 26095, sigs: 3956535, f-level: 63, builder: raynman) main.cvd database is up to date (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr) bytecode.cvd database is up to date (version: 332, sigs: 93, f-level: 63, builder: awillia2) Freshclam updated the DB ClamAV 0.103.1/26095/Mon Mar 1 12:10:16 2021 Scanning /scan LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed Quote Link to comment
Aceriz Posted March 2, 2021 Share Posted March 2, 2021 So I think I might have figured out where the problem is coming from and a possible solution... again I am not that great at the backend stuff But was able to get a working copy of the CLAMav So again not sure what the difference is... but rather than pulling the CLAMav that is from TQ's community application, I went onto the docker hub search in the apps tap finding TQ your https://hub.docker.com/r/tquinnelly/clamav-alpine/ I pulled this one down setting it up with alll of the same permissions and settings as is found within your community applications but without the post arguement of " -i" not sure what it does?? (your input would be helpful). I then am able to run a scan using this newly downloaded docker. and it outputs active details into the log of what it is scanning.. there in an initial error of "CRITICAL: fmap() failed" then it says out of memory error and directs at the docker image.... this version however continues to scan (outputting the files it is scanning into the log file). So I can see it is progressing I have tested quickly changing Squids user script to direct at this docker (testing with a smaller /scan directory)... and it is able to produce the notifications, start the scan and report the outcomes... So TQ it would be great to find out perhaps is the CRITICAL: fmap() failed being due to scanning of the docker.img ?? also again not sure what the "-i" post arguement is supposted to do... still relatively new to things so trying to look into this. Again this was my attempt at a solution... Quote Link to comment
Aceriz Posted March 2, 2021 Share Posted March 2, 2021 On 2/29/2020 at 6:08 PM, tazire said: No i never did after. Just couldnt get it working right. I didnt give it an awful lot of thought after the initial effort though. In case you never figured this out I was able to using an alternate version of ClamAV the "mk0x/docker-clamav:alpine" found on the docker hub... Then within nextcloud set the AV to the following settings in the image. . (both had to be on the proxynet to work ... Quote Link to comment
WishmastR Posted March 2, 2021 Share Posted March 2, 2021 Thanks but I guess I'm wait for the official package to be fixed Quote Link to comment
Aceriz Posted March 6, 2021 Share Posted March 6, 2021 On 3/2/2021 at 2:05 PM, WishmastR said: Thanks but I guess I'm wait for the official package to be fixed So I am not sure what the post arguement parameter does. But in the official ClamAV on appstore not the one I describe above). I note that the appstore ClamAV has a post arguement of " -i" I have been doing a bunch of testing. When I remove this. . I can see in the log folder that clam appears to be scanning somewhat properly- (can now see the specific files. still get some of the errors but now it appears to continue to scan. It also appears to work with the reporting from squids user script... (had run a few smaller folder tests.. am currently running a full server test of this... Will report back once finished to ensure getting correct output.from script.. If On 3/1/2021 at 8:44 PM, TQ said: Alpine: 3.13 ClamAV: 0.103.1-r0 I just scanned a decent chunk of files and did not see the behavior we all were seeing. Perhaps TQ can you clarify what the post arguement of -i does ? is this important for the function.... what It appears to do from what I am seeing is change what is shown in the logs? ? Quote Link to comment
TQ Posted March 7, 2021 Author Share Posted March 7, 2021 On 3/5/2021 at 10:34 PM, Aceriz said: Perhaps TQ can you clarify what the post arguement of -i does ? is this important for the function.... what It appears to do from what I am seeing is change what is shown in the logs? ? It gets passed to the clamscan binary as an arg. REF: clamscan(1): scan files/directories for viruses - Linux man page (die.net) -i, --infected Only print infected files. Quote Link to comment
TQ Posted March 7, 2021 Author Share Posted March 7, 2021 On 3/1/2021 at 11:03 PM, Aceriz said: So TQ it would be great to find out perhaps is the CRITICAL: fmap() failed being due to scanning of the docker.img ?? also again not sure what the "-i" post arguement is supposted to do... still relatively new to things so trying to look into this. Again this was my attempt at a solution... I don't think so. I stuff my docker.img file elsewhere and I was seeing the issue prior to the latest build. I'll run another scan and see if I indeed still see the behavior. ...working on a clamd version as well. Quote Link to comment
wgstarks Posted March 7, 2021 Share Posted March 7, 2021 I’m trying the current version. Looks like the issue still persists. My current scan has been running for about 8 hours (normally it’s <30 minutes). Logs aren’t showing anything other that the start of the scan though. Quote Link to comment
TQ Posted March 7, 2021 Author Share Posted March 7, 2021 Just now, wgstarks said: I’m trying the current version. Looks like the issue still persists. My current scan has been running for about 8 hours (normally it’s <30 minutes). Logs aren’t showing anything other that the start of the scan though. Have you tried removing the -i post-arg ? Quote Link to comment
wgstarks Posted March 7, 2021 Share Posted March 7, 2021 1 minute ago, TQ said: Have you tried removing the -i post-arg ? I’m not sure how to add that? Quote Link to comment
TQ Posted March 7, 2021 Author Share Posted March 7, 2021 Edit the container settings and remove the below: Quote Link to comment
wgstarks Posted March 7, 2021 Share Posted March 7, 2021 43 minutes ago, TQ said: Edit the container settings and remove the below: My template doesn’t have that setting. Not sure how to remove it? Quote Link to comment
wgstarks Posted March 7, 2021 Share Posted March 7, 2021 1 minute ago, wgstarks said: My template doesn’t have that setting. Not sure how to remove it? Never mind. I just needed to switch to advanced view. Quote Link to comment
wgstarks Posted March 7, 2021 Share Posted March 7, 2021 I removed -i and restarted the scan. Scan is still running after 3 hours with lots of errors- LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E02 - Reunion.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E02 - Reunion.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E03 - Virtù e Fortuna.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E03 - Virtù e Fortuna.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E04 - The Riddle of the Sphinx.WEBDL-1080p.h264 EAC3-DEFLATE.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E04 - The Riddle of the Sphinx.WEBDL-1080p.h264 EAC3-DEFLATE-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E05 - Akane No Mai.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E05 - Akane No Mai.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E06 - Phase Space.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E06 - Phase Space.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 /scan/XBMC/TVTorrents/Westworld/Season 02/S02E07 - Les Écorchés.WEBDL-1080p.h264 EAC3-DEFLATE.mkv: Can't allocate memory ERROR LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E07 - Les Écorchés.WEBDL-1080p.h264 EAC3-DEFLATE-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E08 - Kiksuya.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E08 - Kiksuya.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E09 - Vanishing Point.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E09 - Vanishing Point.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK LibClamAV Warning: fmap: failed to get MD5 LibClamAV Error: CRITICAL: fmap() failed /scan/XBMC/TVTorrents/Westworld/Season 02/S02E10 - The Passenger.WEBDL-1080p Proper.x264 EAC3-NTb.mkv: Can't allocate memory ERROR /scan/XBMC/TVTorrents/Westworld/Season 02/S02E10 - The Passenger.WEBDL-1080p Proper.x264 EAC3-NTb-thumb.jpg: OK /scan/XBMC/TVTorrents/Westworld/season02-banner.jpg: OK /scan/XBMC/TVTorrents/Westworld/season01-landscape.jpg: OK /scan/XBMC/TVTorrents/Westworld/tvshow.nfo: OK /scan/XBMC/TVTorrents/Westworld/season03-poster.jpg: OK Quote Link to comment
pokerchip Posted March 10, 2021 Share Posted March 10, 2021 @wgstarks Are those files big? I get an error message like that (with debug mode) when I try to scan files that are bigger than the default file size limit (25mb). For my error message, which makes sense: https://github.com/Cisco-Talos/clamav-devel/blob/e4e3149368d2feab1363f17e27d0271c932ff97c/libclamav/fmap.c#L524 Did you already set max file & scan size limit for clamscan to 4000M? --max-filesize=#n - files larger than this will be skipped and assumed clean --max-scansize=#n - the maximum amount of data to scan for each container file Quote Link to comment
wgstarks Posted March 10, 2021 Share Posted March 10, 2021 (edited) 7 hours ago, pokerchip said: Are those files big? I get an error message like that (with debug mode) when I try to scan files that are bigger than the default file size limit (25mb). I’m sure a lot of the movies (almost all) are larger than 25 GB. I’m also getting memory allocation and fmap failed errors for tv video files which are all <10 GB. Edited March 10, 2021 by wgstarks Correct units for file size. MB->GB Quote Link to comment
wgstarks Posted March 10, 2021 Share Posted March 10, 2021 6 hours ago, pokerchip said: Did you already set max file & scan size limit for clamscan to 4000M? Where is this configured? Quote Link to comment
pokerchip Posted March 11, 2021 Share Posted March 11, 2021 14 hours ago, wgstarks said: Where is this configured? You would configure this in the post arguments section of advanced settings of ClamAV. Check options here: https://www.clamav.net/documents/scanning#clamscan If you want more details: https://linux.die.net/man/1/clamscan Quote Link to comment
wgstarks Posted March 11, 2021 Share Posted March 11, 2021 50 minutes ago, pokerchip said: You would configure this in the post arguments section of advanced settings of ClamAV. Check options here: https://www.clamav.net/documents/scanning#clamscan If you want more details: https://linux.die.net/man/1/clamscan But this would have ClamAV ignore files larger than 4 GB correct. I don’t want to do that. Quote Link to comment
pokerchip Posted March 11, 2021 Share Posted March 11, 2021 3 hours ago, wgstarks said: But this would have ClamAV ignore files larger than 4 GB correct. I don’t want to do that. No, the default for clamscan is 25mb. So if you want clamscan to scan your files that are smaller than 4000M, then you want to set the config to what I said. Also, clamscan has a hard limit of 4000M. If you want a workaround, you will need to chunk your files before clamscan reads them. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.