[Support] ClamAV


Recommended Posts

On 2/21/2021 at 10:49 PM, wgstarks said:

I have started having problems with this docker recently. Not sure exactly when they started but I noticed this error repeating in the log-


LibClamAV Error: CRITICAL: fmap() failed

I get this every time I attempt a scan.

 

 

brunnhilde-diagnostics-20210221-2247.zip 173.6 kB · 1 download

The worst thing about this bug is that I still get notifications that the scan completed successfully and no infections were found even though the scan is failing.

Link to comment

Sorry guys, life's priorities always win.

 

Just updated the "Latest" tagged image (which the Unraid version is working from)

 

Alpine: 3.13

ClamAV: 0.103.1-r0

 

I just scanned a decent chunk of files and did not see the behavior we all were seeing.

 

2021-03-01T03:45:47+0000 ClamAV process starting

Updating ClamAV scan DB
ClamAV update process started at Mon Mar  1 03:45:47 2021
daily database available for download (remote version: 26094)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-ded1d73233782def9a1cb364ec3fb77b.tmp-daily.cvd' ...
Database test passed.
daily.cvd updated (version: 26094, sigs: 3955353, f-level: 63, builder: raynman)
main database available for download (remote version: 59)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-462eb68c40e1e0c4badfd63fb77fc22d.tmp-main.cvd' ...
Database test passed.
main.cvd updated (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
bytecode database available for download (remote version: 332)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-27c26d1f3ba016bc7ba80a8909c019f2.tmp-bytecode.cvd' ...
Database test passed.
bytecode.cvd updated (version: 332, sigs: 93, f-level: 63, builder: awillia2)
WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory


Freshclam updated the DB


ClamAV 0.103.1/26094/Sun Feb 28 12:14:26 2021

Scanning /scan


----------- SCAN SUMMARY -----------
Known viruses: 8505015
Engine version: 0.103.1
Scanned directories: 2867
Scanned files: 35955
Infected files: 0
Data scanned: 44733.52 MB
Data read: 34502.84 MB (ratio 1.30:1)
Time: 6768.326 sec (112 m 48 s)
Start Date: 2021:03:01 03:46:10
End Date:   2021:03:01 05:38:58

2021-03-01T05:38:58+0000 ClamAV scanning finished

 

Pull the latest and cross your fingers.

Link to comment
3 hours ago, TQ said:

Sorry guys, life's priorities always win.

 

Just updated the "Latest" tagged image (which the Unraid version is working from)

 

Alpine: 3.13

ClamAV: 0.103.1-r0

 

I just scanned a decent chunk of files and did not see the behavior we all were seeing.

 


2021-03-01T03:45:47+0000 ClamAV process starting

Updating ClamAV scan DB
ClamAV update process started at Mon Mar  1 03:45:47 2021
daily database available for download (remote version: 26094)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-ded1d73233782def9a1cb364ec3fb77b.tmp-daily.cvd' ...
Database test passed.
daily.cvd updated (version: 26094, sigs: 3955353, f-level: 63, builder: raynman)
main database available for download (remote version: 59)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-462eb68c40e1e0c4badfd63fb77fc22d.tmp-main.cvd' ...
Database test passed.
main.cvd updated (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
bytecode database available for download (remote version: 332)
Testing database: '/var/lib/clamav/tmp.e75353b543/clamav-27c26d1f3ba016bc7ba80a8909c019f2.tmp-bytecode.cvd' ...
Database test passed.
bytecode.cvd updated (version: 332, sigs: 93, f-level: 63, builder: awillia2)
WARNING: Clamd was NOT notified: Can't connect to clamd through /run/clamav/clamd.sock: No such file or directory


Freshclam updated the DB


ClamAV 0.103.1/26094/Sun Feb 28 12:14:26 2021

Scanning /scan


----------- SCAN SUMMARY -----------
Known viruses: 8505015
Engine version: 0.103.1
Scanned directories: 2867
Scanned files: 35955
Infected files: 0
Data scanned: 44733.52 MB
Data read: 34502.84 MB (ratio 1.30:1)
Time: 6768.326 sec (112 m 48 s)
Start Date: 2021:03:01 03:46:10
End Date:   2021:03:01 05:38:58

2021-03-01T05:38:58+0000 ClamAV scanning finished

 

Pull the latest and cross your fingers.

 

 

So I just tried  deleting the entire CLAMav docker removing the app data folder, and re installing 

But continue to get the following error ....   

3 hours ago, TQ said:

Pull the latest and cross your fing

Did you change anything in the Docker edit page when pulling down the image?  change the /scan 

 

 

 

 

 

2021-03-02T04:10:19+0000 ClamAV process starting

Updating ClamAV scan DB
ClamAV update process started at Tue Mar 2 04:10:19 2021
daily.cvd database is up to date (version: 26095, sigs: 3956535, f-level: 63, builder: raynman)
main.cvd database is up to date (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
bytecode.cvd database is up to date (version: 332, sigs: 93, f-level: 63, builder: awillia2)


Freshclam updated the DB


ClamAV 0.103.1/26095/Mon Mar 1 12:10:16 2021

Scanning /scan

LibClamAV Warning: fmap: failed to get MD5
LibClamAV Error: CRITICAL: fmap() failed

Link to comment

So I think I might have figured out where the problem is coming from and a possible solution...  again I am not that great at the backend stuff But was able to get a working copy of the CLAMav :)

 

So again not sure what the difference is... but rather than pulling the CLAMav that is from TQ's   community application,  I went onto the docker hub search in the apps tap  finding TQ  your https://hub.docker.com/r/tquinnelly/clamav-alpine/     

 

I pulled this one down setting it up with alll of the same permissions and settings as is found within your community applications but without the post arguement of "  -i"   not sure what it does?? (your input would be helpful).

 

I then am able to run a scan using this newly downloaded docker.   and it outputs active  details into the log of what it is scanning.. 

there in an initial error of "CRITICAL: fmap() failed"    then it says out of memory error and directs at the docker image.... 

 

this version however continues to scan (outputting the files it is scanning into the log file).  So I can see it is progressing

 

I have tested quickly changing Squids user script to direct at this docker (testing with a smaller /scan directory)... and it is able to produce the notifications, start the scan and report the outcomes... 

 

 

So TQ it would be great to find out perhaps is the CRITICAL: fmap() failed  being due to scanning of the docker.img  ??    also again not sure what the "-i"   post arguement is supposted to do... still relatively new to things so trying to look into this.  

 

Again this was my attempt at a solution... 

Link to comment
On 2/29/2020 at 6:08 PM, tazire said:

No i never did after. Just couldnt get it working right. I didnt give it an awful lot of thought after the initial effort though. 

In case you never figured this out I was able to using an alternate version of ClamAV  the "mk0x/docker-clamav:alpine"     found on the docker hub...  Then within nextcloud  set the AV to the following  settings in the image. .   (both had to be on the proxynet  to work ...  

image.png

Link to comment
On 3/2/2021 at 2:05 PM, WishmastR said:

Thanks but I guess I'm wait for the official package to be fixed

So I am not sure what the post arguement parameter does.   But in the official ClamAV on appstore not the one I describe above).   I note that the appstore ClamAV  has a post arguement of  " -i"  

 

I have been doing a bunch of testing.  When I remove this. . I can see in the log folder that clam appears to be scanning somewhat properly- (can now see the specific files.  still get some of the errors but now it appears to continue to scan.

 

It also appears to work with the reporting from squids user script...  (had run a few smaller folder tests.. am currently running a full server test of this... Will report back once finished to ensure getting correct output.from script..

 

 

If 

On 3/1/2021 at 8:44 PM, TQ said:

Alpine: 3.13

ClamAV: 0.103.1-r0

 

I just scanned a decent chunk of files and did not see the behavior we all were seeing.

Perhaps TQ  can you clarify what the post arguement of -i   does ?  is this important for the function.... what It appears to do from what I am seeing is change what is shown in the logs? ? 

Link to comment
On 3/5/2021 at 10:34 PM, Aceriz said:

Perhaps TQ  can you clarify what the post arguement of -i   does ?  is this important for the function.... what It appears to do from what I am seeing is change what is shown in the logs? ? 

 

It gets passed to the clamscan binary as an arg.


REF: clamscan(1): scan files/directories for viruses - Linux man page (die.net)

-i, --infected
Only print infected files.

 

Link to comment
On 3/1/2021 at 11:03 PM, Aceriz said:

So TQ it would be great to find out perhaps is the CRITICAL: fmap() failed  being due to scanning of the docker.img  ??    also again not sure what the "-i"   post arguement is supposted to do... still relatively new to things so trying to look into this.  

 

Again this was my attempt at a solution... 

 

I don't think so. I stuff my docker.img file elsewhere and I was seeing the issue prior to the latest build.

 

I'll run another scan and see if I indeed still see the behavior.

 

...working on a clamd version as well.

 

Link to comment
Just now, wgstarks said:

I’m trying the current version. Looks like the issue still persists. My current scan has been running for about 8 hours (normally it’s <30 minutes). Logs aren’t showing anything other that the start of the scan though.

Have you tried removing the -i post-arg ?

Link to comment

I removed -i and restarted the scan. Scan is still running after 3 hours with lots of errors-

LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E02 - Reunion.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E02 - Reunion.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E03 - Virtù e Fortuna.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E03 - Virtù e Fortuna.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E04 - The Riddle of the Sphinx.WEBDL-1080p.h264 EAC3-DEFLATE.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E04 - The Riddle of the Sphinx.WEBDL-1080p.h264 EAC3-DEFLATE-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E05 - Akane No Mai.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E05 - Akane No Mai.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E06 - Phase Space.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E06 - Phase Space.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E07 - Les Écorchés.WEBDL-1080p.h264 EAC3-DEFLATE.mkv: Can't allocate memory ERROR

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E07 - Les Écorchés.WEBDL-1080p.h264 EAC3-DEFLATE-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E08 - Kiksuya.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E08 - Kiksuya.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E09 - Vanishing Point.WEBDL-1080p.h264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E09 - Vanishing Point.WEBDL-1080p.h264 EAC3-NTb-thumb.jpg: OK
LibClamAV Warning: fmap: failed to get MD5

LibClamAV Error: CRITICAL: fmap() failed

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E10 - The Passenger.WEBDL-1080p Proper.x264 EAC3-NTb.mkv: Can't allocate memory ERROR

/scan/XBMC/TVTorrents/Westworld/Season 02/S02E10 - The Passenger.WEBDL-1080p Proper.x264 EAC3-NTb-thumb.jpg: OK
/scan/XBMC/TVTorrents/Westworld/season02-banner.jpg: OK
/scan/XBMC/TVTorrents/Westworld/season01-landscape.jpg: OK
/scan/XBMC/TVTorrents/Westworld/tvshow.nfo: OK
/scan/XBMC/TVTorrents/Westworld/season03-poster.jpg: OK

 

 

Link to comment

@wgstarks

 

Are those files big? I get an error message like that (with debug mode) when I try to scan files that are bigger than the default file size limit (25mb).

 

For my error message, which makes sense:

https://github.com/Cisco-Talos/clamav-devel/blob/e4e3149368d2feab1363f17e27d0271c932ff97c/libclamav/fmap.c#L524

 

Did you already set max file & scan size limit for clamscan to 4000M?

--max-filesize=#n - files larger than this will be skipped and assumed clean
--max-scansize=#n - the maximum amount of data to scan for each container file

 

Link to comment
7 hours ago, pokerchip said:

Are those files big? I get an error message like that (with debug mode) when I try to scan files that are bigger than the default file size limit (25mb).

I’m sure a lot of the movies (almost all) are larger than 25 GB. I’m also getting memory allocation and fmap failed errors for tv video files which are all <10 GB.

Edited by wgstarks
Correct units for file size. MB->GB
Link to comment
3 hours ago, wgstarks said:

But this would have ClamAV ignore files larger than 4 GB correct. I don’t want to do that. 

No, the default for clamscan is 25mb. So if you want clamscan to scan your files that are smaller than 4000M, then you want to set the config to what I said. 
 

Also, clamscan has a hard limit of 4000M. If you want a workaround, you will need to chunk your files before clamscan reads them. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.