Log File: idmap_hash_initialize: The idmap_hash module is deprecated and should not be used.

I think that you're having the instability because you've assigned a container a specific IP address.  Some users seem to have crashes because of this...  A fix is coming in 6.10

Thanks, but I don't have any containers running with an IP address on this server. I did in the past and it settled down when I stopped everything and just used this unit as a file share. 

 

I have Pi-Hole running on other unRAID server with an IP and it's fine though.

 

I'll keep digging on this and post what I find. 

Changed Status to Open

Changed Status to Closed

Thanks, but I don't have any containers running with an IP address on this server. I did in the past and it settled down when I stopped everything and just used this unit as a file share. 

 

I have Pi-Hole running on other unRAID server with an IP and it's fine though.

 

I'll keep digging on this and post what I find. 

 

EDIT: Captured the error. I believe this is what your talking about. I've disabled docker on this server.

 

Jul 26 09:41:39 BrewMaster kernel: ------------[ cut here ]------------
Jul 26 09:41:39 BrewMaster kernel: WARNING: CPU: 5 PID: 96 at net/netfilter/nf_conntrack_core.c:1120 __nf_conntrack_confirm+0x9b/0x1e6 [nf_conntrack]
Jul 26 09:41:39 BrewMaster kernel: Modules linked in: macvlan xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 br_netfilter xfs nfsd lockd grace sunrpc md_mod zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) icp(PO) zcommon(PO) znvpair(PO) spl(O) ip6table_filter ip6_tables iptable_filter ip_tables x_tables bonding igb i2c_algo_bit x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper mpt3sas rapl intel_cstate intel_uncore raid_class i2c_i801 nvme scsi_transport_sas i2c_smbus input_leds nvme_core ahci i2c_core led_class acpi_power_meter intel_pch_thermal libahci video ie31200_edac backlight thermal acpi_ipmi button acpi_pad fan ipmi_si [last unloaded: i2c_algo_bit]
Jul 26 09:41:39 BrewMaster kernel: CPU: 5 PID: 96 Comm: kworker/5:1 Tainted: P O 5.10.28-Unraid #1
Jul 26 09:41:39 BrewMaster kernel: Hardware name: Supermicro Super Server/X11SSH-LN4F, BIOS 2.5 11/26/2020
Jul 26 09:41:39 BrewMaster kernel: Workqueue: events macvlan_process_broadcast [macvlan]
Jul 26 09:41:39 BrewMaster kernel: RIP: 0010:__nf_conntrack_confirm+0x9b/0x1e6 [nf_conntrack]
Jul 26 09:41:39 BrewMaster kernel: Code: e8 dc f8 ff ff 44 89 fa 89 c6 41 89 c4 48 c1 eb 20 89 df 41 89 de e8 36 f6 ff ff 84 c0 75 bb 48 8b 85 80 00 00 00 a8 08 74 18 <0f> 0b 89 df 44 89 e6 31 db e8 6d f3 ff ff e8 35 f5 ff ff e9 22 01
Jul 26 09:41:39 BrewMaster kernel: RSP: 0018:ffffc900001bcd38 EFLAGS: 00010202
Jul 26 09:41:39 BrewMaster kernel: RAX: 0000000000000188 RBX: 0000000000005bdd RCX: 00000000c250a2bb
Jul 26 09:41:39 BrewMaster kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffa032e174
Jul 26 09:41:39 BrewMaster kernel: RBP: ffff88815f3c52c0 R08: 000000005796f9da R09: ffff888156c4cb60
Jul 26 09:41:39 BrewMaster kernel: R10: 0000000000000000 R11: ffff8881011b7b00 R12: 00000000000031eb
Jul 26 09:41:39 BrewMaster kernel: R13: ffffffff8210b440 R14: 0000000000005bdd R15: 0000000000000000
Jul 26 09:41:39 BrewMaster kernel: FS: 0000000000000000(0000) GS:ffff889037b40000(0000) knlGS:0000000000000000
Jul 26 09:41:39 BrewMaster kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 26 09:41:39 BrewMaster kernel: CR2: 000014ed89d89000 CR3: 000000000200a003 CR4: 00000000003706e0
Jul 26 09:41:39 BrewMaster kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jul 26 09:41:39 BrewMaster kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Jul 26 09:41:39 BrewMaster kernel: Call Trace:
Jul 26 09:41:39 BrewMaster kernel: <IRQ>
Jul 26 09:41:39 BrewMaster kernel: nf_conntrack_confirm+0x2f/0x36 [nf_conntrack]
Jul 26 09:41:39 BrewMaster kernel: nf_hook_slow+0x39/0x8e
Jul 26 09:41:39 BrewMaster kernel: nf_hook.constprop.0+0xb1/0xd8
Jul 26 09:41:39 BrewMaster kernel: ? ip_protocol_deliver_rcu+0xfe/0xfe
Jul 26 09:41:39 BrewMaster kernel: ip_local_deliver+0x49/0x75
Jul 26 09:41:39 BrewMaster kernel: ip_sabotage_in+0x43/0x4d [br_netfilter]
Jul 26 09:41:39 BrewMaster kernel: nf_hook_slow+0x39/0x8e
Jul 26 09:41:39 BrewMaster kernel: nf_hook.constprop.0+0xb1/0xd8
Jul 26 09:41:39 BrewMaster kernel: ? l3mdev_l3_rcv.constprop.0+0x50/0x50
Jul 26 09:41:39 BrewMaster kernel: ip_rcv+0x41/0x61
Jul 26 09:41:39 BrewMaster kernel: __netif_receive_skb_one_core+0x74/0x95
Jul 26 09:41:39 BrewMaster kernel: process_backlog+0xa3/0x13b
Jul 26 09:41:39 BrewMaster kernel: net_rx_action+0xf4/0x29d
Jul 26 09:41:39 BrewMaster kernel: __do_softirq+0xc4/0x1c2
Jul 26 09:41:39 BrewMaster kernel: asm_call_irq_on_stack+0x12/0x20
Jul 26 09:41:39 BrewMaster kernel: </IRQ>
Jul 26 09:41:39 BrewMaster kernel: do_softirq_own_stack+0x2c/0x39
Jul 26 09:41:39 BrewMaster kernel: do_softirq+0x3a/0x44
Jul 26 09:41:39 BrewMaster kernel: netif_rx_ni+0x1c/0x22
Jul 26 09:41:39 BrewMaster kernel: macvlan_broadcast+0x10e/0x13c [macvlan]
Jul 26 09:41:39 BrewMaster kernel: macvlan_process_broadcast+0xf8/0x143 [macvlan]
Jul 26 09:41:39 BrewMaster kernel: process_one_work+0x13c/0x1d5
Jul 26 09:41:39 BrewMaster kernel: worker_thread+0x18b/0x22f
Jul 26 09:41:39 BrewMaster kernel: ? process_scheduled_works+0x27/0x27
Jul 26 09:41:39 BrewMaster kernel: kthread+0xe5/0xea
Jul 26 09:41:39 BrewMaster kernel: ? __kthread_bind_mask+0x57/0x57
Jul 26 09:41:39 BrewMaster kernel: ret_from_fork+0x22/0x30
Jul 26 09:41:39 BrewMaster kernel: ---[ end trace afda9ed42ec3be7f ]---

SOLUTION

 

add the following lines to the beginning of  /boot/config/smb-extra.conf

 

[global]

idmap config * : backend = tdb

idmap config * : range = 1000-4000000000

 

Restart Samba with the command 'samba restart'

Changed Status to Solved

On 7/30/2021 at 1:52 PM, Holmesware said:

SOLUTION

 

add the following lines to the beginning of  /boot/config/smb-extra.conf

 

[global]

idmap config * : backend = tdb

idmap config * : range = 1000-4000000000

 

Restart Samba with the command 'samba restart'

 

Thanks, this resolved it for me. 

@Holmesware Can you give a short explanation of what the solution does? Is it a file size limit via samba that we're increasing with the range setting?

Thanks!

The direct answer to your question is the range is SID range on the local unraid box is used by Samba to map domain users and groups to local file ownership.

 

There is FAR more to this that I've been meaning to add. I have 2 test machines and 10+ production unraid boxes that I manage myself and third parties.

 

There are different ways to map domain users to the local SID's on in Samba.

By default Unraid uses idmap_hash and it specifically says DO NOT USE THIS BACKEND

https://www.samba.org/samba/docs/current/man-html/

I don't know why Unraid did this is. Someone please post so we all can know.

You end up with all those log entries, but things seem to work fine for the most part. Makes it hard to troubleshoot when you have something obvioulsy wrong in the log files.

 

I learn by trial and error so I went though the man pages and tried each type of Backend listed here.

One thing I did learn is you need to set the starting range above the highest SID of local users on the Unraid box.

Look at the file /etc/passwd and look for the number after the second :

eg. scans:x:1092:100:ftpuser 1092 is what you need to start above.

If you don't, file ownership get screwed up. DomainUser1 gets assigned the SID 1092 and command line file level viewing is impossible to understand. A file for DomainUser1 is now shown as belonging to local user scans. The domain knows the SID on that file is for DomainUser1 and the ownership functions for that user as expected, but it is also assigned to the matching local user SID from the view of the Unraid box which could cause an issue.

 

The fix for this was changing the starting range, reboot, reassign all permission. Not fun in some cases.

 

I had one instance where switching to the tbd backend did mess up file ownership after a reboot but only for domain accounts. Any local accounts below the starting range were not affected and none of the ACL's were not affected.

This *MAY* of been caused but the owner of this unraid box adding a user at the same time I was working on it. 

 

Generally this fix works as long as the range is set correctly OR you don't use any local accounts on your unraid box beyond the ones it comes with.

 

If I'm wrong on any of this someone please correct me, but I'm speaking from results of things I've actually done.

 

EDIT: setting the starting range to 1000000-2000000 did work for me in all instances.

Edited December 8, 2021 by Holmesware

On 7/30/2021 at 7:52 PM, Holmesware said:

SOLUTION

 

add the following lines to the beginning of  /boot/config/smb-extra.conf

 

[global]

idmap config * : backend = tdb

idmap config * : range = 1000-4000000000

 

Restart Samba with the command 'samba restart'

 

 

 

As mentioned by @Holmesware This solution conflicts with local user permissions, I kindly ask that you modify your solution. I don't know which ranges are the best, this may vary per user, and I don't know what ranges are used exactly by Unraid but i think the following values are safe

On 12/8/2021 at 8:28 PM, Holmesware said:

If I'm wrong on any of this someone please correct me, but I'm speaking from results of things I've actually done.

 

EDIT: setting the starting range to 1000000-2000000 did work for me in all instances.

So your config would look something like this:

Quote

SOLUTION

 

add the following lines to the beginning of  /boot/config/smb-extra.conf

 

[global]

idmap config * : backend = tdb

idmap config * : range = 1000000-2000000

 

When you create local unraid users, it maps those users starting from 1000, that I know for certain.

You can also check the id for any user (active directory or local) by the id command in the unraid terminal 
Example: "id [username/groupname]"

 

Edit: 

I only now realized you corrected yourself, I mistakenly read your solution as ready and therefore implemented it myself, without reading any after-marks. Should've read completely the first time around

Edited August 19, 2022 by marlon420bud

This is the definitive answer and solution to this issue.

 

If you have an unraid server joined to a samba domain and are getting repeated entries in your syslog:

 

<TIMESTAMP> <SERVERNAME> winbindd[6589]: [<TIMESTAMP>,  0] ../../source3/winbindd/idmap_hash/idmap_hash.c:115(idmap_hash_initialize)
<TIMESTAMP> <SERVERNAME> winbindd[6589]:   idmap_hash_initialize: The idmap_hash module is deprecated and should not be used. Please migrate to a different plugin. This module will be removed in a future version of Samba

 

* CAUTION * Making these changes you will have to re-apply all permissions on files/folder that are Domain related
* CAUTION * All Unraid servers in your Domain will need the same exact settings

 

Modify your /boot/config/smb-extra.conf file like this:

 

    idmap config * : backend = tdb <-- Used to be hash>
    idmap config * : range = 3000-7999 <-- Range enough for local users>
    idmap config <SHORTDOMAINNAME> : backend = rid
    idmap config <SHORTDOMAINNAME> : range = 10000-4000000000

 

The following is a paraphrase of an explanation of how this works. Link below for source.

 

All Domain users, groups and computers have a SID. The last part of the
'SID' is called the 'RID' and these are all unique and are set when the
object is created and normal users etc usually start at 1000 (though
this will be different depending on which DC they are created on).

You cannot rely on the RID to identify what the object is, '1000' could
be a user, '1001' could be a group, but, if that is the case, there
will never be a user with the RID '1001'. To put it another way, RID's
are issued consecutively to the next object, no matter what it is.

Now you know how Windows issues ID's, how does Samba map them to Unix
users and groups ?

 

This can be done by winbind and the 'rid' idmap backend (there are
other backends). If you do use the 'rid' idmap backend, it uses this
formula:

 

   ID = RID + LOW_RANGE_ID

   'ID' is the required Unix ID
   'RID' is the Windows user or group ID
   'LOW_RANGE_ID' is the number set in smb.conf (which is '10000' in the example I supplied).

So, if the RID was '1000', the calculation would become:

 

   ID = 1000 + 10000

 

So the 'ID' is '11000' and always will be, even on other Samba fileservers, provided you use the same basic smb.conf
    
Rowland - https://www.spinics.net/lists/samba/msg175409.html

 

Thank you Rowland.

On 12/8/2021 at 11:28 AM, Holmesware said:

I don't know why Unraid did this is. Someone please post so we all can know.

It's historic... waaay back when AD was first integrated there was no such warning.  'hash' was selected because it uses an algorithmic method of mapping SID's to UID/GID's which didn't require maintaining a database file which would have to be kept on the USB flash device.

 

On 12/8/2021 at 11:28 AM, Holmesware said:

eg. scans:x:1092:100:ftpuser 1092 is what you need to start above.

In Unraid OS users created via the Users webGUI menu item are assigned sequential UID's starting at 1000 and all users are put into the 'users' group (GID 100).  The 'nobody' user (UID/GID 99/100) is used for 'guest' access to Public shares.  There should be no UID's above 1000 other than created named users.

 

On 9/19/2022 at 12:15 PM, Holmesware said:

This is the definitive answer and solution to this issue.

Maybe...

 

On 9/19/2022 at 12:15 PM, Holmesware said:

This can be done by winbind and the 'rid' idmap backend (there are
other backends). If you do use the 'rid' idmap backend, it uses this
formula:

But your example is using 'tdb' backend which requires maintenance of a tdb database file.

 

On 9/19/2022 at 12:15 PM, Holmesware said:

* CAUTION * Making these changes you will have to re-apply all permissions on files/folder that are Domain related
* CAUTION * All Unraid servers in your Domain will need the same exact settings

Those statements really should be BOLDED because changing the backend is going to be painful for some users, or an outright security issue.

 

AD integration is pretty complex.  One thing we can do before coming up with a definitive answer is that we can patch the Samba code to get rid of that syslog entry.  We also want a solution that lets an Unraid server function as an AD Domain Controller (something not possible when we first integrated AD).

WONDERFUL! THANK YOU!

 

This syslog entry has been a real pain when trying to troubleshoot a server for other issues. 

So much more makes sense now. 

 

Patching that samba code to get right of the syslog entry would be really helpful.

edit: oh, kernel recompiling...  

 

Thank you again.

 

Edited September 22, 2022 by Holmesware

This change was put into 6.11.0: