BLKMGK
-
Posts
978 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Report Comments posted by BLKMGK
-
-
Just wanted to post and say THANK YOU!!!!!!
Thanks to the efforts of the guys recompiling for the NVIDIA driver I was able to load up RC7 tonight and gve it a spin. The issues described above appears to be FIXED! My Go script no longer has to create a link to a cleartext password in order for me to boot my server - woohoo! I manually entered my password and the server started just fine - big Snoopy Dance! My thanks to the @limetech guys and to @dlandon for solvin this - much appreciated!!
-
On 11/9/2019 at 8:29 PM, Squid said:
Ah, the magical update to unRaid which doesn't actually exist yet
You have to admit updates have been coming pretty quickly, thus far I've seen no huge showstoppers and am pretty excited about the new code! Lots of improvements and some of the speed issues of the past seem solved.
-
I was mentioning it as an aside as it was something odd I had noticed and I'm not sure what's causing it, RC related or otherwise. You need not be so defensive.
-
I'm now on RC5. One thing I've noticed going down is that a second drive I've got mounted with Unassigned Devices seems to hang and be forced unmounted. Each time the array comes up it forces a parity check now. This started a few RC back but I figured it had to do with other things going on. It's formatted XFS and I see errors about the XFS drive not unmounting go by as it goes down and that drive is the only one I've got formatted XFS. Not clear to me what does it, I can say I don't stop VMs or containers before rebooting but this drive isn't used for that anyway <shrug>. Just mentioning it...
-
8 hours ago, limetech said:
I feel your pain. Here's what I think we can do. I'll create a new config setting, call it "BLKMGK mode", when set to Yes then when emhttpd initiates the mount sequence it will write the plaintext passphrase to /root/keyfile and then at end of mount sequence it will delete the file. That way UD can pick up the passphrase - but bear in mind, any other plugin could possibly pick it up too. Sound good?
Would that allow for the entry of a text passphrase? In essence that undoes the feature you've implemented but with the added benefit of deleting the file. I'm okay with that risk! I recognize other plug-ins could snatch it and that's a good catch to think of but I can't be but so paranoid 😮
5 hours ago, dlandon said:The problem with this is that if the disk is unmounted, the keyfile will not be there to re-mount. Or if an encrypted disk is added after the array is mounted. The keyfile will be missing.
You bring up a good point, I guess I'd ask - is this taking an edge case and extending it? Alternately, would it be possible to re-prompt for the passphrase when this occurs? I don't know what control you get when a drive mounts to fire a prompt. It occurs to me that if you could do that you could even use a different password for the mount maybe? This would allow for your "USB storage for backups to be attached "use case and also allow encryption. Not sure it's doable but maybe a solution?
I mentioned my drive occasionally dropping out on my mobo controller, I've since moved it. In this case the issue you've pointed out would trip me up but when it's occurred I've always shut down in order to restart my containers and VMs so in a sense it's already hit me. The question would be - would this impact anyone else? So far I seem to be the only one whining, that said @SpaceInvaderOne was the one I got the idea from so others might be doing it too but not on the RC yet
-
I originally moved my VMs and Containers off of the cache because the size of the storage wasn't enough for moving large files - not without spending a pile. This was of particular importance when running the mover began tanking performance of the entire system. Like many I move video files around and those can get BIG. Run out of space on the Cache drive while moving files around and things come to a halt. I needed as much space for actual cache as possible! I currently move files nightly but can perhaps stop that now that mover doesn't kill things. If I can use more than one drive for cache and keep it encrypted then sure, that solves the issue and hopefully moving things over will be easy. Bear in mind that if you run Plex with a decent library you're talking over a million (tiny) files for metadata. My Containers and VMs take up over 240Gig right now so yeah, I moved them off my cache drive and I'm betting others have too. Putting this on the array is a non-starter, the pause when a sleeping drive spins up and general performance of spinning media makes that clear. I didn't move my files on a whim and I encrypt them to deny a thief, get your stuff stolen a time or two and you get angry like that. Look back and guess who it was that asked for crypto to be added years ago
As for backing stuff up - I'd want that encrypted too! Why bother encrypting otherwise? If I'm pulling sensitive data off for safe keeping I'd like it safe. Currently my personal backup is cloud storage and that's heavily encrypted too. Being able to plug in USB for transfer, connect to other datastores, and things I've never thought of though are awesome features but I've not been using those a much.
Let me clear up how I'm currently forced to run my system. I've got a cleartext file on my USB containing my password. I have a line in my GO script that creates a link to it in the ephemeral filesystem on boot. My system currently boots hands-off and is completely insecure this way. I've left it this way to continue testing the RC. I've seen some weirdness with UD dropping drives that I've attributed to controller issues and my logs filling with crap, but otherwise it's been stable overall. I still badly need Swarm support though!
I've never tried uploading a file to boot my system, doing it on mobile might not even be possible. Using a file like a JPEG or whatever hadn't occurred to me until I saw someone else mention it. Switching to something like that at least makes it less obvious what cleartext file on my system is the password but mobile is still an issue. I wonder if the browser will try to remember my file selection? I have another server I can play with so perhaps I'll use that to work things out instead of risking my primary.
I'll move to the larger cache "pool" when it's available if that keeps things secured but it's sounding like that's not exactly around the corner.
-
13 hours ago, bonienl said:
You would need to store your passphrase in a file somewhere else, e.g. your PC.
When starting the array then choose the keyfile method and select the file you have created above.
This approach allows UD to work properly.
Disclosure: Unraid 6.9 will support multiple pools, and allows pool assignments to specific VMs and/or docker. Effectively taking over some functionality of UD.
So if I'm remote and need to restart my computer from my phone or tablet - not a PC - I may not be able to do so? It would seem we've just lost functionality here. I'm often far away from home when I'm forced to reboot or need to enter a password to decrypt and I do it from my phone or tablet - IOS.
Storing a cleartext password on my PC vs storing it ephemerally on my USB stick (with an option to remove) appears to be less secure. As it stands now I can use a phrase or pull a password from a secured app. I've seen where others have tried to use image files and things for a password file and perhaps that's better were my password not already set.How will multiple pools change this situation? If you're saying that we'll be able to have drives outside of the array for VMs and Containers that's terrific - will we be able to encrypt them?
-
17 hours ago, limetech said:
The "fix" for this issue is: If you are using UD plugin with encrypted volumes, please use the 'file' method of specifying the encryption key, taking care to exclude any line ending characters when you put your passphrase into a file.
How exactly is this a "fix"?! I'm using this method now and it requires me to place a cleartext password on my drive where it's easily accessible - this defeats the encryption completely! Frankly I liked it better where a temporary file was created because at least then it would be ephemeral if power was removed. Am I somehow misunderstanding you here?
If I'm understanding you correctly this completely defeats the purpose of encryption the drives - please reconsider!
-
If you could please include the Overlay networking module for Docker Swarm it would be appreciated! Not yet checked RC4 but it wasn't in RC3, without it Swarm members cannot see other containers off the box and I'd like to include my server in a cluster
To solve the crypto issue I've placed a keyfile in my config directory and added the following line to my Go script prior to the emhttp line. It creates a link file on the dynamic OS to see it on boot. ->
ln -s /boot/config/keyfile /root
-
@dlandon Is RC4 supposed to have anything to help sort this?
-
8 hours ago, dlandon said:
Read several posts up. LT has noted an issue with long passwords. Have you tried rc3?
RC3 hasn't been installed here yet but I'll try to do so today and report back. My password is under 25 chars in length.
Edit: RC3 has the same behavior as RC1 with the added "benefit" that I can no longer go back to a working version of unRAID via the GUI. I've been able to edit the Go file from /boot/config to add my link file back to work around this for now. A parity check was triggered as well.
Manual use of the emcmd against my drive resulted in a hang same as the other user, no output provided at the console from this command.
-
@dlandon any thoughts on the diagnostic I posted the other day? Very curious as to what's going on
-
I have updated UD and will attempt to test tonight. Just so I'm clear, you want me to remove the link fix I've got now, allow it to fail, grab the diagnostic, and pop it up here? I will have to drop back to the old 6.7 RC that's weirdly onboard and then upgrade again to recover so I'm hoping these are the needed steps heh. Server is a touch busy right now but I'll try to do it ASAP so if there's something special you need and you're online drop it here and I'll hopefully spot it before doing this.
P.S. Tom, could you guys please compile in the Overlay module for Docker going forward? I think it's the only thing preventing me from using Swarm with my server
-
51 minutes ago, bonienl said:
If your switch / router supports VLANs, it maybe an option to use VLANs to make network segregation over the same physical interface.
Ah true, I use PFsense and DLink switches that are managed so that mihgt be possible if it doesn't need multiple NIC. Unfortunately it's not something I've ever messed with and I'm pretty ignorant about it. I'll wait and see how others fare and hopefully there's a solution that doesn't require this too. I should read up on this regardless, I've got some IP cameras I ought to better segment anyway. Thanks for the response!
-
Seeing this too, it's amazing how fast my log is filling up! Looks like I may need an hourly cron job for this. Noting that as activity on the interface increases so does the speed with which the messages occur. Downloading a file via a container has got my log rolling so fast I can hardly follow it. Cron job to run hourly has been created!
Edit: sadly this new board has only one NIC so I'd like to be able to share it.
-
Thank you @Squid that looks to be exactly what's going on. I run both VM and multiple containers. I'll keep an eye on the log size! Happy to not be alone
-
2 minutes ago, limetech said:
That would be a good test to confirm the issue.
Ah ha, I linked to root a keyfile stored on my USB (shiver) in the Go file and it's booted! I'm getting some ugliness in the logs though as it seems something is crashing. From the UI all appears normal for now and I'll let it roll. Appreciate the assist!
Here's a link to a snippet of what I see in my logs currently and it began right after a container fired up (it's running fine).
https://pastebin.com/HtKwsfx4 -
34 minutes ago, itimpi said:
I am guessing that this problem is caused by the following change from the
release notes (which was made to increase security):
This probably means that Unassigned Devices can no longer find the pass phrase and is going to need some rework before in can support encrypted disks again using the API mentioned
Aw crap, I missed that! Well okay, that seems to have solved the mystery and explains the change in behaviour. Working as designed and anticipated it seems - d'oh! I suppose for now I can either save to a file or decrypt the drive, not sure which is yuckier lol
-
3 hours ago, bonienl said:
Do you use a passphrase or (binary) keyfile for encryption?
I use a passphrase that's entered at each boot, I am entering the passphrase in order to start the array. I am able to SSH in and see the contents of my protected drives without issue but SMB sharing is apparently not starting and I cannot see those shares from the network. I cannot see the unassigned drive as it never seems to mount. Under 6.7.2 and 6.7.3 rc4 this works.
Now that it's not 1am pushing 4:30 am some clarity lol. My array consists or multiple drives that are all encrypted. My cache drive is also encrypted, my single drive outside of the array used for VM and container storage is also encrypted. When my system boots it requests a passcode before proceeding to mount drives and the Start button won't light without an entry. In the failure case I type in the password, the button lights, I hit start, and the interface eventually refreshes to show my drives up (no more dropdowns) but my Unassigned Drive outside of the array is failing to automount. As this is where VM and Container files are held this is "bad". When this occurs it appears that the normal sharing also fails and the rest of the array doesn't start.
For those wondering - this encrypted drive outside of the array was created by temporarily making it my cache drive, formatting and encrypting it, and then returning my original encrypted cache drive to use. I then mount this drive like any other at boot with automount using Unassigned Drives. My entire array is encrypted as a result and has been since the feature was introduced (I lobbied for it for years). The drive outside the array was only recently encrypted, my thanks to u/Spaceinvaderone for the how-to on that trick just as I was upgrading cache and Container drives
I could reverse this and decrypt that outside drive but it stores data I'd prefer to remain encrypted and it's worked well through multiple versions prior to this.
Questions? Fire away, I'd love to solve this! This might be Unassigned Drives that needs a tweak, I'll hunt down the support thread for that as well, but something has most certainly changed somewhere in this "release candidate". So just to be clear I knew there was a risk here, am NOT upset, and simply want to help get to the bottom of this as there's a TON of fixes in this I'd love to obtain
Much thanks to the NVIDIA guys for compiling against it too!
-
29 minutes ago, Leoyzen said:
Nice！I compiled driver by my self for 6.8.0-rc1 for now. It is a good news for us.
By chance did you happen to find swarm modules enabled in this build already? I seem to recall you had managed to get that working previously. I have still not been able to test it on my test-bed but hope to soon with a friend's help. Would be lovely to have that functionality out of the box with a factory build!
-
33 minutes ago, limetech said:
Please, you've been around here long enough to know:
- Please post diagnstics.
- Please put this in a separate bug report.
To be fair - I've hardly ever had to report bugs but I will happily do so and provide the diag log!
-
Crossposting from Reddit - upgrade failed for me
Some additional info: I have all disks encrypted including the disk with my containers on it using Unassigned drives. This is X570 hardware, 3700X. 6.7.2 (NVIDIA) ran fine for me other than issues we had all seen. When I boot the RC I see it seem to hang as mounting drives but I can SSH in and see everything except my flash and my unassigned SSD. There are two other SSD I don't automount but I don't think they're causing issues and could pull for troubleshooting if requested. Working my way back to 6.7.2 right now but from the looks of it I can test versions freely without trashing things and am willing to do so. If you need more info holler as I'd love to help, not sure what logs are best or safe to post!
Edit: am on 6.73 RC4 running fine (so far) now. Let me know how I can help!
========
Just a heads up - this is NOT going well for my system so far. System is a TaiChi 3700X build. Docker isn't starting for some reason - NVIDIA build. I use encrypted disks but those seem to have mounted fine. Still troubleshooting and the logs look fine but no shares are showing despite mount showing a list and my being able to go into the shares via SSH. No containers are starting, VMs are down too. Screen says "mounting disks" and there it hangs. Am about to grab the standard version of this release and see if it does any better - bummer for me and I'll update as I learn more. u/sittingmongoose cross your fingers lol. Tempted to try a straight reboot first but I'll do the standard build and then try for NVIDIA. Hope I don't end up doing a parity!
Edit: Doing a straight reboot first from the UI. Looks like it had to force shutdown - grr! coming up it says dirty and won't give the start button as it's waiting on Mounting disks
Looks like I'll drop to the standard RC1 and if that fails back to 6.7.2 NVIDIA. Noticed it's not automounting my XFS encrypted drive for containers. Tried the Mount button and it's hanging. This disk isn't showing as mounted from SSH but the others are. Maybe this is the issue?
Edit2: fun fact, cannot download another unRAID version to my USB while things won't mount. I may have to pull this from the rack. I'll try SFTP and hope I have a handy backup around
SFTP gets me in at least but my share for Flash isn't showing dammit.
Edit3: Normal unRAID upgrade tool had a backup version of 6.7.0 RC6 in it, guess it's been awhile since I used that
Managed to boot it! For science sake I'll try the normal unRAID 6.8 RC1 and see if it works better using that tool. If not I at least have a working go-back and we'll know if this was NVIDIA or not.
Edit4: nope same issue with normal 6.8 RC1 for me. It looks like my use of encrypted disks could be an issue? Possibly because I have an encrypted secondary disk using Unassigned devices - it's not clear to me but it looks like I can get back to 6.7.0 RC6. I will then use normal tool to get to 5.7.2 and then NVIDIA 6.7.2. Bummer, I really wanted this to work
This is what I see when I try to manually mount my Lux encrypted container disk:
Oct 14 01:28:59 Minion unassigned.devices: Adding disk '/dev/mapper/docker_vm'...
Oct 14 01:28:59 Minion unassigned.devices: luksOpen: key file not found - using emcmd to mount.
[6.8.0 RC1] encrypted unassigned drive fails to mount, docker fails to start, SMB fails to start
-
-
-
-
-
in Prereleases
Posted
I believe the answer is yes, I'm on 6.8 RC7 with no issues