pwm

Yes, most definitely. Deleting the encrypted files is only a good idea if there is a backup of unencrypted files or if the content is so irrelevant that it doesn't matter if the files are lost. Now and then, the police manages to catch hackers. So even if someone doesn't hand over keys for free, there are always chances that more keys will be found out.

Note that the keys in the linked article is mostly related to master keys, and not the symmetric keys that are used for the actual file encryption. But the symmetric keys created by a number of these ransomware applications have been cracked because they have been generated with bad random data - so lots of encrypted files have been possible to decrypt without ever having access to the master key. With a 128-bit symmetric key generated with a known 32-bit pseudo-random generator, it's possible to crack the key almost instantly. This is similar to the LUKS encryption that unRAID supports - the LUKS header contains a volume-local symmetric encryption key. But the volume crypto key itself has been encrypted and requires a passphrase or key file to be unlocked. So the LUKS security depends not only on the passphrase or key file being strong. It also depends on the volume encryption key having been generated using cryptographically strong random data. If I can predict the random generator, then I don't need to care about the LUKS passphrase or key file to attack the volume encryption. The world is full of programs that uses getpid() and/or time() to seed the standard random generator and then keeps calling rand() until they have enough data for their encryption key. By disassembling the ransomware applications, the security researchers can see how the programs generated their keys and if it's possible to duplicate the data used when the keys were generated.

Yes, I have seen it. I'm mostly curious of how/why these keys have been published. If the hackers have been hacked by other hackers or if security agencies have been involved.

"who" only shows users who have logged in so they have a command-line prompt. In that case you see the account name and the IP number they connected from. But "who" you will not let you see accesses over a disk share. Dynamix Active Streams helps with keeping track of open files. The command "smbstatus" lets you see machines connected to Samba and any open shares.

Exactly. But you, on the other hand, can't expect every single forum user to make individual posts "no". So you have to accept the fact that there isn't a well-known issue with unRAID and Caddy. It still boils down to what I did write earlier "If people aren't running their unRAID through Caddy then people aren't keeping track of issues with that usage case." I'm not forcing anything onto you. You may run with an unencrypted connection if you feel that is fine. Run Caddy if you want - just that you can't expect much help in a unRAID forum if people aren't much using Caddy. But you might get help in a Caddy forum, since people on a Caddy forum are quite likely to use it and/or have knowledge of how to turn on suitable logs to figure out what goes wrong. There is a Caddy Docker for unRAID. Below is the support thread for this Docker. The silence in the thread indicates that people are either not using it or are using it without having any issues. And you haven't mentioned Docker anywere so your problem is most probably not related to @cmer's docker. If you post that password-based solutions are secure, then I have a reason to note that they aren't as secure as certificate-based solutions. This may be "your" support thread, but all support threads are open and other people will read this thread too. So you can't demand that people mustn't recommend VPN. There are too many happy VPN users so if anyone finds this thread using a Google search it's best that they clearly see that the general recommendation is to use VPN. A reverse proxy is common when lots of users should reach a system - and the backend system is then often placed in a separate, firewalled, network. A VPN is much more common when there are a limited number of specific people that should have access but the security requirements are high.

But when you run a ssh or web server that requires certificates, your logs will show one connect attempts from scanners and then they will go away when they get a request for the certificate. It's just not meaningful to attack a certificate-based installation unless it is incorrectly configured or too old to have all the required bug fixes and the algorithmic strengthening. When you run a ssh or web server that requires passwords, you will see a huge number of login attempts where password databases are used. Some scanners will try the 100 most common admin accounts with their default passwords. But some scanners will switch over to use very extensive password databases since bandwidth is so cheap and they have robot machines that performs the attack meaning the login attempts comes from many different IP and aren't consuming the bandwidth of the attacker. This is the reason why lots of commercial systems combines the password requirement with 2FA, where you have to supply a time-limited code. Password login are very scary - especially since the machine you login from might have a keyboard logger. You got the answer "No" to that question. And instead got workaround solutions. That made you angry - the attitude you blamed on others.

Note that companies normally have some form of cost for every transaction. So it's more expensive for a company that make 100 transactions at $1 each than having a single transaction of $100. But in this case, it's the standard business deal of: "Buy three books and pay for two." So you can often buy a book trilogy cheaper than buying the books one at a time.

The pot calling the kettle black.... If people aren't running their unRAID through Caddy then people aren't keeping track of issues with that usage case. So looking into Caddy support is most probably the best route to start hunting for the problem. People are regularly running unRAID with VPN. So you have to accept that people suggests a VPN - since it not only provides good security but also allows access to the shares etc.

A: I can't get my car to run faster than 300 km/h - it only does 220 km/h. B: Get a bigger motor and possibly update the gear ratio for the transmission. A: I don't want suggestions of a bigger motor - I want it to run faster than 300 km/h. Anyway - I think the first step is to look for Caddy support, since it would most probably be log files created by Caddy that might tell what isn't working as expected.

The "good" ransomwere programs creates unique encryption keys for each attacked installation. With public-key encryption, the ransomwere program can then encrypt the encryption key and send out to a master machine. Only the master machine has the secret key to decrypt this message and extract the hostage key. So even if you have a recording of the network communication the key can't be extracted. So someone needs to crack the public-key encryption - or someone needs to hack the master machine and extract the private key. Some older "good" ransomwere programs were buggy and incorrectly encrypted in a way that not even the master could decrypt. I think these buggy ransomwere programs have been removed from the market - it gives a bad reputation (as if any ransomwere could have a good reputation) to run ransomware and not be able to decrypt if the user pays. Too many users making it know that paying isn't meaningful kills the market.

How do you manage this? The SMB protocol only allows a single login between two machines. Are you giving your unRAID machine two names, or are you connecting using both IP number and name? And will your backup program perform a logout when done? Because a spy program enumerating all shares will happily find the Syncovery session too. I always use a client-server backup solution where the backup client in the Windows machine uses an SSL tunnel to talk with the backup server. And only the server itself has any access right to the server machine. When ripping movies etc, I normally have the server mount the Windows share to draw data. And if Windows needs access to a read/write document directory, then the ownership of the files on this share are changed every night to lock down existing files. So if I need to edit a Word document, I need to create a new copy to edit. And next night, this copy will also be locked. So almost a WORM storage.

The main reasons for not moving to unRAID v6 is that the machine runs a 32-bit processor (current unRAID versions only works with 64-bit processors) or if the machine runs with very little RAM. And since 32-bit only systems are so old, they have already passed their economical lifetime and may be close to their physical lifetime. There are lots of cheap 64-bit alternatives. But running a system without the additional supervision of current v6 (or own solutions of similar capability) is quite dangerous. Most data losses for RAID systems (except user errors like file overwrites etc) is when people don't take immediate actions when one drive fails - they do not notice anything until a second drive fails and the parity can't emulate anymore. At which time it's too late. It isn't until a couple of version steps into v6 that unRAID can be seen as a real commercial solution. The progression the last two years have been huge.

I have a machine running 5.0.4 (but with lots of own sw changes introduced for supervision and hardening). It will probably stay with 5.0.4 until I decide to deprecate the 4 TB disks and possibly deprecate the rest of the hardware too. It's a HP N54L with 5 drives, so a bit light on CPU capacity. The obvious choice is to move as fast as possible from v4 and v5 of unRAID to get proper supervision of the system. Without regular SMART checks and mail system, it's hard to spot problems before a larger data loss is imminent. There are lots of other dangers involved - you have an old version of Samba and an old version of sshd. And the http server is also vulnerable. So not having WAN routing to the machine is in no way enough to secure the machine from a virus on any of the other machines. At the very least, you need lots of creative firewall rules introduced to protect an older unRAID machine from inhouse attacks.

But the front fan is replaced with the two fans at the back of the hotswap bays.

Just note that there is a limit to how large errors the CRC can detect. A well selected x bit wide CRC polynomial can catch all odd bit errors and a single burst error up to x-1 bits long. But if there are two bit errors that are further away than the burst capacity, then the CRC may start to accept broken packets. For a busy system with a single CRC error now and then, that isn't a problem, because the probability of bit errors is then low and the probability of having multiple bit errors in the same packet is then even lower. But for a system with constantly ticking CRC, there is a danger that a packet may have more than one bit error at different locations in the packet - and this may result in a broken packet being accepted. So it's generally ok to have a system where the counter ticks up once/month while it's directly dangerous to have a system where the CRC counter ticks hundreds of times/day. Each user must then decide where their comfort zone is, when transfer errors must be fixed. Note that the frequency of CRC errors must be put in relation to the number of transfers - an idle disk that ticks a CRC error or two every day could produce a huge number of errors if a TB-sized transfer is started.

This is often caused by a bug somewhere or a damaged file system or similar. So one thread waits in uninterruptible sleep for a condition to end. But that condition never does end, meaning the thread waits indefinitely. And any other thread that needs the same resources waits for the previous thread and so also ends up waiting forever. Do you have any remote mounts (NFS, CIFS) that might fail because of network issues or the remote machine rebooting? That, or use of USB disks, are the only causes I have had myself for threads ending up permanently in the D state where only a reboot have been able to solve the issue. In those situations, any "ls", "df" etc accessing the stuck share has resulted in one more process being permanently stuck. In high-load situations, a number of threads can show up in the D state but will then normally one-by-one unroll as the load goes down and thread-by-thread gets the required resource lock so they can finish their task and exit their wait. A couple of days ago, someone on this forum had a huge number of threads hanging in the D state while waiting to compute hashes for files - but when the mother process was ended so no more hashing requested was started then the backlog of hanging threads cleared up in maybe 10 minutes.

The probability is low - but as I mentioned you could suffer from data overwrites caused by brownouts or power surges could fry everything. A good UPS do way more than supply power until the computer can safely shutdown. It also takes care of overvoltages and undervoltages and filters power surges. You could get a surge arrestor to protect from surges but only a good UPS will make a good job handling over- or under-voltages.

The important thing if deciding to encrypt the drives is to make sure to have a backup of the encryption key. Neither unRAID nor any other system will be able to read an encrypted volume if the key is lost. At least for the normal people who haven't a relative working at NSA.

Often not - but no guarantee. There are many steps where things can go wrong. For the individual file systems, the file system developers tries to make use of barriers and journaling so the file system can roll back to a previous state if only some parts of the writes did reach the disk when the power was lost. Better hard disks tries to make use of stored rotational energy to finish any pending writes if the power is lost. But no disk manufacturer is interested in posting any scientific paper giving the probabilities that the drive may goof if performing writes when the power is lost. With unRAID you have one more factor - unRAID needs to write changes to multiple disks to keep the parity drive(s) up to date with changes to data disks. It's impossible to force concurrent writes to multiple disks and get all disks to either accept or reject the write in case of a power loss. So you will normally get a number of differences in the parity data after a power loss - how many differences will depend on how much writes the machine was busy doing. Normally, unRAID wants to correct the parity after you turn it on again after an unclean shutdown. But if you happen to then find one or more disks broken so you really need to depend on the parity to recreate a data disk, then these blocks of incorrect parity data can result in a number of random data being written to the recreated data disk - and you will not know where they happen to end up. So you might get broken data files. But you might also get a broken file system in a way that the file system developers didn't expect to happen. So you should definitely consider getting a UPS for your server if you have as many power outages as you seem to indicate. You are playing Russian roulette without knowing how many chambers and how many bullets - one of the power failures just might result in a larger data loss. Another thing is that the actual mains voltage is often very unreliable around a power failure - there may be brownout situations and power spikes. And this can break the computer hardware or make the machine issue incorrect writes to the disks. If you fry the PSU, it could burn every single disk in the machine for a perfect 100% data loss.

The SMART data claims that the drive isn't faulty. The older uncorrectable error need not represent any error with the disk - when the disk writes, it always writes blind. It first aligns to the track in read mode. Then it counts sectors waiting until it's about to spot the correct track. Then makes a blind realign of the write head over the track and performs the write. The drive does not know if the write goes well or not - it isn't until you later try to read the sectors that the drive will find out if they could be read. Drives with a vibration sensor tries to abort writes if vibrations are seen. Drives without vibration sensors will just produce garbage writes if there is too much vibrations. When the drive aligns, each track is two-digit nanometers wide. So 1000 tracks are about the same width as a human hair. And the resolution used when aligning the head is in one-digit nanometers. 10 nm is about the width of 20 silicon atoms. So lots can go wrong when the drive tries to properly align the head and write the data. There are videos showing how the drives in a server rack stops producing data if a person shouts at the machine - the voice vibrations are enough to make the enterprise disks stop their tasks and wait for the vibrations to end. The open question is why the drive did disconnect. But Sense 0x5 means an invalid command. And ASC=0x21, ASCQ=0x0 means block out of range. Sense Key : 0x5 [current]  Jul 26 04:41:33 unRAID kernel: sd 5:0:11:0: [sdk] tag#1 ASC=0x21 ASCQ=0x0 If we assume the drive has been up continuously, then the time stamp in the SMART data was Mon Jul 30 12:00:49 2018 and the power on counter then was 6732. The error in the SMART log happened at 6629, so 103 power-on hours earlier - 4 days and 7 hours. So somewhere around Jul 26, 05:00. That agrees with the with the unRAID log printout of Jul 26 04:41:33. So possibly the command was dropped because of a transfer error. Or a software bug. But the error doesn't represent a broken disk - just that the disk couldn't perform the task because the requested task wasn't valid.

Exactly. Include is when a share should not make use of new disks added to the array. So it requires an explicit action to allow the share to grow to more disks. Exclude is when a share should automatically be allowed to make use of new disks. But since include/exclude is about what should happen when new disks are added, a share have no use for specifying both include and exclude. Best would be if the GUI could could disable one of the fields the other field is non-empty.

I would consider it best to download a zipped backup after having made reconfiguration (and possibly even before, if not trusting the state of the previous backup and the changes might need to be undone). When manually downloading a zip file, it's easy to name the zip file in a way that it's possible to remember what changes was made.

What didn't look too good was: Jul 26 04:41:33 unRAID kernel: sd 5:0:11:0: [sdk] tag#1 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x08 Jul 26 04:41:33 unRAID kernel: sd 5:0:11:0: [sdk] tag#1 Sense Key : 0x5 [current] Jul 26 04:41:33 unRAID kernel: sd 5:0:11:0: [sdk] tag#1 ASC=0x21 ASCQ=0x0 Jul 26 04:41:33 unRAID kernel: sd 5:0:11:0: [sdk] tag#1 CDB: opcode=0x8a 8a 08 00 00 00 00 74 75 19 b0 00 00 00 08 00 00 Jul 26 04:41:33 unRAID kernel: print_req_error: critical target error, dev sdk, sector 1953831344 Jul 26 04:41:33 unRAID kernel: md: disk6 write error, sector=1953831280 and in the SMART data: Error 2 [1] occurred at disk power-on lifetime: 6629 hours (276 days + 5 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 10 -- 51 00 00 00 00 74 75 19 b0 c0 00 Error: IDNF at LBA = 0x747519b0 = 1953831344 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 61 00 08 00 00 00 00 74 75 19 b0 41 00 2d+17:37:21.022 WRITE FPDMA QUEUED e5 00 00 00 00 00 00 00 00 00 00 40 00 2d+17:37:21.022 CHECK POWER MODE ea 00 00 00 00 00 00 00 00 00 00 40 00 2d+17:37:20.942 FLUSH CACHE EXT e5 00 00 00 00 00 00 00 00 00 00 00 00 2d+17:37:20.939 CHECK POWER MODE 40 00 00 00 01 00 00 00 00 00 00 40 00 2d+17:37:11.720 READ VERIFY SECTOR(S) Error 1 [0] occurred at disk power-on lifetime: 6533 hours (272 days + 5 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 01 00 00 02 86 6d 4e 00 e0 00 Error: UNC 256 sectors at LBA = 0x2866d4e00 = 10845244928 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 25 00 00 01 00 00 02 86 6d 4e 00 e0 08 2d+03:05:41.795 READ DMA EXT 25 00 00 04 00 00 02 86 6d 4a 00 e0 08 2d+03:05:41.755 READ DMA EXT 25 00 00 02 00 00 02 86 6d 47 80 e0 08 2d+03:05:41.733 READ DMA EXT 25 00 00 00 40 00 00 9a 84 31 80 e0 08 2d+03:05:41.707 READ DMA EXT 25 00 00 01 00 00 00 9a 84 30 00 e0 08 2d+03:05:41.705 READ DMA EXT So the drive has earlier reported 256 uncorrectable sectors at 6533 hours. Not sure if that was before or after the drive was connected to the unRAID. All the SMART-data tells is the number of power-on hours - so @tillkrueger must count backwards and figure out if the UNC errors happened before/after the drive was bought. If before the drive was bought, then it might be possible to ignore this error - maybe power issue in the original system or maybe someone tried to move the machine while the drive was busy. The more recent error (IDNF) at 6629 hours could have been caused by the drive disconnecting - so the "IDNF" was because the drive did no longer have a connected controller to interact with. In which case it could be a cable problem. Or a controller card issue. Not knowing the exact conditions when the two errors happened makes it harder to guess the reason.

No, you can't start a clear on the drive if it has already been added to the array and completely or partially rebuilt - since it's part of the array, any writes to it will update the parity state based on the writes. So a clear would teach unRAID that the disk is expected to be empty.

It means you are in uncharted territory. I don't see any obvious reasons for the failures and then the perfect SMART test, so no easy way to figure out exactly what to fix. You could try to clear the drive again and see if it works better. Or you could try to switch PSU.