Hi guys,

 

First: I have been a total idiot. I wanted to delete files from a folder, so using the terminal I used cd to get to that folder and then typed 'rm -rf /*'. I was hoping to delete all files from that directory, but I immediately saw that the UI was giving errors. After some googling I found out that I deleted files from the root (USB?). Needless to say I was a bit disappointed in myself.

 

As I am licking my wounds, I'm trying to see how I can recover what can be recovered. The server is in another location, so as of now I don't have physical access and I do not know what has been deleted (I did cancel the delete quite rapidly). I found out that if the config folder still exist, I can recover everything, right? I did make a backup once (quite long ago), but I did store that on the array (which is now not accessible). 

 

However, I did download diagnostics just this morning, which also happens to have a config folder. Are these the same config files I need? It does also have the super.dat file. Or is there anything else I should check? And is there something else that might give some hope that I can be up and running quite soon again?

 

Thanks for thinking with me

1 hour ago, pietjebell said:

I used cd to get to that folder and then typed 'rm -rf /*

More as a warning to other folks who see this thread.  After using cd, always do a pwd to see that you went where you thought you want to go!  Before ever doing a rm -rf always do an ls -al on the path that you are going to be using to see the list of files/directories that you will be deleting!  (Unfortunately, the UNIX/Linux command line commands assume that anyone using them is an UNIX/Linux expert...)

 

@pietjebell, if you had done what I suggested with the ls command, you would have seen that you were going to remove the entire Linux file structure.  However, since you did start things rolling (and most of that file system is on a RAM disk), thousands of entries can disappear in a few milliseconds.   The real question, did you get it stopped before it got to the /boot  directory which is your Unraid flash drive.  IF it did not, you can simply reboot the server.  (A reboot of Unraid does a complete install of everything from scratch every time!)  You need to get to the server  and look at the contents of the flash drive.  It should look something like this:

root@Elsie1:~# ls -al /boot
total 253856
drwx------  9 root root     32768 Dec 31  1969 ./
drwxr-xr-x 20 root root       460 Mar 13  2019 ../
drwx------  3 root root     32768 Jan 12  2018 EFI/
-rw-------  1 root root  20738048 Apr  7  2021 bzfirmware
-rw-------  1 root root        65 Apr  7  2021 bzfirmware.sha256
-rw-------  1 root root   4818656 Apr  7  2021 bzimage
-rw-------  1 root root        65 Apr  7  2021 bzimage.sha256
-rw-------  1 root root  12562432 Apr  7  2021 bzmodules
-rw-------  1 root root        65 Apr  7  2021 bzmodules.sha256
-rw-------  1 root root 147946760 Apr  7  2021 bzroot
-rw-------  1 root root  72698620 Apr  7  2021 bzroot-gui
-rw-------  1 root root        65 Apr  7  2021 bzroot-gui.sha256
-rw-------  1 root root        65 Apr  7  2021 bzroot.sha256
-rw-------  1 root root     19276 Apr  7  2021 changes.txt
drwx------ 12 root root     32768 May  2 19:06 config/
-rw-------  1 root root      8996 Dec  2  2015 dynamix.hot.fix-2015.12.02.txz
drwx------  2 root root     32768 Jan 20  2016 extra/
-r--------  1 root root    122308 May  8  2018 ldlinux.c32
-r--------  1 root root     69632 May  8  2018 ldlinux.sys
-rw-------  1 root root      7972 Apr  7  2021 license.txt
drwx------  2 root root     32768 Apr 11 17:32 logs/
-rw-------  1 root root      1760 Apr  7  2021 make_bootable.bat
-rw-------  1 root root      3291 Apr  7  2021 make_bootable_linux
-rw-------  1 root root      2428 Apr  7  2021 make_bootable_mac
-rw-------  1 root root    150024 Apr  7  2021 memtest
drwx------  2 root root     32768 Mar 27  2016 packages/
drwx------  2 root root     32768 Apr 11  2021 previous/
-rw-------  1 root root     14322 Apr 20  2017 readvz
drwx------  2 root root     32768 Oct 15  2019 syslinux/
-rw-------  1 root root       492 Apr  7  2021 syslinux.cfg-
-rw-------  1 root root         4 Dec  5  2018 update.assistant.tmp

 

Edited May 4, 20224 yr by Frank1940
One more thing, if there is anything on that flash drive, make a backup it, RIGHT now!!!

I think I tried rebooting it via IPMI, but the server did not come back online. As I said, it is in a remote location, so other factors could be in play. I'm just not counting on it.

 

So I guess files on the USB have been deleted. I'll go to the server tonight and see what files are still left and make a backup of those as well. But questions still are:

 

Can I use the config files from the diagnostics report? Is it as simple as just replace those in a new install?\

 

And if that not works: I got an old backup somewhere on the array. Can I somehow boot the array with a new install and then 'reinstall' when I got my hands on the backup again?

 

Thanks for the advice as well. I was in another directory with cd (could see that in the terminal). But apparently for all files in the folder to be deleted, you need to use ./*. Still would never do it again like this haha

 

1 hour ago, pietjebell said:

you need to use ./*.

You would also have got away with just * - adding the leading / made it an absolute path rather than one relative to the current folder.

 

I always tend to do a ls command before using a rm command as a check on what the rm command will operate on.

 

5 hours ago, pietjebell said:

Can I use the config files from the diagnostics report?

Yes.  But there is a lot of other stuff in the config folder besides the limited data that is copied for the Diagnostics file.  For example, none of the password files are copied over-- security reasons, Share configuration file info is (probably) anonymized, etc.  This also assumes that you have not make any physical disk changes since you downloaded the diagnostics.  It will basically get you back to running a basic server.  (If you have an up-to-date backup of the flash drive somewhere besides on the server, you would be home free..)  Using those files will save you a lot of time.  But the anonymization will require that you figure out the settings that have been so treated..

 

If you have Notifications setup on this server, you will find the array disk assignments in the report.  Even if you delete them daily from your incoming mail server, they might still be in the 'mail send' folder on the sending server. 

Omg I found a flash backup from juli last year. I have not made any significant changes since then (I think). I did recently remove the cache disk and replaced it for another. I think this would result in a error when it is booting. I can then just reassign the new SSD as cache?

 

What would be the best way to recover this? Manually copy the files or completely flash that zip again to the USB?

 

Thanks a lot for your help, and let this be a lesson to other people who read this BEFORE they do anything stupid 😂

Edited May 5, 20224 yr by pietjebell

Personally, I would be more cautious then that.   The disk assignments are in the  super.dat  file in the config directory.  This file is basically machine readable only. (But the serial numbers for the array drives are there if you want to verify them.)  I would definitely  edit the   disk.cfg    file to change the   startArray    parameter to "no".   Be sure you use a Linux aware editor if you need to use <Enter> key as I am not sure of which end-of-line format this file uses!

 

Here is a link to the manual for basic instructions to your situation:

 

      https://wiki.unraid.net/Manual/Troubleshooting#Lost_boot_drive_and_do_not_know_which_are_the_parity_drives

 

Luckily, the rm command should not have affected the formatting on the disk and the required parity updates should have slowed things down so that your stopping the process may have minimized the file losses on the array.  The principal concern to make sure that a current data disk does not get assigned as a parity disk because a parity check will most surely start when the array starts up!

 

Just take slow and easy.  If something does not look right stop before proceeding further

Alright I thought to give a little update on my mistake. Because I did cancel the delete really quick, only some files from my config folder were deleted. So with the files I had left on the USB, my diagnostics file and the old backup I started puzzling. With that, I managed to get the server back up and running relatively easy. My dockers were still running, but I did lose all the templates. So I have to be very careful with there to re-create all of them with templates, but with portainer I can see the current settings of the running containers. 

 

So yeah, it all works again. Thanks for all the help, I think this thread can be closed. Or placed on a wall of shame 🙂