After the server is started up, any drive or pool that I use is read only. The smart reports don't really show anything and none of my drives are even remotely close to being full. I tried booting into safe mode, which didn't help. Any ideas on why this is?

tower-diagnostics-20210902-1019.zip

Data corruption was detected on the pool, you should run memtest, then backup and reformat the pool.

 

 

@JorgeB thx. I'm kind of new to this problem. How would I go about running a memtest?

Boot menu of Unraid, there's a memtest option, only works for CSM/legacy boot, not UEFI boot.

1 hour ago, GhostJumper said:

@JorgeB thx. I'm kind of new to this problem. How would I go about running a memtest?

Running a memtest is one of the options on the boot menu that is displayed when booting Unraid.

Memtest doesn't seem to find any problems with my RAM.

 

Do the logs say anything specific about where the error originates from?

11 hours ago, GhostJumper said:

Do the logs say anything specific about where the error originates from?

No, they just show data corruption, bad RAM is the #1 reason for that, but there could be other reasons, or the problem is not being detetced by memtest, it's not always

Then I'll try to backup all my data. Since I am unable to mount any kind of storage device, is there a way for be to backup all the data that I couldn't normally reach from outside the server? I thought about mounting an external SMB share, but it also doesn't auto mount it.

For this you just need to backup the cache pool, though you should already have backups of anything important.

Yeah. My question was more along the lines of how I backup something with access rights if I cannot connect anything to the Server itself.

If you can copy the files somewhere, then presumably the copies would not be read-only. This isn't an access rights problem. Corruption is causing the read-only.

 

Do you know how to work with files on your server directly instead of over the network?

4 minutes ago, GhostJumper said:

if I cannot connect anything to the Server itself.

Why not? You should be able to back up to the array, or for example to an unassigned device mounted with UD plugin.

@trurl I do know how to manage files from within Linux itself.

@JorgeB I already tried that. It does not let me mount any drives via the UD tab.

 

Edit:

UD seems to work after restarting.

Thx. I'll make a backup this way.

Edited September 3, 20214 yr by GhostJumper
UD seems to work after restarting.

Just now, GhostJumper said:

It does not let me mount any drives via the UD tab.

Do you mean you can't mount an external USB drive, for example? Or are you trying to mount the disk with the problem?

Sorry, as stated above: I edited my message. After restarting the server once gain, I was able to mount a disk with UD again.

I'm already copied the '/mnt/cache' folder. The backup seemed to have almost double the size of the original cache folder as per 'du -sh'.

I'm re running the rsync copy with the -l flag this time.

I couldn't get the folder size down this way. The original cache folder is 309GB in size. The copy is 695GB in size. How is that possible?

The command I used was 'rsync -r -l --stats --progress /mnt/cache ./'

 

It also seems that there are file duplicate Files among the cache and the pool. (E.g. a 55GB movie with the exact same hash value)

 

How do I go about the backup being more than double the size and the fact that there are file duplicates? 

Edited September 4, 20214 yr by GhostJumper
Added Images

To avoid the duplicates you needed the  --remove-source-files option and if any of the files are sparse files (e.g. vdisks for VMs) then you want the  --sparse option to stop them being expanded to their full size.    Note that the option to remove files on a successful copy does not remove folders so you may have to manually delete those.

 

Does it matter if I copy ALL the files with the --sparse flag?

I tried creating a 1 to 1 copy for a couple of days now without success.

 

Using the sparse flag makes the copy folder smaller since there are files that are now falsely copied.

And without, the copy is way bigger since there are a quite many sparse files.

 

Is there an 'easy' way to just make a one to one copy of ALL the contents of my /mnt/cache folder?

2 hours ago, GhostJumper said:

Using the sparse flag makes the copy folder smaller since there are files that are now falsely copied.

And without, the copy is way bigger since there are a quite many sparse files.

Sparse means a file has space allocated for possible future use, but not currently used. What do you mean by files falsely copied?

4 hours ago, GhostJumper said:

Using the sparse flag makes the copy folder smaller since there are files that are now falsely copied.

And without, the copy is way bigger since there are a quite many sparse files.

A sparse file is one where it does not store data for empty sectors, but still the system returns the ‘empty’ data for those sectors if an attempt is made to read them, so as such it is not a ‘false’ copy.  As such there is little downside for backup purposes as they will still have all their data and the space saving is worthwhile.    
 

You can always copy a ‘sparse’ file to remove the sparseness so the physical space equals the logical space.   The one caution is that using ‘sparse’ files allows you to over-commit the space on the drive so that the total of all the logical size exceeds the physical size, so if you are writing to those files you have to remember this can gradually remove the ‘sparseness’ so you might run out of space unexpectedly.   

@JonathanM my issue lies in the fact that I need to copy a folder with sparse files that also contains non sparse files.

 

Copying the folder WITHOUT the -S makes the copied folder more than double the size cause the actual sparse files aren't copied the way they were intended to be used.

Copying the folder WITH the -S makes the copied folder smaller than the original. I presume that rsync mistakes some files for sparse files.

 

My issue: making a 1:1 copy of ALL the files in a folder. So that the du command shows the exact same size.

Edited September 8, 20214 yr by GhostJumper
grammar

17 minutes ago, GhostJumper said:

Copying the folder WITH the -S makes the copied folder smaller than the original.

That's not a problem, it's normal that some files could be more optimized by "re-sparsifying", especially vdisks or similar, it won't damage any files by always using the sparse flag.