Hello, everyone.

This may be somewhat long and unfocused post, but i dont even know where to start diagnosing or what to post here, so i'll just ramble about the problem that I have.

I actually have two problems and I dont know whether they are related or not.

First and the most major problem is that one folder in one of my shares is showing up as empty, even though it is not.

I have "data" share that uses all of my disks and it has multiple folders inside.

All of those folders are accessible except one.

If I open shares>data>media>movies it is empty:

but if I open any disk>data>media>movies then i can see the files inside.

All of those files are accessible and openable.

Second problem that I have is that there are bunch of damaged/corrupted files and checksum errors on two of my disks (disk3 and disk4):

I asked around in discord server and people suggested doing zpool scrub and clear but all in vain.

The problem still persists.

So i need to confirm if these two drives are dead/dying or not.

They already had some problems couple months ago, but i thought that was due to sata adapters that i had and so i took them out and reformatted and thought it was fixed.

But now that these random errors resurfaced i think these two disks may be dying, but need a way to confirm that.

And if these two disks are dying then maybe i could manage to copy data before they finally die? my overall disk usage is about 50% so theoretically if anyone helps me confirm that my disk3 and disk4 are dead i could maybe manage to copy all the data across disk1,2,5 and just replace the 3 and 4.

But first, I just need to fix the first big issue to access my movies share.

Also of note, i have done memtest for about 6-7 hours with no issues (i am saying this, because some people in discord suggested it may be ram issue)

Two disks having data corruption suggest a hardware issue, most often is RAM, but start by posting the diagnostics to see the hardware.

On 6/27/2025 at 10:47 AM, JorgeB said:

Two disks having data corruption suggest a hardware issue, most often is RAM, but start by posting the diagnostics to see the hardware.

sorry for a late reply.

this is the diagnostics file attached.

also how long do i need to run memtest to make sure ram is not an issue?

server-diagnostics-20250628-2353.zip

I'd suggest a minimum of 24 hours or a 1st FAIL

9 hours ago, MowMdown said:

I'd suggest a minimum of 24 hours or a 1st FAIL

alright, will do 24 hours

any suggestions on how to get the content inside the movies folder back though?

or do i have to do the memtest first ?

Edited June 29, 20251 yr by THEVAN3D

You can try a scrub, but most often when there's metadata corruption like that, you will need to backup what you can and restore the data.

3 hours ago, JorgeB said:

You can try a scrub, but most often when there's metadata corruption like that, you will need to backup what you can and restore the data.

what if i do parity check? maybe it can fix something? (but i only have one parity drive and two of the drives have those zpool errors)

btw i already tried scrub and i thought it fixed the errors, but neither the movies folder has re-appeared nor the errors have disappeared completely (they were gone for a few hours and then i still got the errors)

Edited June 29, 20251 yr by THEVAN3D

17 hours ago, THEVAN3D said:

what if i do parity check?

Parity check never changes anything on the data devices, so it won't help for this.

again. sorry for such a late reply. didnt have much time to do memtest, and since EVERYTHING else was working fine on my server i kept using it normally.

so now i got time for that and ran memtest for about 17-ish hours. no errors. ram is fine. i did two separate 3-hour runs (4 passes) and then another longer with more passes that i waited for for about 10ish hours.

so what is my next step now?

how do i make the files re-appear inside the movies folder?

or what do i even do to fix the issue of those random zpool errors.

Did you try a scrub? If a scrub doesn't fix it, the best bet is to backup what you can and reformat.

On 7/18/2025 at 10:50 AM, JorgeB said:

Did you try a scrub? If a scrub doesn't fix it, the best bet is to backup what you can and reformat.

i tried manually removing, clearing and re-adding disk3 and disk4 to the array rebuilding each one of them separately from parity.

issue still persists.

now i started scrub again and this is the current situation as of right now:

I'm going to reboot system after scrub and post here again.

i'm so confused at this point. how can everything inside share be accessible except one folder.

and how can even that one folder still be accessible through individual disks?

what is the problem with my server exactly? are the drives dying? are the cables faulty?

Disk3 is still corrupt; disk4 appears to be OK.

2 hours ago, JorgeB said:

Disk3 is still corrupt; disk4 appears to be OK.

so if i go into disk 3 and manually move all the files from it to another disk (1,2,4 or 5) will that fix the issue that i have? will i be able to access the movies folder afterwards?

and two more questions:

1) how do i actually do that? from within the webui? or do i use plugins like unbalanced?

2) why are there no other errors anywhere? no notification in webui. no error in hdd smart data? etc.

You can try moving everything you can from disk4 to other disk(s), you can use the mover, the integrated file manager or any other tool.

Data corruption is typically RAM related, or CPU in some cases, mostly for the Intel CPUs most affected by the 13/14 gen issue, though it's not impossible that it would be the disk or controller, but much less likely.

but i ran memtest for a long time with no issues. how can the ram be faulty?

i am going to throw this disk3 away completely after this scrub is done. and i'll monitor disk4 too. in case it also starts acting up imma throw it too.

i have 12th gen i3. is that risky too?

18 minutes ago, JorgeB said:

You can try moving everything you can from disk4 to other disk(s), you can use the mover, the integrated file manager or any other tool.

its just the disk that is corrupted right? not the files. i mean moving data from disk3 to other disks wont damage them too, right?

3 minutes ago, THEVAN3D said:

i have 12th gen i3. is that risky too?

That should be fine.

3 minutes ago, THEVAN3D said:

but i ran memtest for a long time with no issues. how can the ram be faulty?

memtest is only definitive if it finds errors, if you have multiple sticks try using the server with just one, if the same try with a different one, that will basically rule out bad RAM.

3 minutes ago, THEVAN3D said:

its just the disk that is corrupted right? not the files.

It's the data/filesystem that is corrupt, try moving the files. Any corrupt files detected won't be moved, you'll get an i/o error.

does this definitely mean disk4 is healthy and only disk3 is corrupted?

or do i need to run some additional commands to make sure disk4 is definitely healthy?

what do i do now? do i start copying data from disk3 to other disks?

and whats the next step after i copy that data? do i just remove the disk3 from array and keep using the server? idk how to properly REMOVE the drive from array, so far i've only ever added new drives to the server.

10 hours ago, THEVAN3D said:

or do i need to run some additional commands to make sure disk4 is definitely healthy?

If you haven't yet, you can run a scrub to confirm.

16 hours ago, JorgeB said:

If you haven't yet, you can run a scrub to confirm.

i just finished scrubbing disk4 with this result

so i think disk4 is healthy, right?

i'm going to start moving data from disk3 to disk5 now and post an update afterwards.

8 hours ago, THEVAN3D said:

so i think disk4 is healthy, right?

Yep

On 7/30/2025 at 11:33 AM, JorgeB said:

Yep

so what do you think i should do next?

even after complete reformat and rebuild of the disk3 the movies folder is still not accessible.

i cant just delete EVERYTHING worth of 60TB and re-acquire all the data. there must be another way somehow. all the files are accessible in disks.

Post the output from ls -la /mnt/user/movies

On 8/1/2025 at 10:26 AM, JorgeB said:

Post the output from ls -la /mnt/user/movies

Sorry, I meant:

ls -la /mnt/user/data/media