so parity been running 4 days 3 more days to go.. i asked already here but no one replied if you can speed up parity...

 

but something happened...  and error logs shwo something but i dont know what it means  now my disk 1 has all the disk space still used  but i lost all the file folders.. whats going on.. is this all the result from bad memory i posted in my last question  that no one replied guess no one knew  how to answer it...  so i been stuck..  so just allowing parity check to go

 and been transfering files to my backup server

 

will the folders and files come back once unraid is rebooted and or after the parity and sync errors are fixed

 

and is is best to have  2 parity drives to prevent these errors from happening

tardis-diagnostics-20230518-0900.zip

Edited May 18, 20233 yr by comet424

May 18 03:54:33 Tardis kernel: BTRFS error (device md1): parent transid verify failed on 4524609208320 wanted 9223372036885778408 found 31002600
May 18 03:54:33 Tardis kernel: BTRFS: error (device md1: state A) in btrfs_run_delayed_refs:2157: errno=-5 IO failure
May 18 03:54:33 Tardis kernel: BTRFS info (device md1: state EA): forced readonly

 

Filesystem on disk1 is corrupt, and it's a clear bitflip:

 

hex(31002600)=                                             1D90FE8

hex(9223372036885778408)  =  8000000001D90FE8

 

Run memtest

i did run mem test on the ram that was in there..  the 32 gig and it failed..  

 

so is it saying  the the 16 gig ram i took out of my desktop has also corrupted?

 

parity is 4 days 15 hours its been running and another 3 4 days estimated....    are my files gone?  is parity holding up... 

i also got 444 sync errors.. it my other post  i as asking help for why parity slowed down  i mentioned i had like 300 sync errors before i rebooted cuz it seemed to stall lik 50 percent..

 

whats a bitflip   like 0 is now a 1...  does that mean the files are gone

 

and then id need to run mem test onthe desktop pc.. since unraid is headless

Edited May 18, 20233 yr by comet424

@JorgeB should i stop the parity then   as i dont wanna damange more and loose my important files

gonna shut down unraid  and move the ram back in the desktop and run the memtest and ill get back to ya..  should be a few hours to do 1 run maybe less

Problem is not parity, synching parity doesn't touch the data on the array disks, you need to fix the RAM problem first, then will need to see if you can recover the data from disk1 and re-format.

ah ok fingers crossed..  so far i on test 9  no errors  ill run it twice  in my desktop pc... and then i going to pull the video card out and shove it in my server and run it twice in there...  the 32gig i pulled out of unraid i posted a diagnostic  in my other post  that was with the 32gig ram in it  and i found  multiple errors on test 6 and 8... so i switched to the 16 gig ram  and it was doing the sync error correct  and running at 5 m/s  so thats why it says its going to take a week.. i dunno why its taking a week  is it not enough ram  that i need 32gig or more  and 16 gig slows things down?

 

ill get get back when i finish doing 2 tests in the desktop and then 2 tests in the unraid

 

oh and things were ok  last night i was using syncthing to copy files from the server to the backup server.. notsure if that was the reason  and that i should have left parity  thing just run on its own

Edited May 18, 20233 yr by comet424

@JorgeB  so i ran memtest86 twice on my desktop  and twice on the unraid box  the 16gb ram that is currently replacing the 32gb ram that has errors on it..

and te 16gb  passed 

i also altered the disk.cfg  and set the array so it wont start.... and i currently have unraid  idling  array offline

 

now what can i do to test..without loosing data

and here is the diagnostic...  with array offline right now and can u see why unraid takes forever  to boot up why it seems to hang around the br3 br4 br5  says they dont exisit  but hangs there so long  maybe  u can see whats wrong there   if it shows up  and or can i mount and copy to the backup server...  or like the special mounting of bfrs or if disk 1 corrupt doesnt the parity drive copensate for that 

tardis-diagnostics-20230518-1629.zip

Edited May 18, 20233 yr by comet424

i tried mounting  in maintance mode  it doesnt do anything  it just shows all the disks green circles like it was working..  you can run parity check i see..  should parity be ran in maintance mode will it fix things  what does it do in maintnace mode  and  or should i mount it normally

 

 

update...

 

so i took chance and started array  parity check didnt come on  but my shares are back  here is diagnostic if u can see any problems  is it safe to re run parity check or what should i do  continue to copy back to the backupserver 

tardis-diagnostics-20230518-1951.zip

Edited May 18, 20233 yr by comet424

May 18 19:48:59 Tardis kernel: BTRFS info (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 23570564, gen 0
May 18 19:50:24 Tardis kernel: BTRFS info (device md6): bdev /dev/md6 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0
May 18 19:50:29 Tardis kernel: BTRFS info (device nvme1n1p1): bdev /dev/nvme1n1p1 errs: wr 0, rd 0, flush 0, corrupt 17, gen 0
May 18 19:50:29 Tardis kernel: BTRFS info (device nvme0n1p1): bdev /dev/nvme0n1p1 errs: wr 0, rd 0, flush 0, corrupt 46, gen 0

 

Data corruption detected on multiple devices, likely from the  previous RAM issues, run a scrub on disk 6 and both pools, for disk1 with such high corruption and previous errors would recommend backing up and re-formatting, assuming RAM issues are fixed.

@JorgeB  so i ran scrub twice  it aborted twice  under 2 hours.. i did scrub on disk 1  and didnt check off repair errors..  is this drive meaning still corrupt and needs formating... can i move files to another drive on my array.  like disk 7   or would the corrupt files be moved.. what can i do now?  or would another parity check fix things?

May 18 20:00:04 Tardis root: Fix Common Problems: Warning: No destination (browser / email / agents set for Warning level notifications
May 18 20:00:23 Tardis root: Fix Common Problems: Other Warning: ca.mover.tuning Support Updated
May 18 22:17:29 Tardis kernel: BTRFS error (device md1): parent transid verify failed on 4524609208320 wanted 9223372036885778408 found 31002600
May 19 01:11:39 Tardis kernel: BTRFS error (device md1): parent transid verify failed on 4524609208320 wanted 9223372036885778408 found 31002600

 

tardis-diagnostics-20230519-0904.zip

sorry i didnt see your previous message ill run scrub on disk 6 now

does it hurt to run scrub on all drives?  and on disk 6 should i check off fix errors

 

 

and does unraid have anything to give you warnings when there is ram issues.. and should i change over to ECC would that solve issues later  my motherboard does support ECC

 

 

oh and i read somewhere cant remember where.. but if i use btrfs on single drive pools  scrub doesnt work as it check if its corrupt or not is that true?  is the scrub only good on the array... and should i convert my single drive pools of btrfs to xfs?   or i need to get a 2nd drive to each pool   dont have the space for them at the moment need to make my own hard drive caddy to do that

 

and what be best  move  the files off disk1 to disk 7 using krusader  or copy it to the backupserver using krusdader...  

and if krusdarder disk1 to disk7  does it bring the corruption...  and  i have a 14tb  drive preclearing should be done 3 runs by tomorrow... would having 2 Parity drive solve memory failure..  or least the disk1 corruption  like if i just pulled the drive out and let it parity rebuild.. or so...  

Edited May 19, 20233 yr by comet424

55 minutes ago, comet424 said:

  it aborted twice  under 2 hours.. i did scrub on disk 1  and didnt check off repair errors..

It won't be able to repair data since array disks are single device filesystem, and like mentioned disk1 should be reformatted.

 

 

ah ok so it can only check  for corruption it just cant fix it.. so then i guess i should change my pools to xfs  from btrfs  or is it ok to keep...  disk6 gonna take like 6 hours to scrub check..   maybe sooner.. i appreciate the help so far..  as i didnt even know  pools were affected too.. so far  no issues  the check.. so thats good..  doing the vm's pool now...so very appreciative 🙂

 

@JorgeB so after 6.5 hours  scrub on disk 6  showed no errors.. so i guess that means memory didnt affect that right?  so currently i copying Disk 1  to Disk6 some files i not able to copy is that the corruption?    and is there a way to know what files got corrupt?  or is that only a Truenas thing they have with snapshots..  and is there a plugin to detect in the system logs when ram fails?

 

and i ordered this ram  to change over to ECC  since my Asus Strik x570 board supports ECC and so does the ryzen cpu...  here is link  you think that should be good and less issues

https://www.amazon.ca/Samsung-Server-Memory-M393A8G40AB2-Registered/dp/B09C7MP442/ref=sr_1_5?crid=P0VSNYTXEWK5&keywords=samsung+ecc+64gb+ram&qid=1684525022&sprefix=samsung+ecc+64gb+r%2Caps%2C810&sr=8-5

and i appreciate you help so far..  cuz i didnt 100% know  what was going on or what was affected

 

oh i had question  if disk1 is corrupt.. how come i couldnt just pull the drive out  put in a new drive and the parity drive would fix things?  or would the files still be corrupted?

12 hours ago, comet424 said:

so currently i copying Disk 1  to Disk6 some files i not able to copy is that the corruption?

Most likely, and if that is the case you'll be able to see the errors on the syslog.

 

7 hours ago, comet424 said:

oh i had question  if disk1 is corrupt.. how come i couldnt just pull the drive out  put in a new drive and the parity drive would fix things?  or would the files still be corrupted?

Parity can't help with filesystem corruption issues, since it will rebuild the same corrupt fs.

 

@JorgeB so i did a move of all files i guess when you do move it copies the files in krusdader  as some files  said it couldnt be moved.. i guess those are the corrupted files.. but i couldnt figure the file names as they too long etc...  so i did do move  what i could do.. then rebooted the server.. then did move.. and did this a few times in hope i could get the best off the server...

 

now how do i format the drive.. and then what do i do  scrub all the drives?   or  parity check etc?  and  is there anyway to check for bad memory..

 

as how do you protect  copying files from  main server to the backup server.. incase  main server ram is bad  or drive is corrupt that it doesnt copy and overwritte the corruption on the backup server..  that it protects it..  whats the best   protection..  and i know this not truenas forum but is best practice to backup to unraid server and a truenas server as they do 2 different file servers..  i just wanna protect myself  from this again  that in future  my backups wont get affected  incase  main server files are affected.. what do you do for maximum protection

41 minutes ago, comet424 said:

now how do i format the drive..

Click on the disk, change fs to a different one, start array, format, chanhe back to the intended fs, format again.

 

42 minutes ago, comet424 said:

as how do you protect  copying files from  main server to the backup server.. incase  main server ram is bad  or drive is corrupt that it doesnt copy and overwritte the corruption on the backup server..

Use btrfs or zfs then send/receive to the other server, if corruption is detected it will abort.

@JorgeB  ok  thats good  it wont copy corruption  as i worry you backup  corruption

 

so i tried that stop array set to xfs  start up array format.. then stop  and set it to btrfs  and go back online and format  

but its doing Data Rebuilding so i guessing its creating the corrupted files again.. how can i scrap  data rebuild and have it just do a clean parity setup

 

and i swapped out the 12 tb drive with a 14tb  drive so its newer  incase its gone faulty... so i want it to not data rebuild just parirty sync

Edited May 22, 20233 yr by comet424

i found the parity  problem assistant under tools  i set it to write corrections to parity and hit check...  its doing something but the gui is locked out for me  but i can ping unraid  i just got no gui  

 

 

update\

 

after 5 min of no access to the gui... the hard drive lights stop running  gui works  but it wont run  a clean party setup  still only option is data rebuild which i dont want...  cuz or what do i do  just let it rebuild  and then format again

 

 

so i just going to run the data rebuild... is the data going to be corrupt?  since all the sync errors...  or whats going to happen?

 

Edited May 22, 20233 yr by comet424

8 hours ago, comet424 said:

so i tried that stop array set to xfs  start up array format.. then stop  and set it to btrfs  and go back online and format  

but its doing Data Rebuilding so i guessing its creating the corrupted files again.. how can i scrap  data rebuild and have it just do a clean parity setup

Not following, post new diags.

well what i did was 

stop array

clicked disk 1

set to xfs

started array

cancled data rebuild

formated disk 1

stopped array 

clicked disk 1 

set to btfrs

started array

cancelled data rebuild

formated disk 1

 

 

i also changed out the disk 1  from 12tb to 14tb

 

what i ment was i didnt want data rebuild  i figured it would just do a clean parity check  like the first time you add a parity drive to your non parity array

it only writes to the parity  not to disk1    no data rebuild its just set parity

 

and i though on tools page the  Parity problem  assistant  where you select yes  to write to the party    was to  reset the parity drive    but when i ran it  it froze up the GUI  of unraid  it was not responsive.. it took  5 min before i could get it back and it didnt do no 14tb check

 

so i turned it to no and ran regular data build....   i copied some stuff over to the disk1

 

and parity that normally took 21 hours says there is 8 more days till parity rebuild  which is nuts  

its running at 14mb/s  estimated speed  

 

here is the diagnostic..  sorry my dislexia gets best of me how i read it and write looks fine to me but not always to others

 

 

tardis-diagnostics-20230523-0818.zip

6 minutes ago, comet424 said:

cancled data rebuild

Why was the disk rebuilding?