March 19Mar 19 **** Looks like it's actually causing corruption the other way, when copied/read from NAS into another device (even when streamed from NAS)*************Have been pulling my hair out for a week now so I figured it was time to ask for help.Running Unraid Unleashed 7.2.4Specs: i5 4690k, z97 G55 Motherboard, 1080ti, 4 x 4GB 1866MHz RAMStorage: 1Tb Cache NVME SSD, 4 x 24TB Exos HDDsHBA: 9207-8iNIC: Intel X540Everything connected locally via 10GBe (Switch etc.. also 10GB)Essentially the issue is as stated in the title. I initially noticed some quality degradation when playing files via Jellyfin or from the NAS with VLC and thought it was related to jellyfin or network issues. Eventually realized the hashes didn't match between files so something was going wrong in the file transfer. I am using Windows file transfer, robocopy and teracopy for each of the following tests (have done every combination so many times)Here's what i've done so far:windows 11 Desktop -> Unraid NAS Cache -> array = MD5 doesn't matchdisable cache for array and then transfer = Md5 MismatchWindows machine to another drive on the same desktop = MD5 MatchWindows Desktop -> Share on Cache = MismatchAt this point I reran memtest for 28 hours , no issues. I also took out the NIC and changed the ethernet cable and Switched to another port on the switch (just in case). All of the above results were the same.Next i tried transfers within unraid.Array -> cache and cache to array = MismatchCache ->array = mismatchthen i used chatgpt to help diagnose the issue and it made me create some 1gb files in ram and hash those a few times to make sure they matched. They didThen made it create a .bin of that on the array and get the hash of that to see if it still matched, it did.When i compare hashes generated (of the same file) within unraid to the hash that teracopy or powershell creates they are different (same as the one in windows explorer - i know it's some utility i added before, but i don't remember what it is). This is confusing. Additionally when i generate MD5 within unraid of a target file it matches perfectly to the same file that I move within unraid (as long as i check the hash with the unraid terminal).But this is where it gets weirder. If i delete that file and copy from the windows machine to the NAS again. The hash is once again different as to the desktop (and different to the last copy where it was also different) but when i run the hash within unraid terminal, it actually matches the previous runs of the same file before deletion. Yet the file is definitely corrupt and the quality loss is clear.Point being i have no idea what is going on and desperately need help.Thank youEdit: here's something interesting. On one test file i transferred from PC to NAS the hash matches but only if i compare the locally derived hash to the locally derived hash (windows side md5 for the windows copy and terminal MD5 for the unraid copy) If I compare them with a utility like teracopy that's supposed to compare them automatically, it doesn't match. And if i use the windows utility to generate MD5 for both of them it doesn't match. I thought the tool might be broken, but unlikely that powershell, file explorer and tertacopy are all faulty. does calculating md5 for a network attached file not work?EDIT 2: Seems like the real issue is in reading files from the NAS. However it's done results in corruption:taking a hash of the file from windows reads as a hash mismatch because of some error during the read.transferring the file from unraid to windows result in hash mismatch.It also looks like jellyfin reading the files and streaming them results in some kind of corruption Edited March 20Mar 20 by Thepn
March 19Mar 19 Community Expert Assuming both PCs have gigabit NICs as well, do a peer-to-peer connection using them and retest; that would rule out the 10GbE LAN completely.
March 19Mar 19 Author 14 minutes ago, JorgeB said:Assuming both PCs have gigabit NICs as well, do a peer-to-peer connection using them and retest; that would rule out the 10GbE LAN completely.I Am not quite sure how to do that. Would the nas still show up as it does right now? or is it a whole new thing to setup
March 19Mar 19 Community Expert Use an IP from a different subnet to set them up, then connect via that IP, that will always use those NICs.
March 19Mar 19 Author 3 minutes ago, JorgeB said:Use an IP from a different subnet to set them up, then connect via that IP, that will always use those NICs.I apologize for my noobness here but I don't know what that means or how to do that
March 19Mar 19 Community Expert Same as this, just in this case you would use the gigabit NICs in peer-to-peer:
March 19Mar 19 Author 5 hours ago, JorgeB said:Assuming both PCs have gigabit NICs as well, do a peer-to-peer connection using them and retest; that would rule out the 10GbE LAN completely.Same behaviour as before
March 19Mar 19 6 hours ago, Thepn said:Next i tried transfers within unraid.Array -> cache and cache to array = MismatchSo, this not network relate. I have same issue in a new build before, after troubleshooting few days, problem gone suddenly and no more again ( I have RMA the mobo later ).6 hours ago, Thepn said:does calculating md5 for a network attached file not work?If you hash remote file through network got mismatch, then it should be other issue.A quicker way to make test ( no need copy whole file and hash ), is use rsync in therminal, copy file, it will recopy again if problem happen.rsync /aaa/bbb/sourcefile /ccc/ddd/destfile --progress Edited March 19Mar 19 by Vr2Io
March 20Mar 20 Author 1 hour ago, Vr2Io said:So, this not network relate. I have same issue in a new build before, after troubleshooting few days, problem gone suddenly and no more again ( I have RMA the mobo later ).If you hash remote file through network got mismatch, then it should be other issue.A quicker way to make test ( no need copy whole file and hash ), is use rsync in therminal, copy file, it will recopy again if problem happen.rsync /aaa/bbb/sourcefile /ccc/ddd/destfile --progressI'm trying to use rsync but i keep getting ssh_dispatch_run_fatal: Connection to 192.168.1.200 port 22: message authentication code incorrect
March 20Mar 20 25 minutes ago, Thepn said:I'm trying to use rsync but i keep getting ssh_dispatch_run_fatal: Connection to 192.168.1.200 port 22: message authentication code incorrectUse build-in terminal also fine.
March 20Mar 20 Author 5 minutes ago, Vr2Io said:Use build-in terminal also fine.how?Also some updates. It seems the issue exclusively occurs when trying to read from the NAS. If i copy files to the nas the hash matches (as long as i check the hash from within unraid - as if i don't i am reading the file from the nas and the issue occurs)22 minutes ago, Vr2Io said:Use build-in terminal also fine.
March 20Mar 20 2 minutes ago, Thepn said:how?Depends on the type of theme, you will found a icon in top right coner or most left sdieFor example
March 20Mar 20 Author 5 minutes ago, Vr2Io said:Depends on the type of theme, you will found a icon in top right coner or most left sdieFor exampleyes but what should i do in the terminal?
March 20Mar 20 12 minutes ago, Thepn said:Also some updates. It seems the issue exclusively occurs when trying to read from the NAS. If i copy files to the nas the hash matches (as long as i check the hash from within unraid - as if i don't i am reading the file from the nas and the issue occurs)I can't remember the behavior of copy file corrupt in that time, but it seems not always, it look like as below. So you need systematic to perform testhash remote file through network no problemcopy file from harddisk to hardisk ( same controller ) no problemcopy cross between media ( i.e. nvme <-> harddisk ) have problem, even with other nvme
March 20Mar 20 3 minutes ago, Thepn said:yes but what should i do in the terminal?I means this is a quick way to make different test. Edited March 20Mar 20 by Vr2Io
March 20Mar 20 Author 6 minutes ago, Vr2Io said:I can't remember the behavior of copy file corrupt in that time, but it seems not always, it look like as below. So you need systematic to perform testhash remote file through network no problemcopy file from harddisk to hardisk ( same controller ) no problemcopy cross between media ( i.e. nvme <-> harddisk ) have problem, even with other nvmeI'm sorry you lost me. I don't understand what you want me to do
March 20Mar 20 3 minutes ago, Thepn said:I'm sorry you lost me. I don't understand what you want me to doCould you use rsync to copy file from array to cache and found rsync indicate re-copy again ? Then use rsync to copy file hardisk-to-hardisk, and check any different.Array -> cache and cache to array = Mismatch Edited March 20Mar 20 by Vr2Io
March 20Mar 20 Author 41 minutes ago, Vr2Io said:Could you use rsync to copy file from array to cache and found rsync indicate re-copy again ? Then use rsync to copy file hardisk-to-hardisk, and check any different.Array -> cache and cache to array = MismatchOk there's no issues when coppying internally on the nas as of now. the reason i thought so before was because i used the windows md5 utility to check hashes. But when i check internally on unraid they're fine. However when i trey to transfer from unraid to windows that's when they get corrupted and hashes break
March 20Mar 20 OK, then this not same issue as my case.If you have another PC ( Windows ), then I would suggest you verify does file transfer still corrupt. Edited March 20Mar 20 by Vr2Io
March 20Mar 20 Author 2 minutes ago, Vr2Io said:OK, then this not same issue as my case.If you have another PC ( Windows ), then I would suggest you verify does file transfer still corrupt.Yes i checked on another windows laptop and when i transfer from nas to that one the file still corrupts
March 20Mar 20 Community Expert This doesn't make much sense, if the files don't corrupt locally, if the LAN is not the problem, and they still get corrupt being read from 2 different PCs, there must still be something in common causing the issue.
March 20Mar 20 Author 10 hours ago, JorgeB said:This doesn't make much sense, if the files don't corrupt locally, if the LAN is not the problem, and they still get corrupt being read from 2 different PCs, there must still be something in common causing the issue.yes it's quite confusing. Chat gpt is saying something in the motherboard on the NAS maybe or the operating system itself. Unsure if you read the updates but it looks like the issue exclusively exists when copying from the NAS to another system. Locally it's fine, writing to the nas is fine, but reading from it is where the corruption exists (except when copying to a flash drive, so only when reading from the nas on something network related). I have tested multiple different network setups (NICs, cables, ports) and those all result in the same errors.Rsync also fails, recognizing the corruption
March 20Mar 20 Community Expert You need to replace some parts or try a different PC as the server, for example, until you find the issue.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.