nekromantik Posted April 22, 2019 Share Posted April 22, 2019 hi all Over the last few days I have started to see movie files that are copied to my unraid share via SMB in Windows become corrupt. By this I mean during playback I get skipping and pixelation. In order to rule out the files I playback the local copy on my PC and it plays fine but then I go to the same file via SMB on unraid and it has skipping and pixelation. Any ideas? Quote Link to comment
Frank1940 Posted April 22, 2019 Share Posted April 22, 2019 It seems to me, a few years ago, that there was a particular SATA card that could do this. If you are using any SATA expansion cards, list the Manufacturer(s) and model number(s). One thing to do is to post up your Diagnostics file ( Tools >>> Diagnostics in a new post) and let the real Gurus have a look at it. Quote Link to comment
nekromantik Posted April 22, 2019 Author Share Posted April 22, 2019 Not using SATA card. Running a Dell T20 server Attached is diagnostics Thanks tower-diagnostics-20190422-2234.zip Quote Link to comment
Frank1940 Posted April 22, 2019 Share Posted April 22, 2019 (edited) You have a lot of errors in the syslog that look like this: Apr 21 05:50:05 Tower avahi-daemon[10039]: Withdrawing address record for 2a00:23c4:4e9c:5700::1f27 on br0. Apr 21 05:50:06 Tower dhcpcd[14448]: br0: DHCPv6 REPLY: Address not bound to this interface. Apr 21 05:50:06 Tower dhcpcd[14448]: br0: soliciting a DHCPv6 lease Apr 21 05:50:07 Tower dhcpcd[14448]: br0: ADV 2a00:23c4:4e9c:5700::2000/128 from fe80::4262:31ff:fe01:528c Apr 21 05:50:07 Tower dhcpcd[14448]: br0: REPLY6 received from fe80::4262:31ff:fe01:528c Apr 21 05:50:07 Tower dhcpcd[14448]: br0: adding address 2a00:23c4:4e9c:5700::2000/128 Apr 21 05:50:07 Tower dhcpcd[14448]: br0: renew in 2250, rebind in 3600, expire in 7200 seconds Apr 21 05:50:07 Tower avahi-daemon[10039]: Registering new address record for 2a00:23c4:4e9c:5700::2000 on br0.*. Apr 21 05:50:09 Tower ntpd[1636]: Listen normally on 518 br0 [2a00:23c4:4e9c:5700::2000]:123 Apr 21 05:50:09 Tower ntpd[1636]: new interface(s) found: waking up resolver Hopefully, someone who is much smarter about this type of error will jump in and give you a hand. You might not have a file corruption issue. You should compare the md5 sums for the same file on both the server and your PC. Here is one article about how to do that: https://www.lifewire.com/validate-md5-checksum-file-4037391 If you want more information about md5sum, Google md5 checksum Edited April 22, 2019 by Frank1940 Quote Link to comment
nekromantik Posted April 23, 2019 Author Share Posted April 23, 2019 I dont think it could be ipv6 causing the isuses as my Kodi box connects to Unraid NFS over IPV4. I just checked the md5s and they do not match between file on server and file on PC. So something is going wrong when copying it via Windows. Quote Link to comment
Frank1940 Posted April 24, 2019 Share Posted April 24, 2019 I would suggest that you run memtst (found on the UnRAID boot menu) for 24 hours. Quote Link to comment
nekromantik Posted April 24, 2019 Author Share Posted April 24, 2019 13 hours ago, Frank1940 said: I would suggest that you run memtst (found on the UnRAID boot menu) for 24 hours. I found files that are are downloaded by containers like sonarr for example are fine. they playback without any corruption, its only files copied over from my PC. could it still be memory? Quote Link to comment
JorgeB Posted April 24, 2019 Share Posted April 24, 2019 1 minute ago, nekromantik said: could it still be memory? Not likely, also I assume the Dell uses ECC RAM, most likely some network issue. Quote Link to comment
nekromantik Posted April 24, 2019 Author Share Posted April 24, 2019 Just now, johnnie.black said: Not likely, also I assume the Dell uses ECC RAM, most likely some network issue. Yes I have ECC RAM loaded. Anyone got any suggestions. Quote Link to comment
JorgeB Posted April 24, 2019 Share Posted April 24, 2019 Test everything the data uses from the desktop to the server, NICs, switch, cables, other desktop. Quote Link to comment
nekromantik Posted April 25, 2019 Author Share Posted April 25, 2019 On 4/24/2019 at 5:40 PM, johnnie.black said: Test everything the data uses from the desktop to the server, NICs, switch, cables, other desktop. ok will test other devices on same switch Quote Link to comment
Jclendineng Posted April 25, 2019 Share Posted April 25, 2019 Sounds like a network issue. Unraid would not corrupt files, the only way to tell is to play the file on your pc, it should play fine. Copy said file to unraid, and try to read. If it fails, take the md5 sum of the working file and compare it to the one you copied. This will tell you if it got corrupted during the network transfer. Quote Link to comment
nekromantik Posted April 25, 2019 Author Share Posted April 25, 2019 1 minute ago, Jclendineng said: Sounds like a network issue. Unraid would not corrupt files, the only way to tell is to play the file on your pc, it should play fine. Copy said file to unraid, and try to read. If it fails, take the md5 sum of the working file and compare it to the one you copied. This will tell you if it got corrupted during the network transfer. yup done that and the md5s are different what does not make sense is why all of a sudden this happened. no changes to network have been made Quote Link to comment
JonathanM Posted April 25, 2019 Share Posted April 25, 2019 5 minutes ago, nekromantik said: what does not make sense is why all of a sudden this happened. Pretty much anything that fails was working fine at some point. 1 Quote Link to comment
Frank1940 Posted April 25, 2019 Share Posted April 25, 2019 I have a suggestion for one testing mode that you might want to try. Copy a file the works on your PC to a second PC. Check both the playback and the MD5 checksum of the file on the second PC. Now copy the file from the second PC to your Unraid server. Again check the playback and the MD5 Checksum. Quote Link to comment
nekromantik Posted April 26, 2019 Author Share Posted April 26, 2019 (edited) 20 hours ago, Frank1940 said: I have a suggestion for one testing mode that you might want to try. Copy a file the works on your PC to a second PC. Check both the playback and the MD5 checksum of the file on the second PC. Now copy the file from the second PC to your Unraid server. Again check the playback and the MD5 Checksum. So I did a test. I copied a file downloaded from PC to Unraid over SMB to another machine on the same switch and same corruption. However files downloaded on unraid entirely without PC are fine. Edited April 26, 2019 by nekromantik Quote Link to comment
Frank1940 Posted April 27, 2019 Share Posted April 27, 2019 20 hours ago, nekromantik said: I copied a file downloaded from PC to Unraid over SMB to another machine on the same switch and same corruption. Please reword this as I can not figure out what you did. What I asked you to do was: (1) On your PC, select a file of the type that plays correctly on the PC and not after it is copied to the server. (2) Get the MD5 checksum of the file on that PC. (3) Copy that file to a second PC. Now get the MD5 checksum of that new copy on this second PC. (4) Copy that same file from the second PC to the Unraid server. (5) Get the MD5 checksum of that file on the Unraid server. All of the MD5 checksums should be the same! If they are not, on which transfers are they different? (By the way, it is possible that could be different before and after any copy.) What we are trying to see is where file corruption is occuring. Knowing where it is happening will suggest the possible sources/cause of the problem. (You could have bad RAM in one of computers involved, a defective switch/router, etc.) Quote Link to comment
nekromantik Posted April 27, 2019 Author Share Posted April 27, 2019 (edited) 36 minutes ago, Frank1940 said: Please reword this as I can not figure out what you did. What I asked you to do was: (1) On your PC, select a file of the type that plays correctly on the PC and not after it is copied to the server. (2) Get the MD5 checksum of the file on that PC. (3) Copy that file to a second PC. Now get the MD5 checksum of that new copy on this second PC. (4) Copy that same file from the second PC to the Unraid server. (5) Get the MD5 checksum of that file on the Unraid server. All of the MD5 checksums should be the same! If they are not, on which transfers are they different? (By the way, it is possible that could be different before and after any copy.) What we are trying to see is where file corruption is occuring. Knowing where it is happening will suggest the possible sources/cause of the problem. (You could have bad RAM in one of computers involved, a defective switch/router, etc.) Ok So I copied a file from PC to second PC ---- MD5 matches with first PC Copied file from second PC to Unraid --- MD5 Does not match with second PC/first PC Edited April 27, 2019 by nekromantik Quote Link to comment
Frank1940 Posted April 27, 2019 Share Posted April 27, 2019 Now copy that same file from the first PC to the Unraid server. Get the MD5 checksum of this copy on the Unraid server. Does that match the MD5 checksum of the file that was copied from the second PC to the Unraid server? Quote Link to comment
nekromantik Posted April 27, 2019 Author Share Posted April 27, 2019 (edited) 52 minutes ago, Frank1940 said: Now copy that same file from the first PC to the Unraid server. Get the MD5 checksum of this copy on the Unraid server. Does that match the MD5 checksum of the file that was copied from the second PC to the Unraid server? Nope MD5 does not match between file copied from second PC to Unraid and file copied from first PC to Unraid. So I then copied file from second PC to Unraid using NFS directly from second PC rather then via my windows first PC and MD5 matches. So issues is copying over SMB to Unraid on my Windows machine is corrupting the files. Any hints? I would rather use NFS but I do not have Win 10 Pro. Edited April 27, 2019 by nekromantik Quote Link to comment
Frank1940 Posted April 27, 2019 Share Posted April 27, 2019 Let's try one more test. Copy the file from the Unraid server to the PC that you uploaded it from and check the MD5 checksum again. My best guess at this point would be that there is a RAM issue on your server hardware--- but ECC RAM should catch this. (I assume that you are using ECC RAM in this server!) As I understand it, the memtst program that is installed with Unraid will not catch ECC RAM issues. But there is a version which will. You can find it here: https://www.memtest86.com/ It will run from a bootable flash drive. Be sure to run it for 24 hours. Does anyone else have any ideas of what could be causing this problem? Quote Link to comment
nekromantik Posted April 27, 2019 Author Share Posted April 27, 2019 4 minutes ago, Frank1940 said: Let's try one more test. Copy the file from the Unraid server to the PC that you uploaded it from and check the MD5 checksum again. My best guess at this point would be that there is a RAM issue on your server hardware--- but ECC RAM should catch this. (I assume that you are using ECC RAM in this server!) As I understand it, the memtst program that is installed with Unraid will not catch ECC RAM issues. But there is a version which will. You can find it here: https://www.memtest86.com/ It will run from a bootable flash drive. Be sure to run it for 24 hours. Does anyone else have any ideas of what could be causing this problem? Surely it cannot be RAM issue if file copied directly from second PC to Unraid matches md5? All the mis matches are coming when my windows PC is involved, as both my Unraid server and my second PC are accessed via SMB so all the testing apart from the last test has been via my windows PC. By that I mean, when I copied file from second PC to unraid the first time round I used my windows PC to do that. While the second time I did not use my windows machine and did it directly from second PC. Quote Link to comment
Frank1940 Posted April 27, 2019 Share Posted April 27, 2019 From what I gather you have two PC's --- The first one has WIN10 as the OS and the second PC is running some other OS. Is this correct? And that second PC does not use SMB. Is this correct? You said you used the first PC to copy the file from the second PC to the Unraid server. Why did you do this? The object was to use a separate, distinct SMB path between the second PC and the Unraid server. You have to initiate the transfer from the second PC. Itis important to realize that your network could be the cause of this problem. Your router and/or switch have either a dedicated SOC or a CPU controlling the flow of data through them. I doubt if SMB is the root cause of your problem. If it were, this board would have hundreds (if not thousands) of folks screaming... Quote Link to comment
nekromantik Posted April 27, 2019 Author Share Posted April 27, 2019 51 minutes ago, Frank1940 said: From what I gather you have two PC's --- The first one has WIN10 as the OS and the second PC is running some other OS. Is this correct? And that second PC does not use SMB. Is this correct? You said you used the first PC to copy the file from the second PC to the Unraid server. Why did you do this? The object was to use a separate, distinct SMB path between the second PC and the Unraid server. You have to initiate the transfer from the second PC. Itis important to realize that your network could be the cause of this problem. Your router and/or switch have either a dedicated SOC or a CPU controlling the flow of data through them. I doubt if SMB is the root cause of your problem. If it were, this board would have hundreds (if not thousands) of folks screaming... Correct first PC is Windows 10 and I mount shares for Unraid. Second PC is a Intel NUC with Ubuntu on it. That runs my media player and connects to Unraid via NFS. So it looks like my Windows PC is the issue then as when initiating transfer from Ubuntu to Unraid MD5 remains same but when using my Windows PC to copy to Unraid MD5 does not match. My network setup is Windows is wireless and Ubuntu and Unraid are wired in. My router is intel mini pc with pfsense and wifi access point connected. Quote Link to comment
Vr2Io Posted April 27, 2019 Share Posted April 27, 2019 Seems network issue Wire <-> Pfsense <-> Wireless , does those are bridge or route ? If your AP have 2 LAN port, could you try after change as below wire <-> (LAN) AP (LAN) <-> Unraid Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.