Data Integrity-Reliability and HDD speed issues

[DR4GON]

TLDR: Performance is worse than expected with frequent errors while copying data to the share from a Windows PC, and forget about accessing the server from 2 computers at the same time. Send help.

 

The Long:

I'm a recent convert from FreeNAS. I am confident following detailed instructions, and even manage to set up some unconventional jails on my own after playing in the terminal for extended periods. See: Transmission OpenVPN via PIA working where others couldn't get it working. I would frequently copy over 1-3TB of data in the form of backups/videos/documents at 110MB/s to one pc, or average +50MB/s from 2 PCs on any night to a FreeNAS Pool (RAIDZ2 x2 -16 drives across 2 pools of 8 drives with 2 redundancy each.) The system would also constantly manage Plex/Sonarr/Radarr/Transmission/OpenVPN/MineOS/Ubuntu18.04 and more when needed.

 

I was under the impression that I'd be able to get the same or better performance with a transition to Unraid. All because I was looking to implement a RTX2070 to my rig for VM pass through and Plex. FreeNAS does not allow GPU passthrough, so I'm in a position where I need Unraid to work, but if something more robust exists, I'm still in the trail stage.

 

Drive layout now is 2 Parity drives and 14 drives, in a single share. Drive1&2 is for "appdata/domains/isos", while Drives3-14 are for "NAS".

 

The issue is that copying to the server I never get sustained writes over 30MB/s, but I get reads over 100MB/s. When I have prolonged write speeds under 10MB/s I get errors that result in windows spitting an error that there is an issue with the network. If i skip or retry the file, it will either fail again, or the process will continue only to hitch a few files later. If I select "all files" it won't notify me of failures, and yet a massive amount of video files will fail to play. VLC says they are corrupt. I basically have to restart the copy and overwrite files if I get an error. Which is really timely when I'm dealing with 1-3TB at a time.

 

I get no disk errors, the parity drive history says zero errors, but average speed was only 69.8MB/s which seems slow, even thought it is better than 10MB/s.

 

Also I currently have to stop all dockers just to get enough read/write integrity so I can use the drives. I plan on adding a cache, and potentionally upgrading the speed of my parity drives. And lastly I'm using High Water, and drives are set to never spin down. Because they're RED drives.

 

Hardware

CPU = 2x Intel Xeon X5675 @ 3.07GHz

RAM = 6x SK Hynix 16GB 2Rx4 PC3L-10600R-9 DDR3 1333 ECC 

Motherboard = ASUS Z8NA-D6C

HBA = 2x SAS 9211-8i

Boot USB = Toshiba 32GB

Share = 16x 4TB WD RED

Case = Chenbro RM41416 4U Rackmount

Power Supply = 3x Emacs MX1-5375p 375W (in n+1 Redundancy)

Cooler = 2x Thermaltake Water 3.0 Performer C

Fans = Delta? Those loud ones with +200CFM

 

I also generated the diagnostic file, but there's no way to attach a file without an account, so that will be coming soon.

 

Is there something I'm fundamentally doing wrong here? Do I have FreeNAS habits that need to be done differently?

[JorgeB]

See turbo write for better write speeds, but note that having two users writing at the same time will still slow down Unraid a lot, Unraid doesn't stripe disks, so it can never perform as well as RAID soltutions, it does have other advantages though.

[DR4GON]

That might be so, but I currently can't even have one user write to the disk without running into crippling speeds and errors that result in data corruption. So far my transition from FreeNAS to Unraid has rendered 350GB (aprox 1% of my data) corrupt and unusable, requiring a new copy/overwrite that sometimes fails a second time. Attached is the windows error while copying to the drive via windows.

 

What's the solution to circumventing the RAID issue in normal situations? Is there a plugin to handle software RAID? Should I use the shares to segment out drives so two users can write to the server at the same time, by writing to different disks? (Though I assume I'm limited by the Parity disk instead in that case)

 

Just not sure what best practice is, as I need Plex to be able to perform library maintenance while Transmission downloads, or a user is read/writing to the server, and I currently have dockers closed just to copy files...

[JorgeB]

12 minutes ago, DR4GON said:

that result in data corruption.

That's obviously not normal, diagnostics grabbed after a problem might give some clues.

[DR4GON]

When I wake up tomorrow morning I’ll post the diagnostics.

[DR4GON]

I enabled Turbo Write via my phone as I was going to bed last night and today woke up to speeds between 80-120MB/s, but I still had a transfer failure. I've attached my Unraid diagnostics.

 

After some googling, am I bumping against a limitation of both FreeNAS and Unraid? FreeNAS with the NAS functionallity, and Unraid with the VM functionality, should I pivot to Proxmox and then run VM's that way?

dragonunraid-diagnostics-20210313-1145.zip

[JorgeB]

Don't see any errors logged, problem could be the source computer, or some issue with the network.

[DR4GON]

I've lost another 400GB of files. Corrupted 2 seasons of Newsroom. Nothing else on the system has changed. In the past I've moved 40TB back and forth over 2 months with zero issues and no loss of data integrity. I know a large portion of my data can be re downloaded, but the portion that cannot be is now sitting in my office across 18x 2TB drives. If i play a file on an external drive it plays fine. If i play the same file after it was copied to the server, its now a damaged MKV...

 

I have 3 computers that can, at random intervals, reproduce the error message posted above. And If I create and use a VM on the computer, Ubuntu18.04 gives me no error while copying files from USB or SATA to the user share but all my How I Met Your Mother episodes now have weird audio glitches and wont let me time skip them at all..

 

Long story short then, if there is no issue with the server configuration, in my situation do I abandon Unraid?

[JorgeB]

There's likely some hardware issue causing the data corruption, could even be unrelated to the server, like a network problem, but since I don't see any errors on the log you'd need to try various different things, like different source PC, NIC, switch, cables, etc, you can even boot another PC with the Unraid flash and test until you find the problem.

[DR4GON]

So if I can repeat the issue with 3 pcs, the VM on the server and via LAN and WIFI, what do I test next? Loading up unraid on another computer? I dont have another pc to use to stress test.

[JorgeB]

Another NIC, switch, and yes if those don't help a different PC as Unraid, you just need to boot a PC with the Unraid flash drive and have an available disk for testing.