New Happy Unraid User but...

[jonathansayeb]

Hello dear Unraid community, first of all a big thank you to lime technology for this great piece of OS.

Here is a quick introduction:

I'm a photographer and videographer based in Vietnam and after 2 years of being in business and collecting 11tb of photos and video footages I thought that it was time to get rid of my super expensive dropbox advanced subscription and my usb 3 dock of hard drives and go for a proper server setup. Here come my first build for Unraid:

 

Motherboard: Intel Server Board S1200sp

CPU: Intel Xeon CPU E3-1220 v5 @ 3.00GHz

Ram: 2x 8gb Kingston ECC = 16gb ECC

Router: D-Link Dir-880L

Ethernet cables: Cat 5

Drives:

• Parity: WD Red 6tb (soon to be replaced with Hgst 8tb as soon as preclear is done)
• Data: 2 x WD Red 6tb, 2 x WD Blue 2tb, 1 x WD Purple 1tb
• Cache: 1 x OCZ vertex 3 256Gb (Installed in the build but not connected yet since I ran out of SATA slots, motherboard has only 6) Please give me your opinion on the best way to extend

 

Dockers Used so far:

• Krusader (for file management)
• Crashplan (Files backup)
• Deluge VPN (I don't use it much)
• PlexMediaServer
• Digikam (photo Management, just installed it and trying to figure it out)

 

Dockers to be used soon: Your opinion is most welcome

• Owncloud ( To replace my dropbox 600usd/year subscription)
• OpenVPN (for remote access to the server)

 

So far i'm pretty happy with the overall system and its flexibility (Disk Swaps/Upgrades if need be) but after 1 week working 16 hours/day transfering/organizing files from my main workstation (Imac 5K 2015) I'm facing issues that i really hope can be solved:

• Lightroom catalog stored on my Imac takes ages to import files from the Unraid server, i'm not even talking about creating the 1:1 Previews and the Smart Previews. (It's been going on for 2 days already. The catalog has >28000 images)
• Final Cut Pro crashes whenever I try to import footages from the server.
• Unstable network connection (from my Imac the connection to the server will randomly drop and it made me lose countless hours especially during big file transfers, BTW i'm connected via AFP only and disabled SMB since most people here said that in a mac environment it's better to stick with AFP), oh and while writing this it just dropped again so i attached the log 
• Weird thing is on my dashboard i see 2 users (root and myself jonathansayeb) when i click on my name it says the user as been deleted but if i go in the user panel i can clearly see the 2 users and access them. Is it just a bug?

 

The above issues really hurt the workflow of our studio and what used to take minutes now take hours and i'm ready to give up some speed in favor of a secure and centralised system but not too much. I've attached my system stats just so you can see. :D.

 

Here are my plans to try to improve the speed:

• Buy a Pcie Sata controller (To add additional hard drives to my config)
• Connect the SSD as a Cache drive
• Activate the newly released "turbo write" function
• Maybe try to change the cat5 cables to cat6
• Try a SFP+ peer to peer. (1 PCIe SFP+ in the Unraid server, 1Thunderbolt to SFP+ adapter on the Imac)
• Try Tunable (enable Direct IO):
• Switch from AFP to SMB
• Install Sierra VM on unraid server and deal with the Photo Catalogs (lightroom) and Video Catalogs (Final Cut Pro) from within the VM (BTW can I actually mount user shares on a sierra VM)

 

In the hope you guys can point me in the right direction that would be the most cost/efficient, I wish you all a stunning day!

 

[tdallen]

It sounds like you have several things going on, but I can only comment on one or two.

 

It seems like you are rebuilding your Lightroom catalog.  I haven't done that from my unRAID server so I'm not sure how long it will take.  Normally, though, my workflow is to copy files from a photo shoot onto my PC's SSD in a temporary location.  I then run the Lightroom import, with the pictures coming from my PC and going to the unRAID server as the destination.  I find that works well.

 

Some of the most popular PCIex SATA Controllers for unRAID are the IBM M1015 or Dell H310, which are pulled from servers and sold on eBay.  If that doesn't work, you might want to post a couple of the ones you can get locally for comment here.

[wgstarks]

Have you done any testing with SMB? I haven't had any problems with it at all. Don't use Lightroom or FCP though.

 

Might be worth testing SMB with the mac friendly settings enabled. They were a recent addition and really made a huge improvement on my system.

[1812]

Disabling SMB signing on the mac client will help with smb speeds to the server, detailed here:

 

 

also you put: Ethernet cables: Cat 5. If that is correct, then you'll never go fast.

 

I use to use afp with pre 6.2 unraid versions, but from then on, smb seem to do better than afp for speed and reliability. Mac clients connected to the server on 6.2.4 hit about 75% gigabit bandwidth and on the 6.4 release client 6 saturate gigabit.

 

I use final cut pro a few times a week and have no problems copying over large projects to and from the server.

 

 

 

[jonathansayeb]

Thank you all guys for  your comments!

@wgstarks i'll test soon after I finish a 4tb file transfer which will probably be tomorrow morning LOL.

@tdallen sorry for the confusion about the network cable, i meant cat5e.

@1812 when you use final cut, do you move your files from the finder or do you actually import them via the final cut import box?

[1812]

6 hours ago, jonathansayeb said:

Thank you all guys for  your comments!

@wgstarks i'll test soon after I finish a 4tb file transfer which will probably be tomorrow morning LOL.

@tdallen sorry for the confusion about the network cable, i meant cat5e.

@1812 when you use final cut, do you move your files from the finder or do you actually import them via the final cut import box?

 

If it is a project, I move the project to my local desktop or to the raid 0 scratch disks I have setup in the vm, managed by os x. If it is raw footage into the project, then I import that from the array to the locally hosted project.