Hi,

 

I'm using unraid on a server to distribute my music collection to 2 Mac computers (running 10.11.6, El Capitan). One computer is used to manage the library (Hackintosh 3.49 GHz Intel Core i7 with 32 Gb of RAM), add up files etc... The other one is used to stream to my stereo using Audirvana (Mac Mini 2009 with SSD and 8 Gb of RAM).

 

This is now nearly impossible to achieve and I'm very close to throwing the towel and look somewhere (FreeNas, Nas4Free, WD Mycloud... don't know).

 

I'm running 6.2.4. I have a parity drive of 4 Tb and 4 drives of 2 Tb each and a SSD cache drive of 120 Gb. This is on a ASRock - E350M1 motherboard with 8 Gb of RAM.

I'm in the process of having all the drive formatted XFS, not that it makes any difference so far. I have tried mounting the music share with AFP, SMB or NFS.

SMB is terrible. The finder takes forever and I mean forever. AFP was ok from the finder but gives me error in copying files etc... when I use itunes to import. I gave up using iTunes to manage the library but just tried again. Disaster. From the finder it still takes quite a few second before anything happens.

NFS is not really any better.

 

Access from the other computer with Audirvana through AFP. It takes over 2 days to update the library (not even finished). Yes there's 80000 tracks but still, this is unusable. Buy a new album and maybe you can listen to it in a week, assuming the Mac doesn't crash in the meantime.

 

Anyone has any recipe that would make this thing work??

Any verification that I should run? Parity seems ok. I have been transferring to drives that I reformatted XFS. 2 drives are at 50% and one at 33%. Last one is empty.

I have added these lines to the smb configuration (as per some post earlier).

[global]                                                                                                                                                               

 

ea support = yes

vfs objects = catia fruit streams_xattr

fruit:resource = file

fruit:metadata = netatalk

fruit:locking = none

fruit:encoding = native

 

Doesn't make any difference.

 

Any suggestion would be welcome !!

PS: I don't know much about Linux so any knowledge of command in terminal will be limited.

 

Thanks

 

 

For Mac connectivity, see this recent thread and consider either using AFP and moving the Netatalk database onto your cache disk as I did, or switching to unRAID v.6.3.0rc6 and using SMB (but see the notes in that thread about the vfs_fruits options).

 

There may be a problem with your unRAID server that has gone unnoticed. Make sure you have notifications turned on (Settings -> Notification Settings) and if you're unsure of anything post your diagnostics here (Tools -> Diagnostics and attach the zip file to your next post).

 

There may be a problem with your unRAID server that has gone unnoticed. Make sure you have notifications turned on (Settings -> Notification Settings) and if you're unsure of anything post your diagnostics here (Tools -> Diagnostics and attach the zip file to your next post).

^^^This

 

I don't use AFP or Macs but I have no problems with an iTunes library on my server using SMB and Windows.

 

Since you mentioned you are reformatting to XFS that suggests to me that you haven't really established that your server is running well. Best to take things one at a time and get it working correctly before trying to add more complexity.

Is the server presently doing the conversion to XFS?  If so, how are you copying the files from one disk to another if you are are command-line 'limited'?  The method that you are using could be doing a major hit on performance.

 

I really think you need to post up your diagnostics file. 

Attached is the diagnostics run.

Yes I did the XFS conversion using the server.

command was  nohup cp -r -p - v /mnt/disk#/* /mnt/disk#

connected to the server by telnet

I copied one drive onto the empty one and then repeated the operation for each one.

I can enter these commands but I don't know how I would "move the Netatalk database" to the cache for example.

The problem existed before I changed to XFS. Why did I change anyway? Good question. Seemed like the right thing to do for the future.

Anyway, here's the diagnostic. Doing Parity check right now too.

 

Thanks for your help

tower-diagnostics-20170127-0718.zip

I don't know how I would "move the Netatalk database" to the cache for example.

 

For each share set the Volume dbpath to /mnt/user/system/AppleDB and the next time you AFP mount it the .AppleDB folder will be relocated from the root of the share to /mnt/user/system/AppleDB/<Share_name>/.AppleDB

So I ran parity  and it's fine

I ran xfs_repair on each drive with no error. I think it might have killed my SSD drive however.

John_M where do I find the AFP security? I don't seem to have the option that you present

To me AFP is limited to this???

never mind - got it

So all drives, including cache passed the xfs_repair. Everything seems in order...

Just tried to copy files to the music folder... Speed seems good (see attached) at 500 Mb/s but it tooks minutes before it started the transfer !!!!

There's definitely a problem between the Mac and the array talking to each other... Any more suggestion??

I will try AFP again next but I'm doubtful

Do you have your drives set to spin down? Were they spun down? If so, sometimes there is a delay when the drives spin up.

Copying many small files will take a long time due to the overheads per file. How big are your music files? A few MB? Try copying a few large files instead, video files perhaps.

 

One thing I notice is that you have an E-350 processor, which is quite weak. I notice it's working quite hard in your screen grab. I think you could improve throughput by fitting a better NIC which would offload some of the network overhead from the main processor. Disable the onboard Realtek and fit an Intel one.

 

Copying a 15 Gb file using NFS...

About 400 Mb/s... I can live with that

The problem is when itunes or more importantly Audirvana will try to read the library... then it comes to a stall... I will try again to read the library using NFS and report.  Both AFP and SMB just too slow... even with proposed solutions. Maybe it's on the Mac side ?? Then I don't know. One of the Mac is El Capitan without ever entering anything in the terminal (so nothing changed).

 

John,

 

Thanks ! It has crossed my mind that maybe It should upgrade the system but I'm not sure if it will improve speed?? I need a mini-ITX motherboard with ideally 4-5 SATA...

You can also go to 'Settings'  >>>  'Dis kSettings' and then set "Tunable (md_write_method):" to 'write reconstruct'.  This is will about double your write speed.  But that setting will spin up all of your disks rather than just the one data disk and the parity disk(s). 

It's really the interface between the Mac and the array...

Here's audirvana trying to find the files... (hasn't started importing yet). through NFS.

Copying a 15 Gb file using NFS...

About 400 Mb/s... I can live with that

The problem is when itunes or more importantly Audirvana will try to read the library... then it comes to a stall... I will try again to read the library using NFS and report.  Both AFP and SMB just too slow... even with proposed solutions. Maybe it's on the Mac side ?? Then I don't know. One of the Mac is El Capitan without ever entering anything in the terminal (so nothing changed).

 

Make sure that the NIC chip drivers are up to date on the Mac's.  I had an out of date RealTek NIC driver on my Windows 7 computer and that driver would transfer from the server at 100MB/sec or as slow as 2MB/sec.  After I realized that I  had an issue, it took me the better part of a month to finally figure out what was the true problem!

This is ridiculous...

 

I have accessed through NFS the same folder from the two different Mac using Audirvana to build the music library. This took over night but both finished.

One has 33863 tracks, the other one has 87316 tracks and both of them are missing tracks !!!!

 

both of them are missing tracks !!!!

 

Then the problem is not likely to be with unRAID. In this case it's simply dumb storage. It's a bit slow because of the NIC and the processor but if it's told to save a file it will do so - unless the diagnostics report an error.

 

I'd take a look at Audirvana's logs to find out why its library is missing files. I don't use it myself so I can't help there. Does it create a lot of metadata files? Lots of small files across a network cripple performance.

 

Also, if you're troubleshooting you need to keep things constant and just change one at a time. I though you had decided to use AFP - now you say you're using NFS. Put yourself in our position for a moment - it's very difficult to second guess what you're doing.

 

John,

 

I'm sorry but AFP and SMB were unusable. Just way tooo slow

Can I transfer the array to new hardware?

What would be a cheap improvement? going to a Pentium or i3?

Thanks

You can replace the motherboard and processor and keep your existing disks, case and power supply. The configuration is stored on your USB flash, so swapping hardware is easy.

 

A Pentium or Celeron or an AMD A-series APU with the appropriate motherboard would give you more power than you currently have. But before abandon your current system you said you were getting 400 Mb/s on large file copies. That's 50 MB/s, which is about the limit for writes to the parity-protected array. Try changing the tunable that Frank suggested.

 

Thanks John,

 

Yes I will try that. The copying speed to not that fast but I can live with it. As you mention it's only for dumb storage and I only stream music files (SACD iso too) but no Plex or anything demanding.

So while it's a low end set up, it should fill the bill.

The real problem is creating a library and/or modifying it. I'm currently trying to rename files through audirvana to organize library and get the mac wheel for minutes at a time. It shows 90-100% cpu usage and 160 Mb/s network usage but not writing on disks. The problem might not be array related but more interface between the array and the macs.

Was looking at new set up with i3 6100, taking me back $350 or so... could spend it somewhere else

Thanks for your help. Much Appreciated.

 

The thing is, I see the task of organising a library to involve the reading and indexing of lots of files and writing the output to another file. Opening and closing many files across a network is a slow process. You've shown that your existing system can write large files at the maximum speed, so my concern is that you spend money on a new one only to find that it is little or no faster for your application. Have you tried copying the small music file or doing the library creation/organising between your Macs, as a test, using one as a file server for the other? It might be that it's just something that isn't best suited to doing across a network.

 

FWIW, I've seen a couple of threads recently where low power CPUs have been discussed - Atom D525 and AMD C-70, both in a similar ballpark as yours. For a basic NAS without bells and whistles it should be enough.