Hi All -

I inherited an older (circa 2014) server and the idea was to replace my low end Synology (418j) system with it to at minimum be my media host (with various Kodi clients in the home). Talking to my buddy who used the server before donating it to me, he did not experience similar issues serving media (probably larger sized media) so I have to think it's unRAID as he wasn't using unRAID.

 

Trying to iron out issues before purchasing. Thanks!

 

Setup:

Kodi client on Nvidia Shield connecting to Tower (and to Synology) via smb shares.

 

Problem:

Starting media from Tower is tediously slow compared to Synology drive. Testing using same client and same movie I'm seeing start time of approximately 3-4 seconds from Synology share vs. 35+ seconds from Tower.

 

Maybe worth noting:

(1) I keep seeing a single process eating up my Tower CPU. The process is SHFS.

I've found a couple older threads saying the same thing but everyone points to stopping dockers, I don't have any dockers running. Literally just a raid setup and a single user with a samba share established and "working". Currently absolutely nothing being actively served (no client playing anything) but I'm seeing SHFS using 61.6%.

 

(2) Once media starts, it seems pretty seamless (from both Tower and Synology hosts).

 

Log attached.

tower-diagnostics-20230705-1547.zip

Edited July 5, 20233 yr by JRubenol

Are you spinning down the array drives?

 

If you are, trying spinning up all of them.  (You will find a button to do this on the MAIN tab/page.)  Then see what the delay time is.  

 

I had a very quick look at that CPU.  It was introduced in 2011 and has a PassMark rating of 311 which is very low.  SHFS manages access to data files if you acess them through User Shares.  It adds another layer to the file management overhead.  Having a slow CPU will increase it impact even more. 

1 hour ago, Frank1940 said:

Are you spinning down the array drives?

 

If you are, trying spinning up all of them.  (You will find a button to do this on the MAIN tab/page.)  Then see what the delay time is.  

 

I had a very quick look at that CPU.  It was introduced in 2011 and has a PassMark rating of 311 which is very low.  SHFS manages access to data files if you acess them through User Shares.  It adds another layer to the file management overhead.  Having a slow CPU will increase it impact even more. 

 

Agreed the CPU is ancient but given that he had great success using this for a basic server for his media, I'm thinking it's sufficient. But, what am I looking to change here? Attached screenshot of settings.

 

https://imgur.com/a/orvujcd

Ok more information is needed.....very weird stuff going on. Could the amount of stuff in the directory make a difference? I just did a test of a tv show with a lot less inside and it started within 2 seconds......

 

EDIT: I did a test and moved the exact same file into a test directory without any other content, starts within 2-3 seconds......so my above point seems on point but I'm not sure why that would make a difference. Thoughts appreciated.

Edited July 6, 20233 yr by JRubenol

Possibly not the only issue, but there are ATA errors with disk2, replace cables and post new diags after array start.

4 hours ago, JorgeB said:

Possibly not the only issue, but there are ATA errors with disk2, replace cables and post new diags after array start.

Hm I'm just not convinced that this has anything to do with the symptoms I'm seeing.

 

Additional test: moved all media beginning in letter A to a directory (about 200 gigs) and played same file. About 5-6 seconds to start.

 

What is a abundantly clear is that directory size (unclear if it's # of files or size of directory) is impacting the time it takes for media to start but has no impact on the media itself once it has started.

 

To summarize:

File being only file in directory plays within 2-3 seconds

File being in directory with approximately 200 gigs of other files takes about 5-6 seconds

File being in directory with approximately 1.8 TB of other files takes about 35-40 seconds.

 

 

Try using a disk share to see if it makes it any better.

I almost hesitate to suggest this!  You could try the Dynamix Cache Dirs plugin.  (Install it via the Apps tab.)   It basically caches file and directory information in memory.

 

NOW, for the reasons for my hesitation! 

 

First, It does need RAM to work and the amount of RAm required depends on how many files/directories you assign to it.  The fewer the better the performance.  (If you assign more than RAM can accommodate, I understand it will cause disk thrashing as cached entries are deleted as additional are being added.  When these deleted entries are noted , they will be added back resulting in other entries being deleted...)  So only assign those Shares to it where you absolutely need fast response.

 

Second, there are a lot of 'tuning' options for it.   (On the settings page for the app, turn on the 'Help'--- the icon at the right side of the GUI Toolbar.)  Your hardware determines which settings will result in the best performance of the plugin. 

 

Third, I suspect that support is minimal.  I don't believe there is anyone support the script that does the actual work.  Any updates are to the GUI front end that passes the switches to the script.  These GUI updates appear to be to make the GUI front end work with the current version of Unraid. 

14 hours ago, JorgeB said:

Try using a disk share to see if it makes it any better.

Initial tests seem to show that this resolves the problem. Trying to figure out how to create a pool and access it that way.

 

I'm happy to continue testing and providing logs if it'll help diagnose the issue with the user share.

4 hours ago, JRubenol said:

Trying to figure out how to create a pool and access it that way.

With v6.12 if a share is limited to a pool you can enable exclusive share mode, that will give you the same performance as using a disk share.

Any guesses as to why the huge difference? 2-3 seconds to start a media item vs 35-45. Is it something about how unRAID handles arrays ? I'm debating if I start from scratch this weekend with Manjaro and just good old RAID5......

38 minutes ago, JRubenol said:

Any guesses as to why the huge difference? 2-3 seconds to start a media item vs 35-45. Is it something about how unRAID handles arrays ?

As I understand it, The process SHFS is added as another layer in the file read/write protocol.  It apparently combines all of the file table data from all the disks for a share into a common table and files are accessed using that data.  This process is quite CPU intensive and you are seeing that born out with your CPU.

 

The PassMark speed for that CPU (AMD C-60 APU) is 311.  My first Unraid server used a AMD Sempron 140 (Introduced in 2004) and it had a PassMark speed of 475.  (At the time when I purchased it in about 2007, it was considered a dog of a processor.  But I didn't have much money and I just wanted a computer to 'play around' with Linux.  That bare-bone box was the cheapest I could find...)

1 minute ago, Frank1940 said:

As I understand it, The process SHFS is added as another layer in the file read/write protocol.  It apparently combines all of the file table data from all the disks for a share into a common table and files are accessed using that data.  This process is quite CPU intensive and you are seeing that born out with your CPU.

 

The PassMark speed for that CPU (AMD C-60 APU) is 311.  My first Unraid server used a AMD Sempron 140 (Introduced in 2004) and it had a PassMark speed of 475.  (At the time when I purchased it in about 2007, it was considered a dog of a processor.  But I didn't have much money and I just wanted a computer to 'play around' with Linux.  That bare-bone box was the cheapest I could find...)

So I guess my question is if RAID5 would be less CPU intensive ? As I mentioned above, my buddy was running a RAID5 on this system and didn't have any issues. Just trying to learn something. Also, SHFS isn't needed for disk share?

Google found this:

 

    https://premioinc.com/blogs/blog/hardware-vs-software-raid-differences-pros-cons

 

I would suggest that you google to find more information about RAID and CPU interaction.  Remember that many long years ago, serious servers all used hardware RAID controllers rather than software RAIDs.

 

Remember that some people learn to live with 'start playing' delays.  It is also affected by whether or not you spin down drives when they are not being used.  (Most Unraid users apparently do this as it saves power and seems to extend harddisk live.

 

9 hours ago, JRubenol said:

Also, SHFS isn't needed for disk share?

NO.

 

As an aside, I just started a movies playing on my Media server (Spec below) using KODI with a common media database stored by MySQL Docker on the SSD Cache drive which points to a Share on my array.  It took seven seconds from clicking on the movie (one of 1328 movies) to be played until playback began.   This included the spin up time of the hard disk.  A test of the SHFS delay, I did a KODI file loading of a movie from a local hard drive and it took about three seconds.  (So total server overhead was about four seconds.)

 

Most Unraid users live with the delays forced by SHFS and the slow write speeds of using using the array.  Yes, using a RAID solution would fix speed problems but at the cost of the flexibility and ease of use of LimeTech's parity scheme.  Both of my servers have had their storage capacity two or three times each  increased without any hassle on my part beyond replacing the parity drive and either adding or rebuilding with a larger data drive. 

 

2 hours ago, Frank1940 said:

Google found this:

 

    https://premioinc.com/blogs/blog/hardware-vs-software-raid-differences-pros-cons

 

I would suggest that you google to find more information about RAID and CPU interaction.  Remember that many long years ago, serious servers all used hardware RAID controllers rather than software RAIDs.

 

Remember that some people learn to live with 'start playing' delays.  It is also affected by whether or not you spin down drives when they are not being used.  (Most Unraid users apparently do this as it saves power and seems to extend harddisk live.

 

NO.

 

As an aside, I just started a movies playing on my Media server (Spec below) using KODI with a common media database stored by MySQL Docker on the SSD Cache drive which points to a Share on my array.  It took seven seconds from clicking on the movie (one of 1328 movies) to be played until playback began.   This included the spin up time of the hard disk.  A test of the SHFS delay, I did a KODI file loading of a movie from a local hard drive and it took about three seconds.  (So total server overhead was about four seconds.)

 

Most Unraid users live with the delays forced by SHFS and the slow write speeds of using using the array.  Yes, using a RAID solution would fix speed problems but at the cost of the flexibility and ease of use of LimeTech's parity scheme.  Both of my servers have had their storage capacity two or three times each  increased without any hassle on my part beyond replacing the parity drive and either adding or rebuilding with a larger data drive. 

 

This is all extremely useful so I'll start by saying thank you.

 

A bit more for my issue though. With the user share I think SHFS is maxing out my CPU to an extent that I'm actually seeing my folder time out loading. So my options are either leave unRAID and head to a standard RAID5 (not ideal but might test it.....), use disk shares, or I could in theory move my movies into sub-directories based on first letter of the movie. But I do think my answer is generally answered now, SHFS overhead appears to be maxing out CPU which is causing various issues including delayed start of up to 40 seconds and time outs of my folder.

 

I appreciate everyone's time!

One more thought.  Had a look at your Diagnostics again and your have a 1Gb NIC in your MB which is a positive.  Look at the MB manual (you should be able to download it from the manufacturer's website) and see what other AMD processors you might be able to use.  Look on E-Bay for a faster processor.  (Google PassMark CPU) They are often quite reasonable.  (Buy the vendor and try to buy local --- not international.  The 'Professional' vendors on E-Bay are proud of, and really want to protect, their reputations.) 

Thanks all for advice and helping me diagnose. I've decided to divide up my movies by first letter and to continue using unRAID with user shares. I'll keep in mind other options.

 

unRAID docs should probably reflect this in some way to save others the unknown that I hit.

 

All the Best