Server reshuffle

[pras1011]

I have downloaded the 2.3 version of tercopy. Whats W10? Everything is moving along without too much issue using copying on Win 8.1.

 

One issue is that I tried to add the SD share to disk1 via the shares menu but it has not appeared on disk1 and now the SD share on disk12 has disappeared via Win 8.1. But I can still see on the unraid web gui.

[garycase]

Definitely agree that TeraCopy is an excellent copy utility.  But, like Daniel, I only use the stable version 

[garycase]

... Whats W10?

 

Windows 10

 

[garycase]

... Available for free here:  http://windows.microsoft.com/en-us/windows/preview-iso

 

Note that this is a "Preview" version -- the actual release isn't scheduled until mid-summer.

 

It works quite nicely, however => it's a nice mix of the best features of '7 and '8.1  (although Microsoft probably doesn't like that characterization of it)

 

[pras1011]

I already have it on my laptop. I get mine from Teamos.

[pras1011]

I am transferring tiny files from disk to disk (less than 400mb) and the server is struggling! Some files fly through others go down to 2MB/s or lower.

[danioj]

I am transferring tiny files from disk to disk (less than 400mb) and the server is struggling! Some files fly through others go down to 2MB/s or lower.

 

I'm guessing those are "small" files. If so, totally normal.

[pras1011]

I copied a few of the small files from disk12 to my pc and then copied them to disk1.

 

Way way faster. Confused and annoyed!

[danioj]

I copied a few of the small files from disk12 to my pc and then copied them to disk1.

 

Way way faster. Confused and annoyed!

 

A watched pot never boils. You can press en elevator button 100 times it still wont come any faster. Pick one analogy that works for you BUT Dude - step away from the machine. Go do something else - it will finish when it finishes!! 

[garycase]

I copied a few of the small files from disk12 to my pc and then copied them to disk1.

 

Way way faster. Confused and annoyed!

 

When you're doing disk -> disk copies on your PC, the disk policies are set for write caching.  When you're copying across a network this is not the case.    Without write caching, there's a directory seek; and rotational latency delay after EACH file as the directory is updated ... and then you have the same delay to reposition the heads to write the next file.  With write caching all of these "writes" still take place ... but they are cached, so there's no actual disk movement until necessary -- so writes are FAR more efficient.

 

In other words, what you're seeing is completely normal and exactly what you should see.

 

[pras1011]

Thanks.

 

These small files are causing so many problems but I am getting through them now.

[pras1011]

Right I completely deleted disk12 and disk13. How do I remove them from the server?

[trurl]

Right I completely deleted disk12 and disk13. How do I remove them from the server?

New config and rebuild parity.

[pras1011]

I am at another frustrating problem:

 

On disk 1 there is abou 417gb left and the server is refusing to let anything be written to it. It writes for a little bit and then gets stuck at a very low speed. Then I have to delete the partial file and it takes forever to delete it!

 

Any ideas?

[garycase]

Post a screenshot of the Web GUI.  Also, what are you split level settings and your "min free" setting?

 

[pras1011]

I resolved it by rebooting. I noticed in the syslog that the server was complaining that a directory share has gone missing. I had deleted it.

 

My HD min is 80gb. My sd is 15gb. My split level is on Automatically split any directory as required.

 

I think I am hitting a new problem. Using teracopy I am trying to transfer 26gb of 3716 tiny music files. As the transfer progresses the write gets slower and slower and then it stops at about 50%. I then have to manually restart it and its starts writing at full speed. I then pause the transfer when its starts getting slow again. I then unpause and it goes back to a good speed!

 

SMR probably?

[garycase]

I suspect that's indeed SMR.    Interesting data point that you're having this issue.

 

Are you writing to a new (never written) drive?  Or is this a drive that's had files written and erased?

 

One issue with writing a very large number of very small files is that none of the files will be large enough to fill a shingled block; and there will be an intervening seek and write to a different location after each file (the directory update) ... so this is likely causing a LOT of usage of the persistent cache.  If your persistent cache is full, then performance would indeed be abysmal in this case.

 

If you can take a day off to experiment, stop all copy activity;  let the system set idle for a LONG time (10-12 hours) ... this should ensure the persistent cache will get emptied by the drive's firmware;  and then transfer perhaps 10GB of your "tiny music files".  The 25GB persistent cache should be enough to cache both those writes and the associated directory writes; so the write speed should be fairly consistent (still fairly slow since these are all small files, but I'd expect it to be fairly consistent).

 

If you don't have the time (or inclination) to do that, then simply pausing it for a while whenever it happens will likely resolve it just fine -- the longer the pause the better, as it will give the drive more time to clear the persistent cache (assuming that's indeed the issue).

 

 

[pras1011]

I did it in the end using the pause technique. I have about 370gb left on disk1.

 

I think you are right about leaving it and letting the cache clear. Come to think of it, I think it was doing this early this morning. There were no activity lights on and I could hear the hdds doing something.

[garycase]

Hopefully this is relatively static data and large copies like that aren't something you do regularly.  Seagate's mitigations should work well for copies that don't exceed the persistent cache -- but that 25GB can be filled with a lot less than 25GB of copies if you're writing a lot of small files, since every directory entry would also contribute to filling the cache.    With a relatively few large media files, it would indeed work for close to 25GB of data ... but with tiny files it could easily be as little as half of that before there are issues with write speeds.

 

Fortunately, READING all that data isn't impacted by the shingled technology 

[pras1011]

Once I filled these drives up to the top they will very rarely be written to ever again.

 

Is it better to use on board sata ports or to use a sas card? I think the sata ports maybe faster. Is there a lga1150 matx mobo with 10+ ports that's good for gaming and server use? The asus gene vii looks good but it only has 8 sata ports.

[garycase]

An add-in card will be just as fast as the onboard ports as long as it's in a slot with sufficient bandwidth to support the # of ports it has.    Most of the popular cards used in UnRAID servers easily meet that criteria.

 

[pras1011]

I am having the same problem again that I mentioned in post 113. It keeps coming up with unexpected network error. I then try and stop the server and it takes ages.

 

Can someone check my syslog?

New_Text_Document.txt

[pras1011]

Finished the transfer.

 

Look at the parity rebuild speed!!

[ysss]

Finished the transfer.

 

Look at the parity rebuild speed!!

 

Let's see the final average

[garycase]

NICE !!  But of course it will slow down a good bit as it moves towards the inner cylinders.

 

I'd still expect a very nice average, however.