Jump to content
Dave-M

Need some help with a few things please

23 posts in this topic Last Reply

Recommended Posts

I have a few issues that I'm hoping you can help with before I buy my license.

 

I've been trying to transfer around 2 TB of mixed data, movies, photos, DSLR raw files, general documents and a lot of music in various formats.  I'm using a Mac to log into the server and then transfer the files like that, they are samba shares.  It is telling me that I am getting around 26mb/sec at the most, but much of the transfer has been around 4mb/sec or less, a lot of it was running at just a few hundred kb/sec. I have tried with the cache on and off and it does not seem to help at all.  Is there a way to test the system to find out what the problem is?  It is an Asus RT-A68U 1Gbe router and everything is hooked up with decent quality 2m cat 6 cables, yesterday I also tried different cables and also wireless on the Mac, but that had no effect.  The hard drive that the files are coming from is good for over 100mb/sec reads via USB 3.

 

Also, I checked the number of bytes in each folder and while most are the same, some of the larger folders are slightly bigger or smaller, by a few hundred bytes.  What would cause this?  Do I need to delete the folder and re-transfer it?

 

If I use the Dynamix s3 sleep plugin, the server is unresponsive in the morning, the fans are spinning very slowly, the power LED is on, along with the LEDs on the motherboard, but it will not wake up via keyboard presses, mouse movement etc, and there is no connection via a web browser.  It needs to be powered off and back on to fix it.  I would like to have the server sleep when not in use and I know the motherboard allows for this and as far as I can see, all the power saving settings are enabled in the BIOS. Can anyone shed any light on this please?

 

Finally, the terminal button at the top right of the main Unraid screen doesn't work, it just gives me a grey/blank pop up screen.  Likewise, if I choose the command line option from Krusader, again, I get a light grey blank screen.

 

Please let me know what additional info you need and I'll post it ASAP, cheers.

 

3x IronWolf 4TB

Samsung 970 pro 512GB NVMe for the cache

Asus P11C-I

16GB ECC 2400Mhz

i3 9100f

Share this post


Link to post
4 minutes ago, Dave-M said:

Also, I checked the number of bytes in each folder and while most are the same, some of the larger folders are slightly bigger or smaller, by a few hundred bytes.  What would cause this?

This is likely just an artifact of the way different OS and software report size. There is a lot of variation and they don't always tell you whether they are reporting MB or MiB, etc.

 

https://en.wikipedia.org/wiki/Mebibyte

 

9 minutes ago, Dave-M said:

Finally, the terminal button at the top right of the main Unraid screen doesn't work, it just gives me a grey/blank pop up screen.  Likewise, if I choose the command line option from Krusader, again, I get a light grey blank screen.

Is this from the server GUI mode builtin browser or from another browser on a different computer? Which browser?

 

3 minutes ago, Dave-M said:

Please let me know what additional info you need and I'll post it ASAP, cheers.

Go to Tools - Diagnostics and attach the complete diagnostics zip file to your NEXT post.

Share this post


Link to post

Thanks for the quick reply, I have attached the diagnostics.  Some of the folders are the same size exactly, to the byte, others not so, which seemed odd, you'd think they would all be out by a few hundred bytes?

 

Browser is Safari Version 13.0.3 (14608.3.10.10.1) on iMac running Mojave 10.14.6.   I have attached a couple of screenshots so you can see what I mean.

 

Thanks.

 

krusader-console1.thumb.png.2e9f82e0643109ca941d5ae9395f6835.pngunraid-console1.thumb.png.3b5c65744c2fe608ed5fe0f6425359c4.png

unraid-diagnostics-20191126-1336.zip

Share this post


Link to post
2 hours ago, Dave-M said:

Browser is Safari Version 13.0.3

Can you try a different browser to see if you get a different result?

 

 

Share this post


Link to post
26 minutes ago, trurl said:

Can you try a different browser to see if you get a different result?

 

 

Works for Krusader and the console with Firefox 70.0.1.

Share this post


Link to post

Managed to fix a couple of things.

 

The different file and folder sizes seems to be from deleted files on external drives that have not been emptied from the recycle bin on the Mac properly.  I had been deleting duplicates and files and folders that were no longer needed.  It seems to think that the files are still there and attempts to send them to the Unraid array, but at the end of the transfer, it says, could not find "name of the file" "no such file" or something along those lines.  The fix seems to be, before sending large folders (500+GB) to the Unraid array, empty the recycle bin and reboot the Mac.  Hopefully that helps someone else.

 

Thanks to Truro for suggesting trying a different browser.  The Krusader blank screen problem seems to be related to Safari as it works with Firefox.  Hopefully the Safari side of things can be fixed for those of us that use Safari?

 

Regarding the slow transfers, it seems as though my btrfs cache disk was not functioning correctly and there were 110GB of files stuck there and the mover did not shift them, I did some googling and it seems that people describe btrfs as sometimes being a bit brittle, it only lasted a few minutes for me before breaking, I'm not sure why.  I deleted the cache by reformatting the cache drive with xfs.  No idea if this is suitable long term, but it seems to work well at the moment as I can now get 110mb/sec into the array from my Mac.

 

The S3 sleep thing is still broken for me.  Can anyone help with this please?  

 

Anyone for the S3 thing?

 

Share this post


Link to post
5 minutes ago, Dave-M said:

it seems as though my btrfs cache disk was not functioning correctly and there were 110GB of files stuck there and the mover did not shift them

Reviewing your already posted diagnostics, I think it is more likely that you had the Use cache setting incorrect. Mover never moves cache-no shares, and you had two shares with files on cache but set to cache-no.

 

You also have your appdata as cache-yes instead of cache-prefer or cache-only. To get that moved to cache where it belongs set appdata to cache-prefer, disable the docker service, then run mover.

 

See this FAQ for more details on the nuances of the Use cache setting:

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#comment-537383

 

Share this post


Link to post
7 minutes ago, trurl said:

Reviewing your already posted diagnostics, I think it is more likely that you had the Use cache setting incorrect. Mover never moves cache-no shares, and you had two shares with files on cache but set to cache-no.

 

You also have your appdata as cache-yes instead of cache-prefer or cache-only. To get that moved to cache where it belongs set appdata to cache-prefer, disable the docker service, then run mover.

 

See this FAQ for more details on the nuances of the Use cache setting:

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#comment-537383

 

Thanks for the reply.  Here's an updated diagnostics, I changed a lot of stuff since the last one and the cache seems to be working well now, but I will do what you said, if I haven't already.  Would this affect the S3 sleep?

unraid-diagnostics-20191130-1719.zip

Share this post


Link to post
2 hours ago, Dave-M said:

I changed a lot of stuff

You have your appdata on cache now where it should be, but unlike before you have your system share with some files on the array. Best way to fix docker now is to go to Settings - Docker, disable and delete docker image, then enable to recreate docker image on cache where it belongs. App - Previous Apps will add your dockers back just as they were.

 

You want your docker applications and VMs to run on cache so they will perform better and so they won't keep array disks spinning. Do you have any VMs?

Share this post


Link to post

Your system share still has files on array disks 1 and 2.

 

What do you get from the command line with these?

ls -lah /mnt/cache/system
ls -lah /mnt/disk1/system
ls -lah /mnt/disk2/system

 

Share this post


Link to post
sh-5.0# ls -lah /mnt/cache/system
ls: cannot access '/mnt/cache/system': No such file or directory
sh-5.0# ls -lah /mnt/disk1/system
ls: cannot access '/mnt/disk1/system': No such file or directory
sh-5.0# ls -lah /mnt/disk2/system
ls: cannot access '/mnt/disk2/system': No such file or directory
sh-5.0# 

 

Share this post


Link to post
6 minutes ago, Dave-M said:

sh-5.0# ls -lah /mnt/cache/system
ls: cannot access '/mnt/cache/system': No such file or directory
sh-5.0# ls -lah /mnt/disk1/system
ls: cannot access '/mnt/disk1/system': No such file or directory
sh-5.0# ls -lah /mnt/disk2/system
ls: cannot access '/mnt/disk2/system': No such file or directory
sh-5.0# 

 

Have you done anything since you posted your last diagnostics? Is the array actually started?

 

That sh-5.0# doesn't seem right either. Are you actually at the command line of your Unraid server?

Share this post


Link to post
3 minutes ago, trurl said:

Have you done anything since you posted your last diagnostics? Is the array actually started?

 

That sh-5.0# doesn't seem right either. Are you actually at the command line of your Unraid server?

 

Thanks again.  The array is started and nope, I haven't done anything else, other than reinstalled crusader.  Do you mean the command line on the server itself rather than from in the webUI?  The above was from the web command line, I'll do it the other way, sorry, I'm learning!

 

 

Share this post


Link to post

The webUI command line should work the same. Something not right about your results though. From my server the prompt is

root@unserver:~#

where root is the user (the only user that can access the command line) and unserver is my server name.

 

 

I don't know how you could even get it to say

sh-5.0#

 

What do you get with this?

ls -lah /mnt/user

 

Share this post


Link to post
3 minutes ago, trurl said:

The webUI command line should work the same. Something not right about your results though. From my server the prompt is


root@unserver:~#

where root is the user (the only user that can access the command line) and unserver is my server name.

 

 

I don't know how you could even get it to say


sh-5.0#

 

What do you get with this?


ls -lah /mnt/user

 

Are you maybe trying this from Safari again?

Share this post


Link to post

I just opened it and it said that as the prompt.  

7 minutes ago, trurl said:

The webUI command line should work the same. Something not right about your results though. From my server the prompt is


root@unserver:~#

where root is the user (the only user that can access the command line) and unserver is my server name.

 

 

I don't know how you could even get it to say


sh-5.0#

 

What do you get with this?


ls -lah /mnt/user

 

I just opened it and that was the prompt, I have just opened it again and it was root@unraid, so I'm not sure what that was.  Two screenshots of the other things you asked for.  Thanks.

 

cmmdline1.thumb.png.b7628bd8be770cddc7e416110511e4b3.png20191130_220249.thumb.jpg.f7c97a92ca9fe5d644a58af9c3d3aeab.jpg

 

 

 

 

Share this post


Link to post
10 minutes ago, trurl said:

Are you maybe trying this from Safari again?

No the weird command prompt was from Firefox.  I have both open, I really don't like Firefox, so just use it for what doesn't work on safari.

Share this post


Link to post

Since you don't have any VMs yet, you can just disable VMs in VM Manager then delete those libvirt folders. You don't want them on the array and if you recreate them they should get recreated on cache anyway.

 

Same for Docker, disable and then delete those docker folders. Then you can recreate docker image and it should get created on cache where it belongs. Apps - Previous Apps will add your dockers just as they were.

 

Do you know how to work at the command line to delete those folders? I usually just use the builtin mc (Midnight Commander) when working with files and folders directly on the disks.

 

Then the system share should only exist on cache.

Share this post


Link to post

Thanks for taking the time to help, much appreciated!

 

No, I don't really know any command line stuff yet, but am willing to learn.

 

I've not seen midnight commander, I see a Terminal button at the top right?

Share this post


Link to post

At the terminal, typing mc will launch Midnight Commander, a text-GUI file manager similar to the Commander type file managers from the DOS days. It is pretty easy to figure out, and a lot of information on the web about using it.

 

 

Share this post


Link to post

I only work directly with the disks when using mc. You must never mix user shares and disks when working with the files since linux won't know that the source and destination may actually be the same file.

Share this post


Link to post

Cheers, I'll give it a go in the morning, thanks again for the help 👍

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.