[6.8.2] Help with Some Issues

[AwesomeAustn]

I bought a 24 rackmount server in August 2015. Lately, I have been having some issues and was wondering if there are certain components that I could upgrade or replace to make it better.

 

Every time I reboot, a hard drive says unmountable, and what I do is simply rebuild it, and it works as normal again. I have an unRAID system. Sometimes the hard drive is the same, and sometimes it’s a different one.

Another issue is Plex will crash frequently when adding new content. I have disabled thumbnail creation when adding new media. When Plex crashes, it causes all other Docker apps to not respond as well.

 

I also have an issue with the connections to all hard drives. When I try to rename files, no matter the hard drive, it takes a couple to a few minutes to rename one file. It will then give a connection error message. So I try to rename it again, and it says that name is already taken. So, when I get that error, I simply refresh, and the file did indeed rename.

 

Also, my second newest hard drive has space left, but will always time out when adding to it. It doesn’t even try or say no space left. I know for sure there is space left. It also gives a connection error also.

One more thing, the Mover, which moves files from the cache drive to the data drives, frequently says there is not enough space left, when there clearly is. I have to run it multiple times to move all the files.

 

I have replaced all the SATA cables, which did not help, so I put the old ones back. I know I need more RAM. My system runs at 80% RAM. I run 13 Docker applications.

 

Are all of these issues related? Could I upgrade or replace certain components?

 

System Overview

unRAID system: unRAID server Pro, version 6.8.2

Model: Custom

Motherboard: Supermicro - X10SL7-F

Processor: Intel® Xeon® CPU E3-1231 v3 @ 3.40GHz

HVM: Enabled

IOMMU: Enabled

Cache: CPU Internal L1 = 256 kB (max. capacity 256 kB)

CPU Internal L2 = 1024 kB (max. capacity 1024 kB)

CPU Internal L3 = 8192 kB (max. capacity 8192 kB)

Memory: 8 GB (max. installable capacity 32 GB)

DIMMA1 = 8192 MB, 1600 MT/s

Network: eth0: 1000Mb/s, full duplex, mtu 1500

eth1: not connected

Kernel: Linux 4.19.98-Unraid x86_64

OpenSSL: 1.1.1d

P + Q algorithm:20533 MB/s + 31562 MB/s

Uptime: 48 days, 11 hours, 12 minutes, 58 seconds

unraid-diagnostics-20200331-2239.zip

[JorgeB]

2 hours ago, Shrimpyaab said:

Every time I reboot, a hard drive says unmountable, and what I do is simply rebuild it, and it works as normal again. I have an unRAID system. Sometimes the hard drive is the same, and sometimes it’s a different one.

Unraid is not detecting a valid partition, most likely a connection problem, does it happen to just a few drives, e.g., are they on the same controller?

 

2 hours ago, Shrimpyaab said:

When I try to rename files, no matter the hard drive, it takes a couple to a few minutes to rename one file. It will then give a connection error message.

This is most likely because you're still using reiserfs on most disks, convert to xfs.

 

2 hours ago, Shrimpyaab said:

Also, my second newest hard drive has space left, but will always time out when adding to it

Is that one using reiserfs? If yes see above.

 

2 hours ago, Shrimpyaab said:

I have replaced all the SATA cables,

At least parity and disk17 need a new SATA cable.

 

You didn't mention this, multiple disks on multiple controllers dropped offline, was this something you did on purpose?

Mar 16 15:32:11 unRAID kernel: mdcmd (1): import 0
Mar 16 15:32:11 unRAID kernel: md: import_slot: 0 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (2): import 1
Mar 16 15:32:11 unRAID kernel: md: import_slot: 1 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (3): import 2 sdk 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K4UH3DN6
Mar 16 15:32:11 unRAID kernel: md: import disk2: (sdk) WDC_WD40EFRX-68N32N0_WD-WCC7K4UH3DN6 size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (4): import 3 sdf 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K5UHE0A9
Mar 16 15:32:11 unRAID kernel: md: import disk3: (sdf) WDC_WD40EFRX-68N32N0_WD-WCC7K5UHE0A9 size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (5): import 4 sdb 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K3THYYLS
Mar 16 15:32:11 unRAID kernel: md: import disk4: (sdb) WDC_WD40EFRX-68N32N0_WD-WCC7K3THYYLS size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (6): import 5
Mar 16 15:32:11 unRAID kernel: md: import_slot: 5 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (7): import 6
Mar 16 15:32:11 unRAID kernel: md: import_slot: 6 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (8): import 7
Mar 16 15:32:11 unRAID kernel: md: import_slot: 7 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (9): import 8 sdl 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K5RYNAFT
Mar 16 15:32:11 unRAID kernel: md: import disk8: (sdl) WDC_WD40EFRX-68N32N0_WD-WCC7K5RYNAFT size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (10): import 9 sdg 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K6SKEZ5C
Mar 16 15:32:11 unRAID kernel: md: import disk9: (sdg) WDC_WD40EFRX-68N32N0_WD-WCC7K6SKEZ5C size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (11): import 10 sde 64 5860522532 0 WDC_WD60EFRX-68L0BN1_WD-WX71DC8F5THF
Mar 16 15:32:11 unRAID kernel: md: import disk10: (sde) WDC_WD60EFRX-68L0BN1_WD-WX71DC8F5THF size: 5860522532
Mar 16 15:32:11 unRAID kernel: mdcmd (12): import 11
Mar 16 15:32:11 unRAID kernel: md: import_slot: 11 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (13): import 12
Mar 16 15:32:11 unRAID kernel: md: import_slot: 12 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (14): import 13
Mar 16 15:32:11 unRAID kernel: md: import_slot: 13 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (15): import 14
Mar 16 15:32:11 unRAID kernel: md: import_slot: 14 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (16): import 15 sdh 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K2PSAY1U
Mar 16 15:32:11 unRAID kernel: md: import disk15: (sdh) WDC_WD40EFRX-68N32N0_WD-WCC7K2PSAY1U size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (17): import 16 sdc 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K2RK8FFU
Mar 16 15:32:11 unRAID kernel: md: import disk16: (sdc) WDC_WD40EFRX-68N32N0_WD-WCC7K2RK8FFU size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (18): import 17
Mar 16 15:32:11 unRAID kernel: md: import_slot: 17 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (19): import 18
Mar 16 15:32:11 unRAID kernel: md: import_slot: 18 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (20): import 19
Mar 16 15:32:11 unRAID kernel: md: import_slot: 19 missing
Mar 16 15:32:11 unRAID kernel: mdcmd (21): import 20 sdj 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K5SV0HP3
Mar 16 15:32:11 unRAID kernel: md: import disk20: (sdj) WDC_WD40EFRX-68N32N0_WD-WCC7K5SV0HP3 size: 3907018532
Mar 16 15:32:11 unRAID kernel: mdcmd (22): import 21 sdi 64 5860522532 0 WDC_WD60EFRX-68L0BN1_WD-WX71DC8AV0CY
Mar 16 15:32:11 unRAID kernel: md: import disk21: (sdi) WDC_WD60EFRX-68L0BN1_WD-WX71DC8AV0CY size: 5860522532
Mar 16 15:32:11 unRAID kernel: mdcmd (23): import 22 sdd 64 3907018532 0 WDC_WD40EFRX-68N32N0_WD-WCC7K0PDA4XL
Mar 16 15:32:11 unRAID kernel: md: import disk22: (sdd) WDC_WD40EFRX-68N32N0_WD-WCC7K0PDA4XL size: 3907018532

 

Also there's an LSI with firmware 20.00.02.00, it should be updated to 20.00.07.00 as all p20 firmwares except the latest one have known issues.

 

 

[AwesomeAustn]

9 hours ago, johnnie.black said:

Unraid is not detecting a valid partition, most likely a connection problem, does it happen to just a few drives, e.g., are they on the same controller?

Yeah, it happens to a few drives. I'm not sure if they are using the same controller or not. I don't know the order of the SATA ports coming off the drives. Two cables are going to a card, and eight more are going to another card. The rest are on the motherboard. The SATA cables that I have split to four for each row of drives.

 

9 hours ago, johnnie.black said:

This is most likely because you're still using reiserfs on most disks, convert to xfs.

Ah ok. Is there a way to convert to xfs without formatting?

 

9 hours ago, johnnie.black said:

Is that one using reiserfs? If yes see above.

Yes, that drive is using reiserfs. I will have to convert it.

 

9 hours ago, johnnie.black said:

At least parity and disk17 need a new SATA cable.

Ok, thanks. I will replace those, which will also replace the cables of disk5 and disk11 because they share the same cable.

 

9 hours ago, johnnie.black said:

You didn't mention this, multiple disks on multiple controllers dropped offline, was this something you did on purpose?

Yeah, I did that on purpose. I stopped the array and replaced all the SATA cables trying to fix the issue without rebooting and causing a data rebuild due to unmountable partition. The issue was not fixed, so I put the original cables back.

 

9 hours ago, johnnie.black said:

Also there's an LSI with firmware 20.00.02.00, it should be updated to 20.00.07.00 as all p20 firmwares except the latest one have known issues.

How would I update it?

 

Also, Parity check is going now, so I'm not going to do anything until it's done. Lately it takes a long time to finish.

Elapsed time: 11 hours, 27 minutes

Estimated finish: 1 day, 2 hours, 38 minutes

Edited April 1, 2020 by Shrimpyaab

[JonathanM]

20 minutes ago, Shrimpyaab said:

Is there a way to convert to xfs without formatting?

No. You must copy the data elsewhere then copy it back after the format.

 

Note that I said copy, not move. Move involves deleting the source files off the ReiserFS disk, which as you found out, can take what seems forever.

 

After the copy is verified complete, reformatting the drive from ReiserFS to XFS will take a minute or so at most, instead of waiting hours for the files to delete, plus you can verify that the copy was successful before you format the source drive.

[AwesomeAustn]

1 hour ago, jonathanm said:

No. You must copy the data elsewhere then copy it back after the format.

What's the best way to copy the data? I have 14 drives that need to be copied. I tried to connect an external hard drive in the past through USB, and I couldn't get past the mounting part. So I ended up copying through my computer. It took like 24 hours for 4 TB.

Edited April 1, 2020 by Shrimpyaab

[trurl]

8 minutes ago, Shrimpyaab said:

I tried to connect an external hard drive in the past through USB, and I couldn't get past the mounting part.

How were you trying to do "the mounting part"? Doesn't look like you have Unassigned Devices plugin installed. That is the way everyone works with external disks these days.

[JonathanM]

It's going to be slow no matter what you do, but the best method is to have an empty drive that you can format XFS as an array drive, then copy from disk to disk inside the array. Whether that's a new drive that you add, or putting the current content of your largest array disk on others in the array and then formatting that one to start the chain, either way works.

[AwesomeAustn]

1 minute ago, trurl said:

How were you trying to do "the mounting part"? Doesn't look like you have Unassigned Devices plugin installed. That is the way everyone works with external disks these days.

I found some mounting commands for Linux to mount it to a media folder. I will get the Unassigned Devices plugin and try again. Thanks.

 

2 minutes ago, jonathanm said:

Whether that's a new drive that you add, or putting the current content of your largest array disk on others in the array and then formatting that one to start the chain, either way works.

I can move some data around to different disks. Thanks.

[JorgeB]

14 hours ago, Shrimpyaab said:

How would I update it?

Go to Bradcom's support site and download the firmware package, instructions are inside, you need to boot with a DOS flash drive, or install the HBA in a Windows PC to do the update.

[AwesomeAustn]

On 4/1/2020 at 11:51 AM, jonathanm said:

Note that I said copy, not move. Move involves deleting the source files off the ReiserFS disk, which as you found out, can take what seems forever.

Thanks for mentioning copying, not moving.

[AwesomeAustn]

So I just found out xfs reserves more space than reiserfs. My copied xfs disk takes up more space than the original reiserfs. How much space does it reserve? I thought I saw 45 GB used when I formatted it. Is it based on a percentage? I will need to move data around since my reiserfs drives are just about filled up.

Edited April 6, 2020 by Shrimpyaab

[AwesomeAustn]

On 4/1/2020 at 1:31 PM, Shrimpyaab said:

On 4/1/2020 at 1:25 PM, trurl said:

How were you trying to do "the mounting part"? Doesn't look like you have Unassigned Devices plugin installed. That is the way everyone works with external disks these days.

I found some mounting commands for Linux to mount it to a media folder. I will get the Unassigned Devices plugin and try again. Thanks.

The Unassigned Devices plugin works. Thank you!

[JonathanM]

7 hours ago, Shrimpyaab said:

I thought I saw 45 GB used when I formatted it. Is it based on a percentage?

Somewhat. It's a base amount plus percent of total. That reserved space is used for housekeeping and file system enhanced features.

 

If you are that close to capacity, you really need to do 1 of 2 things, either jetson files you don't really need, or add capacity by upgrading or adding drives. It's not a good idea to fill an active array to full capacity, for just the reasons you have been experiencing.

 

I personally tend to try to keep total array free space to approximately the size of a single array drive. That may not be necessary for some people, but it's worked well for me. When it drops below that margin, I have enough time to source my next upgrade before I run out of space.

[AwesomeAustn]

7 hours ago, jonathanm said:

It's not a good idea to fill an active array to full capacity, for just the reasons you have been experiencing.

Ah, ok. I will free up some space on those drives. Thanks!

[AwesomeAustn]

On 4/1/2020 at 1:50 AM, johnnie.black said:

On 3/31/2020 at 10:53 PM, Shrimpyaab said:

When I try to rename files, no matter the hard drive, it takes a couple to a few minutes to rename one file. It will then give a connection error message.

This is most likely because you're still using reiserfs on most disks, convert to xfs.

I converted all reiserfs disks to xfs, and this fixed the issue! Thank you!

Edited May 12, 2020 by Shrimpyaab