3 minutes ago, Elmojo said:

The suspense is killing me... lol

For now, we can assign a 6TB disk to each parity slot, then start the array to begin rebuilding both parity.

That should take many hours, then we can resume after. I won't be available till later tomorrow anyway, but others may chime in during my absence.

While rebuilding, you should see lots of writes to the rebuilding disks, lots of reads from all other disks, zeros in the ERROR column for all disks. If there are any problems, post new diagnostics.

Roger that, thanks so much!
I've assigned 2 new 6TB parity disks, and they're (re)building now.
As you say, that will take quite a while. The estimate is about 9hrs.
I'll be tied up until tomorrow afternoon as well, so maybe we can revisit this once I get home and see what's happened after the parity gets settled.
If all goes smoothly, I'll add the 6TB disks to blank slots, and let them start clearing. I have the preclear plugin, would that be worth running on them over night?

Thanks again for your time and patience.

If you preclear before adding Unraid won't have to clear.

But I don't recommend hotplugging while the array is started.

Just a quick update...
The parity build is complete and valid.
The preclear process is ongoing. It's currently on the zeroing step for all 3 6TB drives, at about 25% complete. Since this is step 3 of 6, and it's been running since we spoke last night, I expect it has a while yet to go.
I'll check back in when they're all done.
I assume the next step will be to either add those 3 drives to the array (if I choose to use them to expand the array) or use them to swap out for the 4TB drives one at a time, right?

You don't need a clear drive to replace a drive since the drive will be completely overwritten from the parity calculation. A rebuilt drive is already in-sync with parity since it is the result of parity.

Adding a drive to a new slot requires a clear (all-zero) drive to make it in-sync with parity since all those zeros have no effect on parity. If you don't preclear a drive before adding it to a new slot, Unraid will clear it.

Sometimes people will use preclear as a drive test even if they don't need a clear drive.

Right, which is why I'm preclearing. I'm testing the drives to make sure I don't have any other issues, plus, I'll then have the option to swap them for the 4TB drives or just add them into the array and increase the overall size. I'm leaning towards just adding them for now, then maybe replacing the 4TB drives later with something even larger if I find a good deal. I'm trying to stick with SAS enterprise drives, both for the native interface, and also for the durability.

1 minute ago, Elmojo said:

maybe replacing the 4TB drives later with something even larger

But not larger than either parity disk 😉

19 minutes ago, trurl said:

But not larger than either parity disk 😉

Ugh...right. I need to remember to buy at least 3 disks, so I can replace both parity drives, and then start upgrading the data drives. I swear, it never ends! lol
Are there any large cap (6TB+) enterprise SAS drives that you're aware of that won't break the bank? I've always had very good success with the DELL/HGST drives, but I'm not sure if those are even still around, and not in larger sizes.

Never used SAS myself.

35 minutes ago, trurl said:

Never used SAS myself.

Really? I love it. It's like the more reliable, "serious" version of SATA.
I've never had a drive drop out due to a sketchy cable, for one thing. I absolutely cannot say that about SATA.
Also, I think the theoretical throughput limit is supposed to be higher for SAS (12Gbps), although in practice you're not likely to saturate either one on an unRAID array, so that hardly matters.

Parity rebuild and disk preclearing is all complete.
At this point, I have to decide if I want to exchange some of the 6TB drives for the 4TB ones, or just add the 6TB drives to the array.
One concern is that a couple of my drives are running rather hot, like 50-52c. I know these SAS drives are rated to run hotter than normal desktop drives, but I think they're only rated for 55c before it becomes an issue. I'm not sure what I should do. The server is in a dedicated, climate-controlled room. The AC is literally blowing directly into the intake of the server. The ambient room temp is maintained at 24c (75F) in order to keep my electric bill from going nuts. :) I can't imagine that Dell build a server that you can't fully populate, so I don't really know what else is going on.
The internal fans are running at about 20%, with the minimum set at 20%, so the machine itself doesn't think there's any need for additional cooling.

Any ideas?

Is there airflow across the drives?

9 hours ago, Elmojo said:

The AC is literally blowing directly into the intake of the server.

8 minutes ago, trurl said:

Is there airflow across the drives?

I assume so. I mean, this is an OEM server chassis, with no mods. I have air blanks installed in all unpopulated bays, per Dell spec. I have a floorstanding AC unit sitting sitting in front of the rack that the server is on, and the cold air from the AC is blowing directly into the front of the server intake, where the drives sit. I can't imagine how it could get much better. 🤷‍♂️

Exhaust?

12 minutes ago, trurl said:

Exhaust?

Just out the back of the machine, and into the room. It has a 12' ceiling, and there's a grille for air to exchange with the space outside the server room. That room stays 24c all the time, due to having it's own climate control.
The server itself is on a rack, not inside. I have a short rolling rack with a disk shelf and UPS in it, and the tower sits on top of that. Oddly, most of the drives in the system hover around the low to mid 40s C. A few are upper 40s, and 1 or 2 are over 50c.
I don't have a good way of identifying which drives are in each physical bay, so I can't say if it's the upper drives that are running hot or lower ones or whatever.

10 hours ago, Elmojo said:

fans are running at about 20%

Can you control this?

32 minutes ago, trurl said:

fans are running at about 20%

Can you control this?

Not directly. I can set the minimum, but even then, it kinda does what it wants based on the load and CPU temps. It doesn't seem to be monitoring the HDD temps, which I find to be a bit odd.
I tried setting the minimum to 25%, but it didn't rise above 20%, unless the CPU temp rose for some reason.

There are some fan control plugins

1 hour ago, trurl said:

There are some fan control plugins

They don't work on Dell servers, unfortunately. I had an IPMI script that I'd use on my T630, but that doesn't work on the T640, since Dell disabled direct access to the IPMI interface on this gen of the platform. :/