CSE-846 and Cooling


Endy

Recommended Posts

This is going to be a bit of a lengthy setup with questions at the end.

 

I ran out of room in my old case and went with a Supermicro 846 with the BPN-SAS2-846EL1. I had also been planning an upgrade for awhile and did that at the same time. I went with a Ryzen 7 3800X on the AsRock Taichi x570.

 

My house does not have a great place to keep servers and networking equipment so for now I am making use of a small hallway closet. Eventually I need to add some type of venting but I have made no decisions on that as of yet. Keeping the door cracked open is working just fine.

 

The 846 just barely fits inside this closet. I ended up making a little cart and I have the server sitting on it's side. The old server sat directly on the floor (tile) underneath the bottom shelf and it seemed to trap a lot of heat. The new server actually sits at the front of the closet before where the shelf starts and the whole closet actually seems to be cooler.

 

Noise was a bit of an issue so I have taken steps to quiet the server down. I took out the power supplies that came with it and replaced them with one of the SQ power supplies. I replaced the 2 80mm fans at the back of the case with 2 Noctua NF-R8 redux-1800 PWM. Because the Noctua are thinner than the fans they replaced there were some large gaps so I 3D printed some spacers to make sure the air flow was exhausting out the back. This picture shows 1 fan with the gap and the other with the spacer.

 

20191205_193825.thumb.jpg.1fbc1c865b194fde255f5d1940cf5b05.jpg

 

I also took out the fan wall with the 3 80mm fans and 3D printed a new fan wall for 3 120mm fans and I used Arctic P12 PWM PST. (I had to make it in 3 pieces because of the size of my working 3D printer.)

 

20191208_170801.thumb.jpg.55670c0eac3cff26bf4e72c28f012225.jpg

 

As I've seen suggested elsewhere, I also sealed up the vent holes on the sides of the case so that air will be pulled through the front across the hard drives. I also taped over the inside vent holes of the drive trays that do not currently have hard drives mounted.

 

All of this has quieted the server down. In fact, I think it's quieter than my old server. However, I am now worried about temps. Initially I noticed my hard drive temps getting close to dangerous levels. This was during my initial testing when I was leaving the lid off the case. That meant the air wasn't getting pulled from the front of the case but rather from the huge gap that was now open between the backplane and the fan wall. Put the lid back on and temps were a little bit better, but still not great. At this point I turned the fan wall fans and the 2 rear exhaust fans to run at max speed in the bios and again, temps got a little bit better. Funny thing is, the hard drive temps dropped even lower once I put the server in the closet. Currently they mostly stay in the 20's and 30's which is good, but a few of them do seem to go over 40 degrees at times.

 

I also have an M.2 NVME drive and 2 SSD's. The NVME drive is mounted in a slot between the CPU socket and the first PCI-E slot. For the SSD's I ended up making a mounting bracket that uses the mounting points of the case as seen here:

 

20191211_163306.thumb.jpg.674a9f1e73bdeadf0e6e2a6d5e53d39e.jpg

 

All 3 of these drives see temperature spikes. The NVME can get up around 50 degrees and the 2 SSD's can see around 46 degrees. 

 

I have only just figured out how to get the cpu and mb temps to show up in Unraid. Looks like the cpu idles around 60 degrees. (Technically 8% load, but I do not see the Ryzen drop below that even though the Intel I7 7700 I had in my last server would.) Currently using the stock AMD heatsink and fan.

 

Here's a picture that shows the overall layout as it is right now:

 

20191212_110837.thumb.jpg.752f19c4c2c3c05552ffb76b36caa4f1.jpg

 

One thing that I am wondering about is if I should switch to a heatsink and fan for the cpu that moves air in the same direction as the other fans? I'm thinking that maybe the current heatsink and fan are impeding the flow a bit. Maybe creating a bit of a whirlwind and keeping hot air inside the case and keeping the cpu temp a little hotter than it could be? Maybe also blowing hot air across the NVME which is why it gets so hot?

 

Because the case sits on it's side, the 2 SSD's are at the top of the case and so maybe they are getting hot because heat rises and there's not enough airflow exiting the case pulling hot air away from them? I have thought that I could make a new mounting bracket that rotates them 90 degrees and make it possible to hold 4 SSD's and have something like a 50mm fan that blows air across them.

 

Should I also be adding fans to the front of the case as others have done to force more air inside across the hard drives?

 

Bonus questions... in the System Temp plugin I'm using k10Temp - Tctl for CPU temp and nct6798 - SYSTIN for motherboard temp. Are these the correct sensors for those? I am also trying to figure out which sensor to use for the Fan Auto Control plugin. There's nct6798 pwm1-7. I am assuming that I would want to find the one that controls the fans on the fan wall, but not sure which sensor goes to which motherboard pwm fan header.

  • Thanks 1
Link to comment

 

 

5 hours ago, Endy said:

Funny thing is, the hard drive temps dropped even lower once I put the server in the closet. Currently they mostly stay in the 20's and 30's which is good, but a few of them do seem to go over 40 degrees at times.

When I first meet rack case, I also face similer problem. Due to its depth ( 670mm ), the airflow / cooling behavior have big change if compare with non-rack case. My 3U case also use three Arctic F12, so almost same spec. as Arctic P12-PST. In simple say I haven't fix cooling problem, but what I don't expect / believe is why if put the rack in closet have above different. In fact, I believe you will got cooling problem on harddisk especially if disk fully occupied.

 

5 hours ago, Endy said:

Bonus questions... in the System Temp plugin I'm using k10Temp - Tctl for CPU temp and nct6798 - SYSTIN for motherboard temp. Are these the correct sensors for those?

My org. hardware in 3U was Ryzen 1700, but I haven't install plugin to monitoring CPU / System temp, so no figure for discuss. But I also googling to see what offset of Ryzen 3800X shoud take, but no real answer.

 

https://community.amd.com/community/gaming/blog/2017/03/13/amd-ryzen-community-update

Product Name     True Junction Temp (Example)   tCTL Offset for Fan Policy  Temp Reported by tCTL

AMD Ryzen™ 7 1800X    38°C    20°C    58°C

AMD Ryzen™ 7 1700X    38°C    20°C    58°C

AMD Ryzen™ 7 1700    38°C    0°C    38°C

 

I also exam that on a tower case with TR 1920x ( offset -29°C ) and it show valid.

TR-29.PNG.955567b0356a2e59319ba675f4735a9e.PNG

 

 

Someone say Ryzen 3000 no offset, this really make me surprise .....

 

https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.4-Hwmon-Zen-2-Thermal

 

https://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git/commit/?h=hwmon-next&id=12163cfbfc0f804cc7d27bc20e8d266ce7459260

Unlike previous Ryzen/Threadripper, Ryzen gen 3 processors do not need temperature offsets anymore. This has been reported in the press and verified on my Ryzen 3700X by checking that the idle temperature reported by k10temp is matching the temperature reported by the firmware.

 

Then what actual meaning of Tctl ? ( Does no K10temp - Tdie for Ryzen 3800X show in plugin ???????? )

 

 

Nice POST about 846 case and DIY 3D print !!!!!

Edited by Benson
Link to comment
40 minutes ago, Benson said:

but what I don't expect / believe is why if put the rack in closet have above different. 

The only thing I can think of is that because the space is so limited, it's increasing air pressure.

 

40 minutes ago, Benson said:

My org. hardware in 3U was Ryzen 1700, but I haven't install plugin to monitoring CPU / System temp, so no figure for discuss. But I also googling to see what offset of Ryzen 3800X shoud take, but no real answer.

 

https://community.amd.com/community/gaming/blog/2017/03/13/amd-ryzen-community-update

Product Name     True Junction Temp (Example)   tCTL Offset for Fan Policy  Temp Reported by tCTL

AMD Ryzen™ 7 1800X    38°C    20°C    58°C

AMD Ryzen™ 7 1700X    38°C    20°C    58°C

AMD Ryzen™ 7 1700    38°C    0°C    38°C

 

I also exam that on a tower case with TR 1920x ( offset -29°C ) and it show valid.

TR-29.PNG.955567b0356a2e59319ba675f4735a9e.PNG

 

 

Someone say Ryzen 3000 no offset, this really make me surprise .....

 

https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.4-Hwmon-Zen-2-Thermal

 

https://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git/commit/?h=hwmon-next&id=12163cfbfc0f804cc7d27bc20e8d266ce7459260

Unlike previous Ryzen/Threadripper, Ryzen gen 3 processors do not need temperature offsets anymore. This has been reported in the press and verified on my Ryzen 3700X by checking that the idle temperature reported by k10temp is matching the temperature reported by the firmware.

 

Then what actual meaning of Tctl ? ( Does no K10temp - Tdie for Ryzen 3800X show in plugin ???????? )

I had read somewhere that Tctl was useful for fan control and that's why I was thinking I should use it, but now that I am thinking about it, Unraid isn't controlling the cpu fan, so if I just want to see the cpu temp I should probably be using Tdie. Chalk this one up to me overthinking things.

 

40 minutes ago, Benson said:

 

Nice POST about 846 case and DIY 3D print !!!!!

Thank you. :) Except for the temperature issues, it's been turning out better than I expected.

Link to comment

Case share ...

 

My rack operate about half year with more and more HDDs add, one day when last slots occupied , my HBA just died on next day. The reason was airflow restrict too much from disk intake area and cause so weak outake ( I seal all other intake ). Finally I take an unusual design, I change the airflow from rear to front.

So, different from past, all area suffer temperature problem and now only disks got problem ( that means disk temp. even worst )

 

How about you feel the temperature at 846 rear area ? ( This is importance sign )

If Tctl means Tdie and in 60°C idle, I will assume 846 rear area ~50°C, this should be dangerous.

 

Sound good if you found some good design and share.

Edited by Benson
Link to comment
13 hours ago, Benson said:

Case share ...

 

My rack operate about half year with more and more HDDs add, one day when last slots occupied , my HBA just died on next day. The reason was airflow restrict too much from disk intake area and cause so weak outake ( I seal all other intake ). Finally I take an unusual design, I change the airflow from rear to front.

So, different from past, all area suffer temperature problem and now only disks got problem ( that means disk temp. even worst )

I don't think I'll be trying that. I do think that I should probably add fans to the front of the case to see about getting more air moving across the hard drives.

 

13 hours ago, Benson said:

How about you feel the temperature at 846 rear area ? ( This is importance sign )

If Tctl means Tdie and in 60°C idle, I will assume 846 rear area ~50°C, this should be dangerous.

I measured the air coming out of the back of the case as best as I could and it's showing around 34 degrees. 

Link to comment
  • 2 months later...
  • 7 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.