Sanity check - 24 drive bays

[demonmaestro]

8 hours ago, uaeproz said:

 

at idle, its reasonable but when at load, man this case screams! I'll post a video on YouTube if you'd like

sure

[uaeproz]

6 hours ago, demonmaestro said:

sure

 

E0C611E9-8573-45C0-AB15-430B595DCC78.MOV

0141C9FC-DE1A-4D4F-9F2A-2F3EC8D7DF1F.MOV

[jon928]

I know I'm late to the party here.  But if your main goal was the ability to  4k transcode multiple streams,  wouldn't it have made more economic sense to move everyone who was using your plex server to a streaming device that could direct play your media content?  Like a nvidia shield TV or chromecast ultra. 

 

Regardless I'm jealous of your new server ?

[uaeproz]

1 hour ago, jon928 said:

I know I'm late to the party here.  But if your main goal was the ability to  4k transcode multiple streams,  wouldn't it have made more economic sense to move everyone who was using your plex server to a streaming device that could direct play your media content?  Like a nvidia shield TV or chromecast ultra. 

 

Regardless I'm jealous of your new server ?

That might be an option if it was 100% dedicated to plex. 

[uaeproz]

1 hour ago, jon928 said:

I know I'm late to the party here.  But if your main goal was the ability to  4k transcode multiple streams,  wouldn't it have made more economic sense to move everyone who was using your plex server to a streaming device that could direct play your media content?  Like a nvidia shield TV or chromecast ultra. 

 

Regardless I'm jealous of your new server ?

Besides, Shield TV and Chromecast has storage limitations and they don't specify number of streams

[jon928]

Fair enough.  Did you ever mention what your old hardware comprised of?  What did you end up doing with it?

[demonmaestro]

Well do have to say it seems quieter than I expected.

[jon928]

On 3/30/2018 at 6:23 PM, uaeproz said:

Besides, Shield TV and Chromecast has storage limitations and they don't specify number of streams

 

I think you misunderstood what I meant.  The nVidia Shield tv can direct stream most x264 encoded content so no transcoding is required from your server.  I've tested the Shield with most of my 4k HEVC content and it also direct plays that as well.  So no transcoding is required.  Since it direct plays the content on your servers, it drastically lowers the cpu requirements on your server.  I've tested with a Fire TV 4k and Chromecast Ultra and it does NOT direct play the HEVC 4k content I have.  My friends Apple TV 4k I believe is 1 or 2 updates away from direct playing 4k HEVC.

[SimonAG]

Not sure if someone already mentioned but you can do GPU transcoding on windows with madvr renderer. On windows the best splitter and filters is lav filters. If you want to get the most out of a CPU I would recommend lav filters.

Sent from my SHIELD Tablet using Tapatalk

[kuakman]

On 3/7/2018 at 2:45 PM, uaeproz said:

@SSD

 

here is the update 

 

 

 

Hello SSD,

 

Got a quick question for you since you seem very experienced building servers.

To give a little bit of background for my question, I started building my new workstation rig which it has similar components to your server, but instead of mounting the X11DAI-N in a server rack, I mounted it in a Tower Chassis. I'm also a beginner building workstations or dealing with pc parts, so that it's stressing me out.

I attached 2 pictures for reference.

 

So I built and installed the PHM, SNK-P0070APS4 with a Xeon Silver 4114 (10-cores) following the instructions and I quickly realized that there was something wrong with the direction of the airflow I set up. Both air coolers were pointing to opposite directions and I thought I made a mistake.

I read out there that the ideal airflow is to make them go in the same direction (Not sure if this is actually true), the intake air from the front and exhaust the air to the rear of the chassis. So I thought "- I have to disassemble one of the PHM." So I did.

 

But, I cannot find anywhere in the internet, where I can buy a replacement of the "thermal pad" that the supermicro heatsink includes pre applied

Do you know where I could possible buy it?

Are there any other alternatives, like applying thermal paste (Artict Silver 5) instead of buying the pad? I had some of those.

 

I started to get a little bit desperate because I cannot finish my build.

Also, I unfortunately disassembled and cleaned the thermal paste before I looked at this post and some of you pictures.

I'm afraid I took it out with no reason now.

 

Well, that's the situation I'm dealing with. Hopefully I would not burn the processors.

 

Thank you! I will appreciate any help or suggestions about this.

 

 

[SSD]

18 hours ago, kuakman said:

 

Hello SSD,

 

If you want a person to get a notification when you mention their name, make sure to put an "@" in front. The name should then get highlighted like @SSD. If it looks like SSD or @SSD it is just text and will not contact anyone. Just FYI.

 

18 hours ago, kuakman said:

 

Got a quick question for you since you seem very experienced building servers.

To give a little bit of background for my question, I started building my new workstation rig which it has similar components to your server, but instead of mounting the X11DAI-N in a server rack, I mounted it in a Tower Chassis. I'm also a beginner building workstations or dealing with pc parts, so that it's stressing me out.

I attached 2 pictures for reference.

 

All my servers are / have been towers. Never done a rack mount.

 

18 hours ago, kuakman said:

 

So I built and installed the PHM, SNK-P0070APS4 with a Xeon Silver 4114 (10-cores) following the instructions and I quickly realized that there was something wrong with the direction of the airflow I set up. Both air coolers were pointing to opposite directions and I thought I made a mistake.

I read out there that the ideal airflow is to make them go in the same direction (Not sure if this is actually true), the intake air from the front and exhaust the air to the rear of the chassis. So I thought "- I have to disassemble one of the PHM." So I did.

 

You really want to create airflow that brings in air low and front and pushes air up and back. Not always possible but that is ideal. I like to push CPU air directly to an exhaust fan if I can. But I have never had dual CPUs. See my comments below.

 

18 hours ago, kuakman said:

 

But, I cannot find anywhere in the internet, where I can buy a replacement of the "thermal pad" that the supermicro heatsink includes pre applied

Do you know where I could possible buy it?

Are there any other alternatives, like applying thermal paste (Artict Silver 5) instead of buying the pad? I had some of those.

 

I always replace the thermal pad with Arctic Silver 5. There are other brands that you can investigate, but I have had good luck with this one and tend to stick with it.

 

The trouble with the pads is that they are made of wax, and if you use one, the wax melts. This can make is difficult to fully remove the compound. But I'd still try and replace with a thermal compound. I use the arctic 2 step cleaner and some of THESE to clean the old compound. They are the best I've found. They are a little pricey so I usually get CPU and HSF as clean as I can with a qtip, and then use one of these with the arctic cleaner. It is amazing how much compound it will pick up after thorough cleaning with qtip.

 

18 hours ago, kuakman said:

 

I started to get a little bit desperate because I cannot finish my build.

Also, I unfortunately disassembled and cleaned the thermal paste before I looked at this post and some of you pictures.

I'm afraid I took it out with no reason now.

 

Well, that's the situation I'm dealing with. Hopefully I would not burn the processors.

 

Thank you! I will appreciate any help or suggestions about this.

 

 

 

It's a little hard to see where the air intake is. The CPU above left looks good. (Assuming that fan to the left of it is exhausting air).  It is very similar to how mine is setup. The CPU to the right that is blowing air to the front of the case is not as good. I'd likely turn it facing up (towards top of case) if that is an option. And install an exhaust fan on the case top. Air should come in from the front and maybe bottom. I think it will cool well. 

 

(I'm assuming your fans are mounted the right way - I am looking at the little white labels on top showing air flow direction).

 

18 hours ago, kuakman said:

 

Good luck!

 

[kuakman]

 

28 minutes ago, SSD said:

 

If you want a person to get a notification when you mention their name, make sure to put an "@" in front. The name should then get highlighted like @SSD. If it looks like SSD or @SSD it is just text and will not contact anyone. Just FYI.

Sorry about that.

 

28 minutes ago, SSD said:

I always replace the thermal pad with Arctic Silver 5. There are other brands that you can investigate, but I have had good luck with this one and tend to stick with it.

 

The trouble with the pads is that they are made of wax, and if you use one, the wax melts. This can make is difficult to fully remove the compound. But I'd still try and replace with a thermal compound. I use the arctic 2 step cleaner and some of THESE to clean the old compound. They are the best I've found. They are a little pricey so I usually get CPU and HSF as clean as I can with a qtip, and then use one of these with the arctic cleaner. It is amazing how much compound it will pick up after thorough cleaning with qtip.

 

It's a little hard to see where the air intake is. The CPU above left looks good. (Assuming that fan to the left of it is exhausting air).  It is very similar to how mine is setup. The CPU to the right that is blowing air to the front of the case is not as good. I'd likely turn it facing up (towards top of case) if that is an option. And install an exhaust fan on the case top. Air should come in from the front and maybe bottom. I think it will cool well.

 

(I'm assuming your fans are mounted the right way - I am looking at the little white labels on top showing air flow direction).

 

Here I attached the same picture, with some arrows showing how I had it before I removed the one in the right. So yea, the left one looks good to me, the one I need to rotate 180 degrees is the right one to point to the same direction as the left one, so all the air should be exhausted to the rear of the chassis. In the front there are 3 rings that should get the intake.

 

I will try using Artic Silver 5 instead of the pad, I just got a message from supermicro customer support and they are not selling the thermal pad separately.
So I will try to spread the paste to cover most of the cpu area. Agreed about the wax, it took me a while to fully clean it and remove the wax, it's gets really messy.

Especially I'm trying to avoid touching the bottom part of cpu, the golden contacts, really hard to handle.

 

Alright, thanks much! I really appreciate your advice. I will let you know how it goes. This week the GPU should be arriving and I should be all set.

[SSD]

27 minutes ago, kuakman said:

Here I attached the same picture, with some arrows showing how I had it before I removed the one in the right. So yea, the left one looks good to me, the one I need to rotate 180 degrees is the right one to point to the same direction as the left one, so all the air should be exhausted to the rear of the chassis. In the front there are 3 rings that should get the intake.

 

I will try using Artic Silver 5 instead of the pad, I just got a message from supermicro customer support and they are not selling the thermal pad separately.
So I will try to spread the paste to cover most of the cpu area. Agreed about the wax, it took me a while to fully clean it and remove the wax, it's gets really messy.

Especially I'm trying to avoid touching the bottom part of cpu, the golden contacts, really hard to handle.

 

Alright, thanks much! I really appreciate your advice. I will let you know how it goes. This week the GPU should be arriving and I should be all set.

 

I understand the exhaust part. It's your air intake I don't understand. There are no fans in the upper back of the case I can see. Cool air will NOT just waft into the case from above without a fan driving it down. And you are working against the natural tendency of warm air rising. Your layout is much more likely to result in hot air accumulating at the top of the case with nothing much to force it out. And that hot air will recirculate and get hotter, reducing your cooling. 

 

How is the exhaust from the CPU on the right getting out of the case? Looks like it is blowing into the back of your drives?? It will tend to rise to the top of the case, as mentioned, will get recirculated back into the CPU coolers.

 

Take a look at this https://www.howtogeek.com/303078/how-to-manage-your-pcs-fans-for-optimal-airflow-and-cooling/ it has a lot of info on case cooling, positive vs negative pressure, etc.

 

Here is what you are trying to achieve. Cool air coming in from low front (could also come in from bottom depending on the case), and hot air going out the back and top.

 

 

Perfection is impossible, If you turn the fan on the CPU on the right to point up, and added an exhaust fan up there, while having a couple of intake fans on the front to get cool air into the case, I think your cooling would be much more effective.

 

#ssdindex - case cooling

[kuakman]

Quote

@SSD

Perfection is impossible, If you turn the fan on the CPU on the right to point up, and added an exhaust fan up there, while having a couple of intake fans on the front to get cool air into the case, I think your cooling would be much more effective.

 

Unfortunately, the heatsink + aircooler (PHM) doesn't allow me to point it "up", it can be either left or right, but what I could do is following your suggestion of installing a couple of ring fans at the top to exhaust the warm air from there. That will do it. It's like you said, perfection is impossible but that should be the best I can with the components I have, to drive the hot air out of the chassis.

 

The good thing is that the chassis is a Tempered Glass 91 RGB from ThermalTake, full modular. It's a gigantic freaking case. You can install an insane amount of chassis fans, top, bottom, front and back.

http://ttpremium.com/product/thermaltake-view-91-tempered-glass-rgb-edition-super-tower-chassis/

 

Again, thanks for the recommendations! It's helping a lot.

Right now I will try to solve the thermal paste issue first and face the heatsink the right direction. That's the step number 1.

Then later, I will start thinking of getting more fans for the top to improve airflow efficiency following the diagram you posted.

Edited April 16, 2018 by kuakman

[uaeproz]

[uaeproz]

Is there any way to use HW transcoding (GPU) in Docker not VM? How?

[ken-ji]

Intel iGPU definitely (especially if its Kaby Lake or older - the in-kernel driver for Coffee Lake is alpha status accdg to the driver devs)

- basically you load the i915 module and have it initialize the iGPU, then see

though some people are having diffiiculty there.

Nvidia/AMD GPU - probably not unless the in-kernel noveau/fglrx(?) drivers are working for the HW accelerated transcoding - since its very hard/next to impossible to install the proprietary binary drivers

[uaeproz]

It looks like my only way is GPU passthrough the VM. Then setup the Plex in VM. 

 

 

[uaeproz]

Can’t wait to see this card capability for transcoding 

[uaeproz]

Moving Ton of data is pain in a$$

[uaeproz]

I need your advice of the Memory configuration, type and size. My intended purpose is to use the VM to run plex in which, it will utilize the nvidia GPU (GP100) for transcoding and I do have a lot of clients going to transcode. Also, I will be using it for gaming through the VM.

 

That being said and if money is no object, what should I use? RDIMM, LRDIMM, NVDIMM? what capacity?

 

Thanks,

 

Abdulla

[uaeproz]

3 hours ago, uaeproz said:

I need your advice of the Memory configuration, type and size. My intended purpose is to use the VM to run plex in which, it will utilize the nvidia GPU (GP100) for transcoding and I do have a lot of clients going to transcode. Also, I will be using it for gaming through the VM.

 

That being said and if money is no object, what should I use? RDIMM, LRDIMM, NVDIMM? what capacity?

 

Thanks,

 

Abdulla

@SSD @ken-ji @demonmaestro @trurl @johnnie.black 

[SSD]

On 6/11/2018 at 5:02 PM, uaeproz said:

@SSD @ken-ji @demonmaestro @trurl @johnnie.black 

 

The memory type will be determined by what your motherboard supports.

 

Transcoding to RAM can offer some performance advantage, which might up your memory needs, but I am really unable to give you guidance on how much is enough. Generally I'd say 32G is ok for a typical server, 64G for a higher power server, and 128G limit for most non-Xeon setups. Not enough RAM can certainly bottleneck a system, but extra RAM (or tweaking the RAM specs) does not correlate with significant performance improvement. Only some experimentation will tell for sure.

 

I am still a bit unclear how you are planning to use the GP100 for transcoding. You say you plan to run in a VM - assuming that means Windows? And that Windows Plex will support the HW decoding with that graphics card. Is that correct? I use the Plex Linux (docker), and AFAIK, only the iGPU decoding is supported for 4k transcode. That is what I mentioned much earlier in this thread. But very interested in your results with the Windows setup. I could reconsider if decoding with an add-on video card is supported.

[jwegman]

@uaeprozI realized that I'm late to the party, however had it been considered that instead of real-time transcoding for each client stream request (not capable of Direct Play), that instead you one-time pre-transcode (perhaps using Plex Optimized Versions feature) for those users?  I suspect that attempting to real-time transcode such large/high bit rate h265 content to as many users as you anticipate may end in tears (and frustration) regardless of how much money/hardware you throw at it.

 

https://support.plex.tv/articles/214079318-media-optimizer-overview/

[uaeproz]

On 7/5/2018 at 5:34 PM, jwegman said:

@uaeprozI realized that I'm late to the party, however had it been considered that instead of real-time transcoding for each client stream request (not capable of Direct Play), that instead you one-time pre-transcode (perhaps using Plex Optimized Versions feature) for those users?  I suspect that attempting to real-time transcode such large/high bit rate h265 content to as many users as you anticipate may end in tears (and frustration) regardless of how much money/hardware you throw at it.

 

https://support.plex.tv/articles/214079318-media-optimizer-overview/

 

On 6/14/2018 at 6:00 PM, SSD said:

 

The memory type will be determined by what your motherboard supports.

 

Transcoding to RAM can offer some performance advantage, which might up your memory needs, but I am really unable to give you guidance on how much is enough. Generally I'd say 32G is ok for a typical server, 64G for a higher power server, and 128G limit for most non-Xeon setups. Not enough RAM can certainly bottleneck a system, but extra RAM (or tweaking the RAM specs) does not correlate with significant performance improvement. Only some experimentation will tell for sure.

 

I am still a bit unclear how you are planning to use the GP100 for transcoding. You say you plan to run in a VM - assuming that means Windows? And that Windows Plex will support the HW decoding with that graphics card. Is that correct? I use the Plex Linux (docker), and AFAIK, only the iGPU decoding is supported for 4k transcode. That is what I mentioned much earlier in this thread. But very interested in your results with the Windows setup. I could reconsider if decoding with an add-on video card is supported.

 

@SSD@jwegman

 

Apologies for my late reply. Been little bit busy trying to figure out few stuff and I had great help from Ed aka Spaceinvader One.

 

@SSD I managed to passthrough the GP100 to windows 10 VM and yes, plex is supporting HW transcoding using the GP100. You are right, GP100 is not supporting 4k transcode unfortunately. However, I'm getting some transcoding done of 4k movies but CPU cores reaches full utilization when that happens.

 

@jwegman

Yes, that is the optimal solution but unfortunately plex optimizer cannot do batch optimization. This is really lengthy specially if you have thousands of video files. I'm using another software to do that right now.