EKWB has come to the party and has sent replacement mounting hardware, so should be able to run this up at 100% in about a weeks time.

From there the migration from my existing baremetal install to unraid as a daily pc begins

 

I'm also super happy with the radiator support bracket I've designed and printed, after 3-4 iterations it is a perfect fit.

At a later stage I'll reprint in a more heat tolerant material, but PLA is fine for now.

 

Now onto those other designs I need to finish...

Edited July 17, 20214 yr by tjb_altf4

Received the replacement parts from EKWB (super fast turn around) and was able to finish the install, did some stress testing and was happy with the results.

16C/32T @ 100% for 15min and was silent nearly the whole time, with only a slight ramp up of fans towards the end... very happy!

 

 

The carbon build now split out and continued below, updates to fortytwo hopefully coming in the next week or so...

 

Since upgrading my workstation (carbon), Fusion360 has not wanted to start and freezes on start up, no amount of reinstalls and fixes has resolved the seemingly common issue, so this has stopped progress on custom drive mounting solution

 

However, I was still able to upgrade a little, my 2TB WD blue which died after a long arduous history, was finally pulled out, replaced with another spare WD Red 4TB from my old array.

So now the scratch pool is a 3 disk raid0 pool (12TB), which amusingly is the size of my first Unraid array I built ~5 years ago.

 

I have also enjoyed mucking around with Chia, so I've expanded into a dedicated pool of 2x 18TB.

A bonus was these particular shucks didn't have PWDIS, which makes life easier. with no power mods needed.

If multi-pools were here I would have made it a seperate pool, I'll look to convert when that option becomes available.

 

Those of you that paid attention will note I now have 16 drives in a 15 drive chassis... well one drive got shoved down the side where the gap for the IO panel and cables is haha

 

This brings the server up to ~150TB across array, pools and UD.

I plan to move towards 0.25PB, once the new drive mounting arrangement is sorted and in place.

After a long wait for some components, and a few hiccups with my 3d printer, v1 disk caddy is nearly ready.

To recap, this will take me from a 15 drive capacity, to 28 drives, with only slightly more space usage. 

This new design should also be quieter than the standard rosewill setup, as I am moving from 120mm to 140mm fans, which while running slower can produce more pressure and airflow... lets see how that goes!

 

I ended up making a few design compromises for simplicities sake as I was crushed for time (and have storage to deploy!), but will address these shortcomings in a v2 design.

So I needed a good base to move forward with, so I've got my existing drives moved into the new caddy system... success! 

 

Post fitment, there are definitely a number of things to fix in a v2 design, but my god it is so much quieter... and I've got 13 extra bays to work with!  WIN-WIN.

For all headaches, this has been a great upgrade so far.

Spaghetti test run:

Added another H310 HBA via M.2 adaptor, so now I have another 8 drives hooked up... thinking about moving all this to another case (JBOD build), wiring is getting out of control and hard to manage nicely.

Also concerned on temp management as we're going into summer and I do like to lean on the 1950x from time to time... something for me to monitor over time.

 

Anyway, this is good enough for now, I've got the expansion I wanted, next stop is making some custom power cables to help fix up cable management.

 

 

 

New disks are setup into pools now and ready for use, which takes total storage in the server to just over 200TB

everything cool, but the cabling What are the avg hdd temps?

2 hours ago, doesntaffect said:

everything cool, but the cabling What are the avg hdd temps?

haha yeah, next design will hide the cables all things going to plan

 

The array is the first row of disks, averaging 28.5C right now.

Second row are mostly raid0 pools, they have been under sustained R/W for a few hours and currently average between 33-36C.

The 140mm fans are running at just under 1000rpm, so the setup is quiet enough to sit behind my desk and not bother me (I work from home).

 

note: ambient temp is 20C

Edited November 4, 20214 yr by tjb_altf4

Doing a bit a maintenance for the last few days and I think I'm done.

 

My cache is now utilising the evo plus nvme drives I installed earlier in the year, and so far one of the old 960 pros being utilised for a system pool.  I managed to do this upgrade as a live migration, but it is not for the faint hearted! You end up in a quasi-land working half in unraid gui, half in cmd line with btrfs directly... not ideal.  The system pool that was created is so far is being utilised for the old system share, so docker.img and libvirt.img, but I've also installed the swapfile plugin, so now I have a fast swap available. 

 

Why do I need a swapfile?  Well running lots of chia forks, transcoding, media apps and vms simultaneously has caught me out with OOM from time to time when usage spikes, so I've added a safety net above the 64GB of RAM I have installed which so far is working well, particularly with the chia fork farming. 

 

System seems much snappier now, particularly on the dockers page when starting and stopping containers, so the cumulation of changes has made a positive impact which is a great outcome.

Designed and 3d printed a bracket this morning so I could mount the sas expander I bought earlier in the year.

This allowed me to repurpose the second HBA, while adding 4 more drives... now my 28 bay enclosure is maxed out and I'm sitting on around 350TB

 

 

 

Which 3D printer are you using and how long did it take to design the mount?

 

 

5 hours ago, doesntaffect said:

Which 3D printer are you using and how long did it take to design the mount?

Prusa MK3S, but there are many great options out there.

It's a simple design, it took me about 30min to measure and finalise in CAD, while the print itself was about an hour as there is purposely not a lot of print volume.

On 9/27/2021 at 12:42 PM, tjb_altf4 said:

So I needed a good base to move forward with, so I've got my existing drives moved into the new caddy system... success! 

 

Post fitment, there are definitely a number of things to fix in a v2 design, but my god it is so much quieter... and I've got 13 extra bays to work with!  WIN-WIN.

For all headaches, this has been a great upgrade so far.

Spaghetti test run:

 

wow, this is really an elegant solution and just what I have been looking for. Any chance you can share the STLs or upload to Thingiverse?

 

So many questions also, like:

- How does the bottom part look like

- How much tolerance do you have on the drives

- Why not 15 drives instead of 14?

- Any vibration issues?

- How is the airflow and drive temps?

- On the read, I see two squares, are these just there to hold up the mount?

- How did you glue/stick/mount them together?

- Did you screw the cage to the rack, or is it just sitting there loose?

Edited January 28, 20224 yr by PeterDB

1 hour ago, PeterDB said:

 

wow, this is really an elegant solution and just what I have been looking for. Any chance you can share the STLs or upload to Thingiverse?

 

So many questions also, like:

- How does the bottom part look like

- How much tolerance do you have on the drives

- Why not 15 drives instead of 14?

- Any vibration issues?

- How is the airflow and drive temps?

- On the read, I see two squares, are these just there to hold up the mount?

- How did you glue/stick/mount them together?

- Did you screw the cage to the rack, or is it just sitting there loose?

The design is pretty crude at the moment, once I finish my current project I'm going to work on the v2 which should have improvements and potentially incorporate easy to source backplanes.   I've got a spare server case to work with, so I'm hoping I can iterate the design quickly over the next few months.  I'll have to dig out the v1 STLs, I'll add them here when I find/regenerate them.

 

- How does the bottom part look like

Identical to the top, but with a layer of thin adhesive eva foam to add some vibration dampening qualities.

I ended up doing it this way as it meant horizontal components, left and right halves, used the same STL.

 

- How much tolerance do you have on the drives

The model was designed with zero tolerance, because I knew the print could go either way and I could adjust

It's ended up as a "firm, but gentle press" haha, but I need to design a mechanism to allow easier retrieval.

 

- Why not 15 drives instead of 14?

I wanted to maintain a reasonable airgap for cooling and 15 wouldn't work for that, besides the overhang from the side of the case restricted use of the full width of the case.

 

- Any vibration issues?

between the base material and the use of plastic/filament for the mounting, its very minimal, even with the current 28 drives.

 

- How is the airflow and drive temps?

Good, no temp warnings over summer here in Oz, typically low 30C for front, and high 30C in back... second fan for push pull would help a lot.  Also using 140mm gave a big improvement in terms of airflow and pressure, while actually being quieter than the 120mm fans they replaced.

 

- On the rear, I see two squares, are these just there to hold up the mount?

Yeah, just in lieu of rear fans... originally made for support on the sides but I miscalculated the thicknesses so they didn't fit there.

Its needed as its a tight fit getting drives in and there's no other support there currently.

 

- How did you glue/stick/mount them together?

These were plastic welded with a filament stick and a dremel, but I've since worked out the filament I 3D print with (Prusament) can be bonded with acetone, which is easier, neater and results in a stronger bond.

(note: acetone bonding doesn't work with plain PLA)

 

- Did you screw the cage to the rack, or is it just sitting there loose?

Friction fit would best describe it, but yes not fastened.

An improved design would see some threaded inserts added to the finished print so I can affix to the chassis

Thanks for the response. This is good info! I actually started to do some test prints and glue tests, and also started to search for eva foam strips. First test prints were promising!

 

However, while doing my research for eva foam, I discovered that one of the case manufacturers actually started producing a 28 vertical bay rack again, and I just ordered one of those. Only downside is that the rack uses 120mm fans instead of 140mm.

 

Try searching for 4u 28 drives, and the case manufacturer pops up on Alibaba and a few sellers on AliExpress. Inter-Tech is selling them under the model 4F28 https://inter-tech.de/en/products/ipc/server-cases/4f28-mining-rack

Here are some pictures of the 4F28. Very promising!

 

Will try to post more pictures in a different thread.

 

 

 

 

 

 

Edited February 2, 20224 yr by PeterDB

Some new bits finally arrived to get the v2 design going

Can you please explain the power supply to me? How many hard drives do you have per connector on the power supply? Is there any issue if all the hard drives start simultaneously?

On 7/19/2023 at 4:10 AM, Micha210 said:

Can you please explain the power supply to me? How many hard drives do you have per connector on the power supply? Is there any issue if all the hard drives start simultaneously?

Start up surge is on 12V rail, no issues with that on a 1200W PSU, 5V rail was at its limit with 28 disks, but no current in-rush to contend with.

I've since moved most of my disks into a Netapp DS4246 (an external disk shelf) which has its own PSUs.