Chia farming, plotting; array and unassigned devices

[sota]

yea i picked them up to be primarily used as cold storage for my security cameras, but figured since I have time before I'd use them all i'd toss a couple into chia use.

[sota]

freezer trick sometimes gets drives to behave.

other times, not.

[sota]

didn't work.  disk makes some evil click noises at startup, then dies (spins down.)

tried a new disk in the MD1000... doesn't recognize.

there's plots being loaded onto one of the other drives right now, so I can't do a restart on the system yet.

wondering now if either the slot has died, or the little adapter board (I forget what they're called) has toasted itself.

[TexasUnraid]

That would suck to have a bay that kills drives.

 

I have a few 4tb drives that give the evil click and spin down as well.

 

Well, have about 10tb live on hpool for the last ~12 hours and so far have farmed 0.50 cents, heck of a lot better then 0 cents lol.

 

Moving the rest of my plots around and will add them in later today to bring the total to around 20tb and then start plotting again.

[Candle]

Hi All,

Can anyone help a newbie with how to install the mad max plotter docker in unraid?

 

Specifically, where do I put all these variables?

 

docker run \

-v <path-to-your-tmp-dir>:/mnt/harvester \

-v <path-to-your-final-dir>:/mnt/farm \

-m 8G \

--cpus 8 \

odelucca/chia-plotter \

-t /mnt/harvester/ \

-d /mnt/farm/ \

-p <pool-key> \

-f <farm-key> \

-r 8

 

A screen shot of how to do it on the docker page would be super helpful. I tried pulling the template from docker hub but it doesn't pre fill anything for me.

[TexasUnraid]

Yeah, someone needs to make an unraid template for it, I have no idea how to upload one to the repo though.

 

I am using a custom docker I made for testing right now but generally with off-repo dockers like that, that don't come up if you search CA (does it show up in CA if you select search dockerhub?) is to copy/paste those commands into the unraid terminal and let it spin up the docker manually. It will then be added to the GUI and you can usually tweak the settings from there easier then creating it from scratch.

[Candle]

Thanks! this is helpful, I didnt think of using unraid terminal.

 

It does show up under search dockerhub but nothing comes over.

[TexasUnraid]

6 hours ago, Candle said:

Thanks! this is helpful, I didnt think of using unraid terminal.

 

It does show up under search dockerhub but nothing comes over.

 

Ok, if it comes up in the search now then you can simply select to install it and then manually add the paths and variables to the tmeplate.

 

The -v are paths, the rest are varibles and would go in the post arguments IIRC. Ignore the CPU's/memory stuff on unraid, it is handled in the GUI.

 

I preferred to make a custom docker that I could terminal into and run everything from inside the docker, I find that easier then trying to use these direct docker run commands on unraid.

 

It was really easy to spin up the docker, just used the ubuntu base docker from dockerhub, mapped the paths to my plots and then installed madmax following the build process in the github.

Edited June 17, 2021 by TexasUnraid

[TexasUnraid]

My extra memory showed up today, giving me a total of 384gb, plenty to plot entirely in ram (also good for very high compression of very large files that I do from time to time) 😉

 

No more wear on SSD's or HDD's!

 

Got to wait for it to copy all the plots off the drives that are in it right now before I can start playing with it though. Looks like that will take ~18 hours for the ~18tb (having to change the formats of the drives from btrfs to NTFS since I will farm on windows with hpool for now).

Edited June 17, 2021 by TexasUnraid

[sota]

Nice. I've resigned myself to just letting what I've got do it's thing for now.  I'm getting ~ 24 plots/day it looks like, which means a 4TB disk is filling up every 1.5 days, and I have only 4 more of those disks to fill at this point.  I'll waste more plotting time trying to get marginally faster, than it'll take to finish these off, and right now the only thing i have to do is periodically change a share to point to the next disk.

 

I do wish that K32 plots were *just* a bit smaller, as i'm leaving 80GB of space unused on each disk.

Edited June 18, 2021 by sota

[sota]

since reddit sucks and I can't ask there, anyone know how to get the farmer to look in subdirectories?  I'd like to not have have 15+ plot directories listed, and just park all the disks under subdirectories (as I do now.)

[TexasUnraid]

3 hours ago, sota said:

Nice. I've resigned myself to just letting what I've got do it's thing for now.  I'm getting ~ 24 plots/day it looks like, which means a 4TB disk is filling up every 1.5 days, and I have only 4 more of those disks to fill at this point.  I'll waste more plotting time trying to get marginally faster, than it'll take to finish these off, and right now the only thing i have to do is periodically change a share to point to the next disk.

 

I do wish that K32 plots were *just* a bit smaller, as i'm leaving 80GB of space unused on each disk.

 

Ditto, on a few disks I am literally a few hundred mb short of an extra plot.

 

When I replot for pools I might do a mix of k32 and k33 to get better fillage.

 

1 hour ago, sota said:

since reddit sucks and I can't ask there, anyone know how to get the farmer to look in subdirectories?  I'd like to not have have 15+ plot directories listed, and just park all the disks under subdirectories (as I do now.)

 

I could not figure out how to do that and resigned myself to just listing a bunch of directories.

 

Although if you are using unraid, you could do a share on it and combine all the disks into a single share. That is the easiest option I see. There is also mergerfs I think it is called on linux that works similar to unraid but is free. Could just use a trial key of unraid for testing short term.

 

I have heard of people having issues with lookup times when using a network share though so be sure to check that.

[sota]

well, I bought something...

Chenbro NR12000 1U 1x 8m QUAD CORE E3-1220 3.1GHz 8GB RAM NO HDD $172.73 shipped.

can take 1x 2.5" and 12x 3.5" HDDs internally.

going to set it up as its own farm, with 12 of the disks I've already plotted.

not going to go the farmer/harvester route though, as I'm thinking it's better to have each machine with disks separated, so if one poops itself it doesn't take out the others.

if I like it, and the vendor has more in the future for the same price I might do another later on.

we'll have to see.

Edited June 18, 2021 by sota

[sota]

how big is a k33 anyways?

[TexasUnraid]

Not a bad setup if you need the 1u setup, my farmer is using a very similar mobo/cpu (1275 vs 1220)

 

Personally I prefer a 2u hot swap chassis for the same number of bays, I have seen some supermicro chassis go for around that price recently depending on what you want in it. Actually considered getting one to swap out my existing 2u chassis that only has 8 bays actually.

 

I also like that 2u has the option to use add-in cards as long as they are low profile.

[TexasUnraid]

3 minutes ago, sota said:

how big is a k33 anyways?

 

224gb?

 

Good site for figuing out the best combo of plots for a drive:

 

https://plot-plan.chia.foxypool.io/

[sota]

well that sucks then.

I'd need to stop plotting k32 right now, and start plotting k33, just to get better utilization.

although a better question is... what size temp disk space do I need to plot k33?

am only using 300GB drives right now.  if I change the script, do I need to re-work my entire plotting setup for disks?

Edited June 18, 2021 by sota

[sota]

looking at some of the setups i've seen, I'm wondering what their $/connected-drive rate is.

that chenbro unit, will be < $15/drive.  that strikes me as pretty decent.

[TexasUnraid]

57 minutes ago, sota said:

looking at some of the setups i've seen, I'm wondering what their $/connected-drive rate is.

that chenbro unit, will be < $15/drive.  that strikes me as pretty decent.

 

Thats not bad at all, some people are paying STUPID prices.

 

At the prices I paid last year I am right about the same $15 a drive, actually got one a month ago for the same price (24 bay for $400).

 

I see people paying $1500-2500 for 24-45 bay systems right and left though, they could hardly give away those systems a few months ago for $300. Boggles my mind.

 

I actually just got in 3x 14tb drives I ordered just before chia hit. They will be used to give me a complete backup of my main server (missing my media drives right now) but maybe I should farm with them for a bit to burn them in lol.

[TexasUnraid]

1 hour ago, sota said:

well that sucks then.

I'd need to stop plotting k32 right now, and start plotting k33, just to get better utilization.

although a better question is... what size temp disk space do I need to plot k33?

am only using 300GB drives right now.  if I change the script, do I need to re-work my entire plotting setup for disks?

 

That is a good point, not sure what the temp space is for K33, guessing double? So you would need 2x drives to hold them I guess.

 

Depending on what the effects of plotting 100% in ram are, I might start plotting k33 and using a temp drive to better fill space.

[sota]

That $15/drive is just for the connectivity of it.  I'm running on those ~$32/drive drives right now.  so I guess you could say total costs per drive farming, is ~ $47/drive, or ~$11.75/TB active.

[TexasUnraid]

14 minutes ago, sota said:

That $15/drive is just for the connectivity of it.  I'm running on those ~$32/drive drives right now.  so I guess you could say total costs per drive farming, is ~ $47/drive, or ~$11.75/TB active.

Yeah I got what you were saying I was talking about the same.

 

People are paying $1500++ for 24-45 bay chassis by themselves without any drives. So that is $30-$50 per bay then they are paying $20-$40/TB for the drives themselves. I don't see them ever breaking even but possible some hardware will come on the market at some point for a good price.

[sota]

my bad on that, if warranted.

 

yea i'm hoping to see some stuff go back on fleabay, at reduced pricing.

 

on some other news: my scripts continue to just plot along (pun intended.)

just watched 'monitor' suspend robocopy because a TMP file started getting copied to the staging drive on the one plotter.

[sota]

 

plot map is getting nicely full with it finding a potential match every round.

[TexasUnraid]

I am having to play musical drives right now converting them to ntfs as the btrfs driver for windows apperantly has a memory leak and if filling up my 32gb in my farming rig when used with a bunch of drives.

 

Still going to use btrfs for the raid setups (raiding the small drives as I would have a lot of wasted space otherwise) but converting the rest to ntfs and juggling plots around.

 

Should finish tomorrow and then get back to plotting myself. Gonna give plotting 100% in ram a go to see just how much the HDD's are slowing the process when using ran just for temp2.