Chia farming, plotting; array and unassigned devices

[sota]

good thing the one harvester has almost 2TB of space to store plots, before they're moved to the farm machine.  farm target drive filled up, and there's 6 plots waiting on the one to get moved, and 2 more on another.

[sota]

been thinking about unraid and this chia thing.

one if the nice things would be, if the disks could stay spun down.

problem is, spin up time is too long for the timeouts in chia.

so the question is: is anyone looking at actual spin up times for drives?  some obviously spin up faster than others. would there be any financial (mostly, power) benefit to equipping a farm with fast start drives.  this almost feels like a perfect task for a hybrid drive.  problem is, how to make sure the needed data is in the SSD portion of said drive.

 

if chia is intended to be "green" technology, then it needs to figure out how to allow for drives that are more near-line vs. on-line.  Of course, the argument that was made by the developer of only using already available storage can still be made, but I think that ship sank the 2nd day after mainnet went live.

[TexasUnraid]

Everything I have seen says not to spin down drives. Apparently they spin up all the time anyways due to the random reads and if you did manage to get a proof it would most likely be too late.

 

I was thinking about doing the same thing.

 

The argument against it was that is the check times were too long then people could create winning plots once they see the challenge.

[TexasUnraid]

Well since my chia wallet is already compermised from using hpool. I figured what the heck, I jumped on Chiagreen and flax

 

https://www.chaingreen.org/

 

https://flaxnetwork.org/

 

As well, they use the same chia plots to farm so basically free. Not sure I put much weight on them but while netspace is still low figure I will see if I can win anything.

[sota]

Chia1
Sun 06/13/2021 13:55:26.35 starting plot 11.1...
Sat 06/19/2021 20:52:58.43 starting plot 18.8...
       06      06:57:32    count: 65 543452s 9057s/p 2.52h/p 9.52ppd

Chia10
Mon 06/14/2021 10:30:52.08 starting plot 11.1...
Sat 06/19/2021 20:25:31.19 starting plot 18.7...
       05      09:54:39    count: 57 467679s 8205s/p 2.28h/p 10.53ppd

Chia11
Sun 06/13/2021 13:59:06.88 starting plot 11.1...
Sun 06/20/2021 06:51:56.60 starting plot 14.10...
       07      17:52:50    count: 38 669170s 17610/p 4.89h/p 4.91ppd

24.96ppd

some #s.

Chia's 1 and 10 plot 8 plots in parallel

Chia 11 only does 4, as I don't have enough scratch disks.

also just moved 'latest' to the first of 3 disks remaining, so I'll be "full" in roughly 4.32 days more of plotting.

 

 

Edited June 20, 2021 by sota

[sota]

also picked up an HP Z420 Workstation 6-Core Xeon E5-1660 3.3Ghz/16GB/DVDRW/NO HD for an unRAID setup for mom and her SageTV PVR system; needed something large enough to hold a bunch of 3.5" disks (got 7 in it now, could have done 8), powerful enough to run unRAID, stuckless sagetv container, and possibly a few other bits.  It has 10 sata ports on the mainboard and can support up to 256GB ram.  With the 7 rusty boys spun down, a 500GB SSD, and the box doing pretty much nothing but waiting, it's idling at 73W.  That's not bad considering it's got an AMD FirePro V4900 video card (WAY overkill for the needs, but that's what it came with, and I don't have anything to swap for it right now; no onboard video.)

 

Cost of the machine was $202 shipped.  4-in-3 can be had for it looks like $40 (china), or $68 (US).  taking the higher end that's $38.50/drive-connected. or if you want to just stuff the disks in the 5.25 bays and let 'em flop around  that's $25.25/drive-connected.

[TexasUnraid]

Thats not bad at all for that setup, 73w is plenty good for a workstation board. I think with no drives my current setup idles around 120w but it has dual CPU's and several add-in cards. with the drives and it running "normally" I am generally closer to 200w, a lot more then I would like but I got tired of constantly trying to fix consumer grade gear.

 

Before I finally went with server gear last year I was using a bunch of old systems I hobbled together. Still got most of them in reserve.

 

I had an old dell mobo stuffed in my old antec 900 gaming case with several 4in3 adapters giving my 12 hot swap bays. Worked really nice overall and without drives idled around ~60w IIRC. But I had a lot of issues with drives dropping out and the like, mostly due to cabling issues with the rats nest that was in there.

 

Found a local guy selling some supermicro chassis and ended up getting 3 fully loaded servers for $600 since he didn't want to ship them and needed them gone. I was going to sell 2 of them to keep the last one for free + some profit but ended up finding uses for all of them. Plus the $35 rack looks bare with just 1 server in it. lol

 

Looks like I won some chiagreen last night, feels strange to actually win something lol.

 

Flax is growing fast, was 2 days to win yesterday, now it is 3 days.

[TexasUnraid]

Well, finally got my musical drives sorted out and can get back to plotting.

 

Just ran a test with the madmax plotter running 100% in ram.

 

34mins, so 4 mins less then using the raid0 10k drive array.

 

I have to say, Madmax did a really good job of optimizing the plotter. Running in ram is great simply because you don't need to worry about thrashing drives but only gains you small amount of extra performance.

 

Multi-threaded pipelined Chia k32 plotter - 2920125
Final Directory: /media/chia/ramdisk/
Number of Plots: 1
Process ID: 6241
Number of Threads: 32
Number of Buckets: 2^7 (128)
Pool Public Key:   
Farmer Public Key: 
Working Directory:   /media/chia/ramdisk/
Working Directory 2: /media/chia/ramdisk/
Plot Name: 
[P1] Table 1 took 14.4916 sec
[P1] Table 2 took 129.525 sec, found 4294858921 matches
[P1] Table 3 took 149.095 sec, found 4294851161 matches
[P1] Table 4 took 184.589 sec, found 4294658713 matches
[P1] Table 5 took 180.881 sec, found 4294349888 matches
[P1] Table 6 took 175.914 sec, found 4293634966 matches
[P1] Table 7 took 133.814 sec, found 4292281384 matches
Phase 1 took 968.327 sec
[P2] max_table_size = 4294967296
[P2] Table 7 scan took 9.15548 sec
[P2] Table 7 rewrite took 41.606 sec, dropped 0 entries (0 %)
[P2] Table 6 scan took 34.6718 sec
[P2] Table 6 rewrite took 47.1672 sec, dropped 581419162 entries (13.5414 %)
[P2] Table 5 scan took 29.8565 sec
[P2] Table 5 rewrite took 45.144 sec, dropped 762174772 entries (17.7483 %)
[P2] Table 4 scan took 29.7342 sec
[P2] Table 4 rewrite took 45.7968 sec, dropped 829009177 entries (19.3033 %)
[P2] Table 3 scan took 29.6592 sec
[P2] Table 3 rewrite took 45.4891 sec, dropped 855165911 entries (19.9114 %)
[P2] Table 2 scan took 29.2397 sec
[P2] Table 2 rewrite took 43.5771 sec, dropped 865576350 entries (20.1538 %)
Phase 2 took 453.969 sec
Wrote plot header with 268 bytes
[P3-1] Table 2 took 58.2705 sec, wrote 3429282571 right entries
[P3-2] Table 2 took 32.0414 sec, wrote 3429282571 left entries, 3429282571 final
[P3-1] Table 3 took 60.8781 sec, wrote 3439685250 right entries
[P3-2] Table 3 took 34.8838 sec, wrote 3439685250 left entries, 3439685250 final
[P3-1] Table 4 took 56.1365 sec, wrote 3465649536 right entries
[P3-2] Table 4 took 33.8989 sec, wrote 3465649536 left entries, 3465649536 final
[P3-1] Table 5 took 55.1382 sec, wrote 3532175116 right entries
^P[P3-2] Table 5 took 35.6375 sec, wrote 3532175116 left entries, 3532175116 final
[P3-1] Table 6 took 64.5304 sec, wrote 3712215804 right entries
[P3-2] Table 6 took 37.5803 sec, wrote 3712215804 left entries, 3712215804 final
[P3-1] Table 7 took 69.2485 sec, wrote 4292281384 right entries
[P3-2] Table 7 took 43.6463 sec, wrote 4292281384 left entries, 4292281384 final
Phase 3 took 587.856 sec, wrote 21871289661 entries to final plot
[P4] Starting to write C1 and C3 tables
[P4] Finished writing C1 and C3 tables
[P4] Writing C2 table
[P4] Finished writing C2 table
Phase 4 took 74.3834 sec, final plot size is 108799781942 bytes
Total plot creation time was 2084.63 sec

 

Just need to juggle a few things around yet and I can get back to plotting full time, might also start my main server plotting as well to knock it out quicker.

[sota]

20 hours ago, TexasUnraid said:

Well since my chia wallet is already compermised from using hpool. I figured what the heck, I jumped on Chiagreen and flax

 

https://www.chaingreen.org/

 

https://flaxnetwork.org/

 

As well, they use the same chia plots to farm so basically free. Not sure I put much weight on them but while netspace is still low figure I will see if I can win anything.

how/where did you find out about those?

[sota]

heh...

running all 3 blockchain on the same machine... might need to get another Z420 just to manage that.

poor C2Q processor is getting CRUSHED, trying to sync up flax.

 

waiting for my NR12000 to ship still.  hopefully that'll do better as the farm.

Edited June 20, 2021 by sota

[TexasUnraid]

15 minutes ago, sota said:

how/where did you find out about those?

 

It was posted in the plotman docker thread on here. There is a 3rd blockchain that can use the chia plots as well, spare.farm but the github seems to be down right now.

[TexasUnraid]

28 minutes ago, sota said:

heh...

running all 3 blockchain on the same machine... might need to get another Z420 just to manage that.

poor C2Q processor is getting CRUSHED, trying to sync up flax.

 

waiting for my NR12000 to ship still.  hopefully that'll do better as the farm.

 

Yeah, my 1275 did the same thing, took it overnight to get fully synced and then it calmed down.

 

Got hppol, flax, chiagreen and nicehash on an old RX570 running on this machine. Need to check power at the wall with the drives in it, guessing around 250-300w. Hardest part is keeping the drive temps at a comfortable level in the 100-120f garage.

 

It has some 7k rpm fans in it now, I ordered some really high static pressure delta 9k fans for $8 from china, gonna see what they do as things get warmer.

 

Right now though I am impressed, the drives are generally around ~40c during the day and low 30's at night. Just worried when temps climb another 10-15 degrees outside as summer progresses.

[TexasUnraid]

I have mined $5 on hpool in the last few days, most of that time was with ~10tb of less online due to playing musical drives.

 

Although Chias price keep falling, so not sure it will ever be profitable to farm it but what the heck, already got the drives and plots, I will let it ride for now.

 

It has been a fun chance to play with my hardware in ways I don't normally get to. Plus if I can make $100 I will be out nothing.

Edited June 20, 2021 by TexasUnraid

[sota]

so I didn't need to do anything to the PLOT files for the other farmers?

both flex and chain are saying they can't open all 402 plots I have right now.

do they need to "touch" them in some way, that requires write access?

of course they haven't synced yet so I dunno if maybe that's part of it?

[sota]

and the dropping chia price is actually good in my book. $200 and i'm just flat out buying like 10 chia.

 

you picked an exchange for buying/selling chia yet?

[TexasUnraid]

8 minutes ago, sota said:

so I didn't need to do anything to the PLOT files for the other farmers?

both flex and chain are saying they can't open all 402 plots I have right now.

do they need to "touch" them in some way, that requires write access?

of course they haven't synced yet so I dunno if maybe that's part of it?

 

I had issues when I didn't allow write access as well with hpool, no idea why.

 

I also saw others say to always start flax/chiagreen first and then hpool/chia.

 

I am currently farming 22tb of plots on all 3 without an issue. Won some chiagreen already.

[TexasUnraid]

6 minutes ago, sota said:

and the dropping chia price is actually good in my book. $200 and i'm just flat out buying like 10 chia.

 

you picked an exchange for buying/selling chia yet?

 

Not sure I have the same faith in chia as you to flat out buy it lol. I REALLY like the idea behind it but the company only seems interested in the IPO side of the company, I don't care for public companies as a whole so that is a bit of a turn off from the start for me.

 

I am also on the hunt for an exchange, don't care for the US based ones with KYC and have not had time to look into the options without KYC. Since I am just a small time player I don't see a reason to go all in with the KYC.

 

Honestly if I could just convert the XCH to BTC that would be enough for me. I figure if I can get $500 of bitcoin that is better then nothing, although could be long while before I get near that at this rate lol.

[TexasUnraid]

Ran into an interesting issue with plotting in ram, having an issue getting the finished plot off the ramdrive fast enough so that it does not run out of room with the next plot.

 

Works ok with faster drives but on the slower drives or as they fill up it can't quite get them off fast enough.

 

So might have to move the finished plot to a 10k drive and then from there to the final drives. Or possibly just go back to using the 10k drive as the temp dir since it only slowed it down a bit.

[TexasUnraid]

I installed 2x 10k drives as a temp holding spot for the plots from the ramdisk, looks like the ~300mb/s is fast enough to copy the plots over with some time to spare. Then it has 30 mins to copy it to the farming drive before the next plot.

 

Could even use an SSD for this since it is only 100gb of writes per plot but I already have these 10k drives.

 

So far seems to be working good, got a script running that checks for a new plot on the 10k drives every 5 mins and automatically copies it to the farming drive.

 

A bit of manual work is required to mount the next farming drive but I can live with that for now.

 

For reference I am seeing a peak memory usage of 251gb of ram plotting 100% in ram. Could possibly do it with 256gb of ram if you have nothing else going at all and/or have a swap setup (it should automatically move the old stuff to the swap).

Edited June 20, 2021 by TexasUnraid

[TexasUnraid]

Also installed the spare.farm blockchain. Looks like they re-started the blockchain to account for the massive growth it had since starting. Makes sense a lot of people spent a lot of time plotting and want to see SOMETHING for the effort lol.

 

Syncing it now, it is amazing how fast the network grew, it was up at 40PB within 130 blocks (something like 24 hours).

 

So far with mutiple chains using the plots, looks like the best option is to open them one at a time, let them finish scanning and then do hpool last. Otherwise I also got invalid blocks.

[sota]

8 hours ago, TexasUnraid said:

 

I had issues when I didn't allow write access as well with hpool, no idea why.

 

I also saw others say to always start flax/chiagreen first and then hpool/chia.

 

I am currently farming 22tb of plots on all 3 without an issue. Won some chiagreen already.

 

Ok.  I'll wait until flax and chain have synced up, then I'll restart everyone in a different order.

also downloading spare.farm now.

[tjb_altf4]

13 hours ago, TexasUnraid said:

Well, finally got my musical drives sorted out and can get back to plotting.

 

Just ran a test with the madmax plotter running 100% in ram.

 

34mins, so 4 mins less then using the raid0 10k drive array.

 

I have to say, Madmax did a really good job of optimizing the plotter. Running in ram is great simply because you don't need to worry about thrashing drives but only gains you small amount of extra performance.

Did you measure power usage by any chance ?

That would be an interesting comparison to see if it was a noticeable or negligible difference.

[sota]

speaking of power, those 3 DL380's plus the farm computer and MD1000, are now running the UPS right on the ragged edge of overload. :eek:

[sota]

17 hours ago, TexasUnraid said:

 

I had issues when I didn't allow write access as well with hpool, no idea why.

 

I also saw others say to always start flax/chiagreen first and then hpool/chia.

 

I am currently farming 22tb of plots on all 3 without an issue. Won some chiagreen already.

figured out my "mistake"... didn't import the mnemonic to the other chains, to allow them to farm them.  figured that out when I looked at the log on one.

[sota]

nagging pet peeve of mine...

 

2021-06-21T07:21:11.926 full_node flax.full_node.full_node: INFO     Requesting blocks: 33312 to 33344
2021-06-21T07:21:18.412 wallet flax.wallet.wallet_node    : INFO     Requesting blocks 2688-2720

 

when log entries are done in different styles. I'm a "consistency" kind of man.  unless of course it's a deliberate stylistic difference, to highlight one being wallet_node and the other being full_node.