[Support] QDirStat, Jcloud - cryptoCoin templates


Recommended Posts

2 hours ago, L0rdRaiden said:

I have kill it and created a new one, now is working 5 MB in one night at least. I think there was something weird with it.

I've notice something like this before, and clicking on the update container function a lot of times does seem to kick start going again. The repository is auto-built every eight, or was it twelve, hours according to the page. I hadn't figured out how to check/test a hypothesis (one part video-games, another part getting over flu and no energy/focus), which is after an update or two, docker client gets dropped by the network because it's not current.  

 

EDIT: Yes, I know it shows up as "up-to-date" in the unRAID webui, no clue why it behaves this way, another item on my mucking-to-do list. Although if anyone has a glimmer of insight, I'm "all ears."

 

Glad to read things are moving towards working for you.

Edited by Jcloud
Link to comment
7 minutes ago, nuhll said:

Dont you use auto updates?

Nope, unRAID auto-checks and I get notices, of when my installed containers need to be updated, but I manually push the update. An artifact of functionality, when I first installed unRAID, never bothered to change because it suites me fine.

Saw it a while ago durring 6.4-rc in settings, but didn't flip the switch, I'll have to go looking again as it's probably worth doing and not being bothered by it. :)

Link to comment
On 3/13/2018 at 4:10 PM, L0rdRaiden said:

Delta was a problem but since yesterday I applied this changes to unraid (minpoll 6 and maxpoll 10) and now is working better  <- @limetech

https://access.redhat.com/solutions/39194

Out of curiosity where did you make and save this change to on your system? I saw ntp pool fields, and in /boot I saw /boot/config/ident.cfg which looks like a master config file for unRAID.  I know you're relatively new to unRAID L0rdRaiden, so I'm checking to see if you may have edited /etc/ntp.conf?  Wanted to double-check that you knew /etc/ gets rebuilt (afaik) every time your server is rebooted.  UNRAID works as a live-boot-disc.

 

And thinking back on your past thread was that what you meant before about your config file being randomly cleared?

Link to comment

Well I figured out how to do this: In theory one could do this for N-number of nodes.  What I have NOT yet figured out, can this be done through unRAID web interface? Can I start all the nodes at once or would I need a script? (presently I'm betting on a shell script)

┌─────────────────────────────────────────────┬─────────┬──────────┬──────────┬─────────┬───────────────┬─────────┬──────────┬───────────┬──────────────┐
│ Node                                        │ Status  │ Uptime   │ Restarts │ Peers   │ Allocs        │ Delta   │ Port     │ Shared    │ Bridges      │
├─────────────────────────────────────────────┼─────────┼──────────┼──────────┼─────────┼───────────────┼─────────┼──────────┼───────────┼──────────────┤
│ 09ecfee3004001336fc2050531a55695ed326ee3    │ running │ 1h 21m … │ 0        │ 139     │ 18            │ ...     │ 4000     │ 17.76GB   │ connected    │
│   → /storj/share                            │         │          │          │         │ 0 received    │         │ (uPnP)   │ (1%)      │              │
├─────────────────────────────────────────────┼─────────┼──────────┼──────────┼─────────┼───────────────┼─────────┼──────────┼───────────┼──────────────┤
│ a403eadbdb8d7b6521e78aff82ef4213eb08e92b    │ running │ 2m 32s   │ 0        │ 133     │ 0             │ 235ms   │ 4001     │ ...       │ connected    │
│   → /storj/Node_2                           │         │          │          │         │ 0 received    │         │ (TCP)    │ (...%)    │              │
└─────────────────────────────────────────────┴─────────┴──────────┴──────────┴─────────┴───────────────┴─────────┴──────────┴───────────┴──────────────┘

 

For those that would like to try and setup multiple nodes here are the steps:  The official instructions are here (used sections 3.3 - 3.5)

  1.  In your /mnt/user/storj/ share create sub folders, one for each node want to setup.  
    root@HYDRA:/mnt/user/storj/# mkdir Node_1

       

  2. Make addition port-forwards in your firewall, i.e 4001, 4002 for each node you're going to make.

  3. docker exec Storj storjshare-create --storj 0xe09a2b205d3a76196a88505f059c78df7d7c556f --storage /storj/Node_1 --size 500GB --outfile /storj/Node_1/config.json --rpcaddress 0.0.0.0 --rpcport 4001 --manualforwarding true --logdir /storj/Node_1

            where 0xe09a2b205d3a76196a88505f059c78df7d7c556f is substituted for YOUR wallet address! 

    1. Also want to take out 0.0.0.0 and replace it with what you use.

    2. For each node you'll want to increment the port, such that each node has a unique port.

    3. For each node you create, remember to change the /storj/Node_# path.

  4. Then you start the node manually
     docker exec Storj storjshare start --config /storj/Node_2/config.json

      

That's all I have for now, I'll try and expand upon it later.

Edited by Jcloud
I goofed fixing/editing
Link to comment
5 hours ago, Jcloud said:

Out of curiosity where did you make and save this change to on your system? I saw ntp pool fields, and in /boot I saw /boot/config/ident.cfg which looks like a master config file for unRAID.  I know you're relatively new to unRAID L0rdRaiden, so I'm checking to see if you may have edited /etc/ntp.conf?  Wanted to double-check that you knew /etc/ gets rebuilt (afaik) every time your server is rebooted.  UNRAID works as a live-boot-disc.

 

And thinking back on your past thread was that what you meant before about your config file being randomly cleared?

 

Yes I did the change in /etc/ntp.conf I didn't know this a temporary file so what I did wasn't useful I guess :$

 

The config file of storj was cleared once, It was weird because It hasn't happen again even after reboots and updates.

 

Regarding having more than one node for the container I read that is limited to the number of cores you have. Could you please please, ^_^ make this setup available through the interface, having more than 1 node?

I also read that right now is not worth it have more than 1TB per node, so probably 500GB per node is enough.

Link to comment
25 minutes ago, L0rdRaiden said:

Could you please please, ^_^ make this setup available through the interface, having more than 1 node?

*ME* Baby-steps figured out how to even do more nodes. ... "Baby-steps get on the bus"  -- What about Bob?   My not 100% sure it's possible with how the unRAID docker templates work, but I'll keep learning as I go. If I can, I will.  

 

That is weird about your config file, at least it hasn't happened since. I can definitely see more nodes needing more resources. Hmm, I wonder how it scales? Also have to be careful, not to accidentally flood one's router with open TCP connections. Other hand, could get nasty and start packet--shaping one's network. :)

You mentioned maxed node size, I could see that, probably a demand-side sort of issue, also I bet bandwidth have to be some racing upload speeds. Which describes no American consumer grade ISP. ;) 

Link to comment
On 3/12/2018 at 7:29 PM, Jcloud said:

Question, question, and more questions -- I may have gotten over my head. >.<

 

I think delta is latency between your node and the bridge (not 100% certain on this fact), but if you're getting 3 dots after a while then your network is closing off your connection and it's timing out. I don't think you'll get any Allocs until this is resolved.  In a past post you mentioned something about a pfsense VM what sort of router(s)/routing platforms are you currently using?

 

 

Storj's instructions are here: https://docs.storj.io/docs/storj-share-daemon-cli looks like starts around section 3.4. The general idea I get from it is to make a series of subdirectories with modified copies of config.json file; once the storj daemon is running, you can load additional config-files/nodes. I dub thy @physikal expert in this, since judging by his question he's already made it this far. :D  I haven't bothered this far yet, life and energy has gotten in the way.

 

I didn't think the docker command would work, but it was the first thing I could think off off the top of my head. Presently, I don't have an answer for you sorry man.

 

Not sure why, if you are going to "edit" in unRAID webui and then clicking on Apply this might be your source of the config file changing on you.

 

 

Lastly, I want to apologize for my sub-par support, I'm a user too, and you guys are asking some good questions.

All good man! I appreciate the support and you building a sweet docker for us.  I'll do my best to answer the question on multiple nodes to help you support this thang :)

Link to comment
On 3/11/2018 at 6:37 AM, nuhll said:

how you add multiple nodes with one docker?

 

Hey there nuhll.

 

Storj Documentation is here: https://docs.storj.io/docs/storj-share-daemon-cli

 

The steps I used to create additional nodes (bare w/ me, off the top of my head):

 

Docker config:

Container Name: Storj

Host Directory: /mnt/user/storj

Docker Directory: /storj | config is /storj/config.json

Additional nodes:

/storj/node2

/storj/node3

/storj/node4

 

The reason I have so many nodes is max storage for a node is I believe 8TB, so I have each node sharing 7TB.

 

Step 1: Create node directories with docker exec so it has the docker perms on the folder (this matters later when you load the snapshot)

command: docker exec containername mkdir /storj/nodeX

 

Step 2: Create the next node

command: docker exec containername storjshare create --storj yourERC20addresshere --storage /storj/node2 --size 7TB --outfile /storj/node2/config.json --rpcaddress xxx.xxx.xxx.xxx(IP or hostname here) --rpcport 4004 --manualforwarding true --logdir /storj/node2

 

Run docker exec containername storjshare status to check if the node is running.  If the node is not running run this command:  docker exec containername storjshare start --config /storj/nodeX/config.json

 

Step 3:  Saving snapshot

Once you have all the nodes running that you want, make sure you save the state by typing: docker exec containername storjshare save

What this allows you to do is start all the nodes with one command after you reboot your unRAID server, or the container.

command: docker exec containername storjshare load  

 

That load command loads the previously saved state of all nodes running.

 

 

Hope this helps!

 

 

Link to comment
54 minutes ago, physikal said:

  I'll do my best to answer the question on multiple nodes to help you support this thang :)

Cool beans. I just got my head wrapped around multiple nodes and been playing around with the template. I think I can make a bash script to incorporate reloading nodes on an container update or reboot of container, but still have some kinks in it so I haven't posted anything yet. 

 

31 minutes ago, physikal said:

) --rpcport 4004

do you increment your nodes when you make them or do they all have the same port?  I've incremented mine, but seeing your instructions I'm wondering if I need to. 

Link to comment
Just now, Jcloud said:

Cool beans. I just got my head wrapped around multiple nodes and been playing around with the template. I think I can make a bash script to incorporate reloading nodes on an container update or reboot of container, but still have some kinks in it so I haven't posted anything yet. 

 

do you increment your nodes when you make them or do they all have the same port?  I've incremented mine, but seeing your instructions I'm wondering if I need to. 

Nice! What's your ETH address so I can donate? I appreciate all you're doing.  Makes my life 10x easier.

Link to comment
1 minute ago, Jcloud said:

Cool beans. I just got my head wrapped around multiple nodes and been playing around with the template. I think I can make a bash script to incorporate reloading nodes on an container update or reboot of container, but still have some kinks in it so I haven't posted anything yet. 

 

do you increment your nodes when you make them or do they all have the same port?  I've incremented mine, but seeing your instructions I'm wondering if I need to. 

Sorry, missed this other question.  Yes, node folder and port increment. I typically give each node a 2 port jump.  So node 1 is 4000, 2 is 4002, 3 is 4004, etc.  Just in case.

Link to comment
4 minutes ago, Jcloud said:

Cool beans. I just got my head wrapped around multiple nodes and been playing around with the template. I think I can make a bash script to incorporate reloading nodes on an container update or reboot of container, but still have some kinks in it so I haven't posted anything yet. 

 

do you increment your nodes when you make them or do they all have the same port?  I've incremented mine, but seeing your instructions I'm wondering if I need to. 

You have to change port

Node 1

4000

4001-4003

Node 2

4004

4005-4007

Link to comment
1 minute ago, physikal said:

Nice! What's your ETH address so I can donate?

Don't have to, but if you want to just go up in the thread to my example of making a node, that address is the eth address I'm using for Storj.

 

1 minute ago, L0rdRaiden said:

Remember this before you do anything "stupid" ;)

 

http://uk.businessinsider.com/bitcoin-pizza-10000-100-million-2017-11

HAHA too late! Before I was on unraid I lost 14.5BTC from a currupted password file, from a failing HD. I've cried many tears over this. ;)

 

Link to comment
4 minutes ago, Jcloud said:

Don't have to, but if you want to just go up in the thread to my example of making a node, that address is the eth address I'm using for Storj.

 

HAHA too late! Before I was on unraid I lost 14.5BTC from a currupted password file, from a failing HD. I've cried many tears over this. ;)

 

lol, I don't think of it like that.  If I sent you ETH now, I HOPE it goes up for you.  :P

 

Sorry for your loss.

 

Use l0rd's ports, he seems to know more about the ports than I do though my current setup is working fine.

Edited by physikal
Link to comment
10 minutes ago, physikal said:

Sorry for your loss.

You make it sound like I lost a child. :P The good news was that I mined all of what was lost, so only thing I lost was time, and cash spent on electric bill; even did the mining with two GPU's so I didn't even have specific-btc hardware costs. It was a while ago too, before BTC hit $1K, the first time.

 

14 minutes ago, physikal said:

Use l0rd's ports, he seems to know more about the ports than I do lol.

In my testing I've done it both single increments and L0rd's separation -- to be honest IDK if I've seen a difference in functionality myself.

 

Link to comment

@Jcloud  Just realized you wrote up a node guide.  I noticed you used mkdir as root on the host.  Let me know if you have any problems with that.  When I did that, every node directory I made had host perms that the docker didn't have permissions to, so when trying to run the storjshare load command via docker container it didn't have permissions to load the other nodes outside of the initial load the docker container made.  So check out my step 1.

 

Just an FYI for when you pimp this thing out :P

Link to comment
5 hours ago, physikal said:

Just an FYI for when you pimp this thing out :P

I just wrote the template, the actual container is not my work. I haven't copied the repository yet, I still have much to learn. 

However....

 

How-To: add a script to template, which will enable reloading your nodes after container upgrade or rebuild:

----------------------------------------------------------------------------------------------------------------------------

   Two Main tasks:

  1.  Write the script file. In this case my example is the MOST BASIC form of the script.
  2. Where and what to change in docker template.

 

Script file:  ** First off, please be careful as this will be in your USB flash drive (/boot/ directory) **

 In your /boot/ directory  (encourage even having in some kind of script subfolder of choice) make a file, in my case I called it, storjmn.sh

Inside the file:

#!/bin/bash
docker exec Storj storjshare start --config "/storj/Node_1/config.json"
docker exec Storj storjshare start --config "/storj/Node_2/config.json"
docker exec Storj storjshare start --config "/storj/Node_3/config.json"
docker exec Storj storjshare start --config "/storj/Node_4/config.json"
docker exec Storj storjshare start --config "/storj/Node_5/config.json"
docker exec Storj storjshare start --config "/storj/Node_6/config.json"
docker exec Storj storjshare start --config "/storj/Node_7/config.json"
docker exec Storj storjshare start --config "/storj/Node_8/config.json"
docker exec Storj storjshare start --config "/storj/Node_9/config.json"
docker exec Storj storjshare start --config "/storj/Node_10/config.json"
docker exec Storj storjshare save

For those not familiar with shell scripts, first line is required, and each line is a single command as if you had typed it in and pressed enter.  The body should look familiar, substitute Node_# for your corresponding folders, and number of nodes you run. The last line saves the state, so if you shutdown the container, and restart it, you can reload the state via:

docker exec Storj storjshare load

Next step for script file have to make it executable, in command line as root, or sudo-er privileged, 

chmod 744 /boot/yourLocation/yourSriptName.sh

 

 

 

For the docker template:

storj_exp.thumb.JPG.1a41a4d42be26f8e3770f2d66e1b5e88.JPG

That's   ;/boot/yourScriptLocationANDname.sh

             ^------>>  Note that leading semi-colon, don't miss it, "or you're going to have a bad time."

 

For those that are curious, the template/webui is a wrapper for command typed in the text-based console. When you use a semi-colon you're effectively ending command-line1, and starting command-line2  -- in this case the script to load our nodes.  Now, this DOES NOT WORK in extra parameters (which is executed at run-time of the container), I've tried - breaks stuff, 

 

"And that's my famous ping-pong ball trick!" -- South Park the Movie

 

Edited by Jcloud
  • Like 1
Link to comment
5 hours ago, nuhll said:

(could this be run as "user script" (the plugin) after array starts)? Would be much easier and safer, correct!?

Yeah definitely could be used with the CA scripts and added to the array start. One thing I saw with that, is if you tinker with it, like shut it down/restart, or update it, then it doesn't hit that condition. But yes, you have the right idea for sure! I would also agree that using the CA Scripts and other available tools plugins is a good idea. 

Link to comment
24 minutes ago, L0rdRaiden said:

or do you plan to release a new version of the template for noobs?

Not soon, and given how templates are done, I'll either have to make a repo and do changes, or it will be a shell/bash script on the side (which is what I'm starting with now as it fits my comfort level of tweaking without making fubars). 

So long as you don't over-write files you should be fine, but IT IS YOUR THUMB DRIVE, so I need to do due diligence with a warning to be careful. Backup your Thumb drive; etc. 

In the end, do what you feel comfortable in doing, just take it step-by-step. 

If you're not comfortable writing the script file, manually making files and what not, I highly recommend (as does all the mods on forums) the user-script editor plugin/app from the Community Applications plugin installer. 

Link to comment
  • Jcloud changed the title to [Support] QDirStat, Jcloud - cryptoCoin templates

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.