[Support] - Storj v3 docker


MrChunky

Recommended Posts

Overview: Support for Docker image Storj v3 storage node.

Application: https://storj.io/

Docker Hub: https://hub.docker.com/r/storjlabs/storagenode

GitHub: https://github.com/storj/storj

Documentation: https://documentation.storj.io/

 

Storj forum: https://forum.storj.io/

 

This is Storj Lab's v3 node client. To participate you must first have to have an authorization token, to get the Authentication Token go here.

 

This template is for running the Docker application only, please follow Storj Lab directions in the documentation to generate your Node Identity certificates using the Authentication Token.

 

!!! IMPORTANT !!!
Two path need to be passed to the docker. Currently Storj requires that the path are mounted using --mount rather than -v. Do not use the Unraid docker template built in path variable.

 

Add the path for the storage and identity folders to the extra parameters in the format described below (extra parameters can be accessed through the advanced view toggle, in the very top right corner of the docker UI) During the first run the -e SETUP=true argument needs to appended before the --mount string to create the required folders. Therefore, on container creation (first run) the string in extra parameters should look like this:

-e SETUP=true --mount type=bind,source="/mnt/user/storj-certs/identity/storagenode/",destination=/app/identity  --mount type=bind,source="/mnt/user/storj/",destination=/app/config

Copy paste the whole string with altered values to match your setup. The identity folder should point directly to the folder where the certificate files are stored, and not a level above. The locations of the identity and storage folders should not be nested on the unRaid system to avoid permission errors. After the first run the -e SETUP=true argument should be deleted. The string should therefore be edited to look as follows:

--mount type=bind,source="/mnt/user/storj-certs/identity/storagenode/",destination=/app/identity  --mount type=bind,source="/mnt/user/storj/",destination=/app/config

 

 

 

Edited by MrChunky
Link to comment

Just discovered this on community applications and since I have a spare drive, a gig unmetered internet connection and static IP, I figured why not. But I can't get it to run. Just fails first thing. Any idea? I created my identity stuff on windows and moved it over, all the paths are right that I'm aware of.

Here's what it returns.

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='storagenode-v3' --net='host' -e TZ="America/Denver" -e HOST_OS="Unraid" -e 'TCP_PORT_28967'='28967' -e 'WALLET'='[Redacted]' -e 'EMAIL'='[Redacted]' -e 'ADDRESS'='[Redacted]:28967' -e 'BANDWIDTH'='20TB' -e 'STORAGE'='5TB' -e 'TCP_PORT_14002'='14002' --mount type=bind,source="/mnt/user/appdata/storj/identity/storagenode/",destination=/app/identity --mount type=bind,source="/mnt/user/StorjData/“,destination=/app/config 'storjlabs/storagenode:beta' 
  
The command failed.

 

Link to comment

I would guess that the --mount commands are failing, possibly because the folders are not found. Could you check the folder structures are indeed in place for both mount commands and that the relevant files are in there?

 

This is my how my run command looks:

 

run 
-d --name='StorjNode-V3' 
--net='bridge'
-e TZ="Europe/Berlin" 
-e HOST_OS="Unraid" 
-e 'WALLET'='xxx' 
-e 'EMAIL'='xxx' 
-e 'ADDRESS'='xxx:28968' 
-e 'BANDWIDTH'='10000TB' 
-e 'STORAGE'='12TB' 
-p '28968:28967/tcp' 
-p '14002:14002/tcp' 
--mount type=bind,source="/mnt/user/appdata/storj_cert/storj/identity/storagenode/",destination=/app/identity 
--mount type=bind,source="/mnt/user/storjv3/",destination=/app/config 
'storjlabs/storagenode:beta' 

 

Another is that my net is bridge, not host. But I doubt this is an issue.

Edited by MrChunky
Link to comment
  • 2 months later...

Hi! 
 

First off, thanks for providing this template!

 

 I was wondering how the update process works. As of this writing the docker hub images are updated less than one hour ago. And I’m urged to update in the GUI. The update from within Unraid does not show any available update. 

 

(from 1.1.1 to 1.3.3)
 

Do you as maintainer have to whitelist the update? 
 

... or is the endpoint storjlabs/storagenode:beta to old now when the project is out of beta? Pretty sure their own docs still points here. 


Thanks!

 


 

 

Edited by emilp
Clarification
Link to comment
20 minutes ago, emilp said:

Hi! 
 

First off, thanks for providing this template!

 

 I was wondering how the update process works. As of this writing the docker hub images are updated less than one hour ago. And I’m urged to update in the GUI. The update from within Unraid does not show any available update. 

 

(from 1.1.1 to 1.3.3)
 

Do you as maintainer have to whitelist the update? 
 

... or is the endpoint storjlabs/storagenode:beta to old now when the project is out of beta? Pretty sure their own docs still points here. 


Thanks!

 


 

 

The "beta" tag which this container uses was last updated by stoj 20 days ago.  (And the "latest" tag was last updated 7 months ago)

 

Presumably, the numerous other tags which are being updated (a new one is created ever update by stoj) are "pre-releases / alpha releases)

Link to comment
14 hours ago, emilp said:

Do you as maintainer have to whitelist the update? 

Hi emilp,

 

As Squid mentioned, the update is reliant on proper tag (beta/ latest) maintenance by storj. All I did was create a template that allows you to use the official storj docker image. As you pointed out the beta tag is still in the documentation and provides the latest beta versions. So when it is updated, UnRaid docker will update the container on a schedule, if you have the plugin, or you can manually update it. Also in the storage node GUI it mentions that the minimum allowed is 1.1.0, so i think you should be fine for a while.

Link to comment
On 4/28/2020 at 7:21 PM, Maggi0r said:

First of all thank you for this container,

However, my container does not come online.
I created my identity and adjusted everything in the container settings.

Hope you have an idea.

Regards Maggi

Hi Maggi,

Could you post your logs output and a screenshot of your config page for the docker or the run command that appears when you start it.

Link to comment
  • 4 months later...

Mine doesn't work either. One diff: I'm pulling latest vs. beta

 

This is the error after attempting to add the container: 

 

-v '/mnt/user/BlackTowerMain/storj/identity/storagenode/':'/app/identity/':'rw' -v '/mnt/user/BlackTowerMain/storj/':'/app/config/':'rw' --mount type=bind,source="/mnt/user/[your identity directory]/identity/storagenode/",destination=/app/identity --mount type=bind,source="/mnt/user/[your storage directory]/",destination=/app/config 'storjlabs/storagenode:latest' 

/usr/bin/docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /mnt/user/[your identity directory]/identity/storagenode/.

 

Something to do with [your identity directory] perhaps?

 

image.png.241cd9533058b4d5ea3653a063f59bea.png

 

since it doesn't fit: /mnt/user/BlackTowerMain/storj/identity/storagenode/

 

Files are there :

 

root@BlackTower:/mnt/user/BlackTowerMain/storj/identity/storagenode# ls
ca.xxxxxx.cert  ca.cert  ca.key  identity.xxx.cert  identity.cert  identity.key

 

root@BlackTower:/mnt/user/BlackTowerMain/storj# ls
config.yaml  identity/  revocations.db  run\ commands  storage/  trust-cache.json

 

Kinda out of ideas for the template. Next I guess I'll just do it via CLI

 

 

 

Link to comment
  • 4 weeks later...
On 9/19/2020 at 1:48 AM, RealActorRob said:

Mine doesn't work either. One diff: I'm pulling latest vs. beta

 

This is the error after attempting to add the container: 

 

-v '/mnt/user/BlackTowerMain/storj/identity/storagenode/':'/app/identity/':'rw' -v '/mnt/user/BlackTowerMain/storj/':'/app/config/':'rw' --mount type=bind,source="/mnt/user/[your identity directory]/identity/storagenode/",destination=/app/identity --mount type=bind,source="/mnt/user/[your storage directory]/",destination=/app/config 'storjlabs/storagenode:latest' 

/usr/bin/docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /mnt/user/[your identity directory]/identity/storagenode/.

 

Something to do with [your identity directory] perhaps?

 

image.png.241cd9533058b4d5ea3653a063f59bea.png

 

since it doesn't fit: /mnt/user/BlackTowerMain/storj/identity/storagenode/

 

Files are there :

 

root@BlackTower:/mnt/user/BlackTowerMain/storj/identity/storagenode# ls
ca.xxxxxx.cert  ca.cert  ca.key  identity.xxx.cert  identity.cert  identity.key

 

root@BlackTower:/mnt/user/BlackTowerMain/storj# ls
config.yaml  identity/  revocations.db  run\ commands  storage/  trust-cache.json

 

Kinda out of ideas for the template. Next I guess I'll just do it via CLI

 

 

 

Sorry for the late reply. You needed to add the storage via the --mount flag in the extra parameters field as described in the documentation and docker description. That's why you were getting the error.

Edited by MrChunky
Link to comment
3 hours ago, MrChunky said:

Sorry for the late reply. You needed to add thee storage via the --mount flag in the extra parameters feald as described in the documentation and docker description. That's why you were getting the error.

There's nothing labeled as 'Extra Parameters' anywhere. Perhaps can you expand on that with a screenshot?

Link to comment
On 10/14/2020 at 11:32 PM, RealActorRob said:

There's nothing labeled as 'Extra Parameters' anywhere. Perhaps can you expand on that with a screenshot?

Sure, here you go. In the top right you need to toggle the Advanced View to see the extra parameters.

Capture1.thumb.PNG.6bc6735f9ef5cacc1fe87aaf1290bb3e.PNG

 

Capture2.thumb.PNG.7a413078a3eda92a329e602a7a3fafc6.PNG

 

Edited by MrChunky
Link to comment

I'm missing something...Here's the template Command from your template:

 

/usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='storagenode2' --net='bridge' -e TZ="America/Chicago" -e HOST_OS="Unraid" -e 'WALLET'='0x111111111' -e 'EMAIL'='xxx@xxxxx.com' -e 'ADDRESS'='xxxx.xxx.xxx:28967' -e 'BANDWIDTH'='100000TB' -e 'STORAGE'='2200GB' -p '28967:28967/tcp' -p '14002:14002/tcp' -v '/mnt/user/storjnode2/identity/':'/app/identity \':'rw' -v '/mnt/user/storjnode2/':'/app/config \':'rw' --mount type=bind,source="/mnt/user/storjnode2/identity/",destination=/app/identity \ --mount type=bind,source="/mnt/user/storjnode2/",destination=/app/config \ 'storjlabs/storagenode:latest'

 

 

/usr/bin/docker: invalid reference format.
See '/usr/bin/docker run --help'.

 

I removed the -v flags from the template as those are deprecated.

 

Now the template generates:

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='storagenode2' --net='bridge' -e TZ="America/Chicago" -e HOST_OS="Unraid" -e 'WALLET'='0x123456789' -e 'EMAIL'='b@b.com' -e 'ADDRESS'='dummy.ddns.net:28967' -e 'BANDWIDTH'='100000TB' -e 'STORAGE'='2200GB' -p '28967:28967/tcp' -p '14002:14002/tcp' --mount type=bind,source="/mnt/user/storjnode2/identity/",destination=/app/identity \ --mount type=bind,source="/mnt/user/storjnode2/",destination=/app/config \ 'storjlabs/storagenode:latest'

/usr/bin/docker: invalid reference format.
See '/usr/bin/docker run --help'.

 

Can you post your command output @MrChunky or tell me the next step?

 

At any rate, the template needs updating to remove -v generation.

 

Link to comment
21 minutes ago, RealActorRob said:

At any rate, the template needs updating to remove -v generation.

 

Not sure which template you are referring to here.

 

21 minutes ago, RealActorRob said:

Can you post your command output @MrChunky or tell me the next step?

/usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker create 
--name='storagenode-v3' 
--net='bridge' 
-e TZ="Europe/Berlin" 
-e HOST_OS="Unraid" 
-e 'WALLET'='0xxxx' 
-e 'EMAIL'='xxx@gmail.com' 
-e 'ADDRESS'='www.xxx.com:28969' 
-e 'STORAGE'='8TB' 
-e 'BANDWIDTH'='10000TB' 
-p '28969:28967/tcp' 
-p '14003:14002/tcp' 
--mount type=bind,source="/mnt/user/appdata/storj_cert/identity/storagenode/",destination=/app/identity 
--mount type=bind,source="/mnt/user/storj/",destination=/app/config 
'storjlabs/storagenode:beta'

In your case you have trailing backslashes: " \ " after the mount commands that I don't think should be there. When you paste the mount commands into the extra parameters, they should look like this:

 

--mount type=bind,source="/mnt/user/appdata/storj_cert/identity/storagenode/",destination=/app/identity  --mount type=bind,source="/mnt/user/storj/",destination=/app/config

 

The docker tag is latest, mine is beta. In addition, make sure the mount command for the identity folder should point directly to the folder with the certificates, and not a level above.

Edited by MrChunky
Link to comment

@MrChunky The trailing backslashes are from the official storj guide at https://documentation.storj.io/setup/cli/storage-node

 

And when I paste my command like they do in the terminal window they work and the node starts and comes online. So maybe the script is munging it because of the trailing backslash?

 

My command: (obviously this is a second node on a diff port but it works so that's fine)

 

docker run -d --restart unless-stopped --stop-timeout 300 \
    -p 28968:28967 \
    -p 192.168.1.175:14003:14002 \
    -e WALLET="0x123" \
    -e EMAIL="xxx@xxx.com" \
    -e ADDRESS="xxx.ddns.net:28968" \
    -e STORAGE="500GB" \
    --mount type=bind,source="/mnt/user/BlackTowerMain/storj/identity/storagenode/",destination=/app/identity \
    --mount type=bind,source="/mnt/user/BlackTowerMain/storj/",destination=/app/config \
    --name storagenode storjlabs/storagenode:latest

Link to comment
4 minutes ago, RealActorRob said:

@MrChunky The trailing backslashes are from the official storj guide at https://documentation.storj.io/setup/cli/storage-node

 

And when I paste my command like they do in the terminal window they work and the node starts and comes online. So maybe the script is munging it because of the trailing backslash?

 

My command: (obviously this is a second node on a diff port but it works so that's fine)

 

docker run -d --restart unless-stopped --stop-timeout 300 \
    -p 28968:28967 \
    -p 192.168.1.175:14003:14002 \
    -e WALLET="0x123" \
    -e EMAIL="xxx@xxx.com" \
    -e ADDRESS="xxx.ddns.net:28968" \
    -e STORAGE="500GB" \
    --mount type=bind,source="/mnt/user/BlackTowerMain/storj/identity/storagenode/",destination=/app/identity \
    --mount type=bind,source="/mnt/user/BlackTowerMain/storj/",destination=/app/config \
    --name storagenode storjlabs/storagenode:latest

The docker run format that you are using here is different then the xml format Unraid docker templates are using. You should remove the backslashes in the template.

Edited by MrChunky
Link to comment
6 minutes ago, MrChunky said:

The docker run format that you are using here is different then the xml format Unraid docker templates are using. You should remove the backslashes in the template.

Ok that was the last issue. Node is started and pulling/pushing data per the logs.

 

Thanks @MrChunky. The only things I'd suggest on the template are:

 

  • removing the generation of the deprecated -v flags
  • help note about no trailing slashes for the mounts since the shell command needs those so that confused me...maybe others
  • help note about selecting 'advanced view'

 

Thanks again.

 

 

Edited by RealActorRob
Link to comment
21 minutes ago, RealActorRob said:

Ok that was the last issue. Node is started and pulling/pushing data per the logs.

 

Thanks @MrChunky. The only things I'd suggest on the template are:

 

  • removing the generation of the deprecated -v flags
  • help note about no trailing slashes for the mounts since the shell command needs those so that confused me...maybe others
  • help note about selecting 'advanced view'

 

Thanks again.

 

 

No Problem. 

 

  • I don't think the -v flags are there normally, at least when I pull the template they are not there.
  • Will do.
  • Already in the description.
Edited by MrChunky
Link to comment
3 hours ago, RealActorRob said:

I believe it's this portion of the template that generates the -v flags @MrChunky This part certainly isn't needed in the template since it's passed via --mount anyway.

 

 

image.thumb.png.b4e343d73cc112c217aa6ebe0f23dbf4.png

These two variables are not a part of the template I am maintaining. Here is the template XML from which it is generated. there are no -v fields.

Edited by MrChunky
Link to comment

For other migrating. Slightly confusing, make sure you use the correct path for storage, i.e., without '/storage' at the end per:

 

https://documentation.storj.io/resources/faq/migrate-my-node

 

"Please, note - we intentionally specified/mnt/storj2/storagenode-new as the data source in the --mount parameter and not /mnt/storj2/storagenode-new/storage because the storagenode docker container will add a subfolder called storage to the path automatically. So please, make sure that your data folder contains a storage subfolder with all the data inside (blobs folder, database files, etc.), otherwise the node will start from scratch since it can't find the data in the right subfolder and will be disqualified in a few hours."

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.