[Support] - Storj v3 docker


Recommended Posts

  • 3 months later...

hey guys cant get this to work, geting this:

 

2021-09-24T03:09:59.538Z ERROR Failed to load identity. {"error": "file or directory not found: open identity/identity.cert: no such file or directory", "errorVerbose": "file or directory not found: open identity/identity.cert: no such file or directory\n\tstorj.io/common/identity.Config.Load:329\n\tmain.cmdRun:149\n\tstorj.io/private/process.cleanup.func1.4:363\n\tstorj.io/private/process.cleanup.func1:381\n\tgithub.com/spf13/cobra.(*Command).execute:852\n\tgithub.com/spf13/cobra.(*Command).ExecuteC:960\n\tgithub.com/spf13/cobra.(*Command).Execute:897\n\tstorj.io/private/process.ExecWithCustomConfig:88\n\tstorj.io/private/process.ExecCustomDebug:70\n\tmain.main:385\n\truntime.main:204"}
Error: Failed to load identity: file or directory not found: open identity/identity.cert: no such file or directory

 

using this

 

-v type=bind,source="/mnt/user/appdata//storj/identity/storagenode/",destination=/app/identity  --mount type=bind,source="/mnt/user//Storj/",destination=/app/config

 

and --mount wont work at all -v gets the container to show up but wont start

and i did try the -e SETUP=true with -v and -mount

 

the files are in appdata / storj / identity / storagenode

 

dont know why its not working if you guys can see something that im missing that would be nice.

Link to comment
  • 4 weeks later...

Hi everyone, this thread has been super helpful but I'm getting stuck somewhere and I can't find any solutions. I ended up following along the YT video posted by geeksheik and after creating the Docker container, it says 'success' but none of the contents of the two folders get altered they do in the video (thus leading to the error posted below). I've tried using -e SETUP="true" as well as -e SETUP=true in the extra parameters line. Here is the error if I try to run the container: 

 

Error: Error starting master database on storagenode: group:
--- stat config/storage/blobs: no such file or directory
--- stat config/storage/temp: no such file or directory
--- stat config/storage/garbage: no such file or directory
--- stat config/storage/trash: no such file or directory

 

This is the full string I'm using in the extra parameters: "-e SETUP=true --mount type=bind,source="/mnt/user/Storj1/identity/storagenode/",destination=/app/identity --mount type=bind,source="/mnt/user/Storj1/data",destination=/app/config"

 

Any help would be greatly appreciated. 

Edited by Ignatious11
more details added
Link to comment
On 1/17/2021 at 8:28 PM, srfnmnk said:

First time start up needs to use - `-e SETUP=true` 

It will not start then either, but then you need to go back to the extra parameters and remove that again and start it back up. This is to provision the initial requirements.

 

Wondering if you might be able to help me: I've done exactly what you stated here but the initial provisions are not initialized when I use '-e SETUP=true' nor if I try '-e SETUP="true"'.... I can see that no folders are created in the two shares I've created and mounted with the container. If I try to run the container after removing the -e command I get the following: 

 

Error: Error starting master database on storagenode: group:
--- stat config/storage/blobs: no such file or directory
--- stat config/storage/temp: no such file or directory
--- stat config/storage/garbage: no such file or directory
--- stat config/storage/trash: no such file or directory

 

any ideas? 

Link to comment
On 10/20/2021 at 3:58 PM, Ignatious11 said:

 

Wondering if you might be able to help me: I've done exactly what you stated here but the initial provisions are not initialized when I use '-e SETUP=true' nor if I try '-e SETUP="true"'.... I can see that no folders are created in the two shares I've created and mounted with the container. If I try to run the container after removing the -e command I get the following: 

 

Error: Error starting master database on storagenode: group:
--- stat config/storage/blobs: no such file or directory
--- stat config/storage/temp: no such file or directory
--- stat config/storage/garbage: no such file or directory
--- stat config/storage/trash: no such file or directory

 

any ideas? 

Nothing immediate comes to mind. Can you post log output when you run it the first time with setup option. Did you generate your certificates correctly?

Link to comment
On 10/24/2021 at 7:25 PM, MrChunky said:

Nothing immediate comes to mind. Can you post log output when you run it the first time with setup option. Did you generate your certificates correctly?

Thanks for getting back to me! Good question, I'm not sure if the certs were done 'correctly' as I've had them for a while now. I think I'll start from scratch again and see if that fixes it. If not, I'll post the log on here. Thanks again! 

Link to comment
On 10/24/2021 at 7:25 PM, MrChunky said:

Nothing immediate comes to mind. Can you post log output when you run it the first time with setup option. Did you generate your certificates correctly?

Okay so I've made some progress but now I cannot get into the docker container to authorize the identity because it immediately shuts down ("gracefulexit") after starting. Here's the latest from the log: 

 

2021-10-29T20:04:22.694Z INFO contact:service context cancelled {"Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2021-10-29T20:04:22.694Z ERROR contact:service ping satellite failed {"Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 1, "error": "ping satellite: rpc: dial tcp: operation was canceled", "errorVerbose": "ping satellite: rpc: dial tcp: operation was canceled\n\tstorj.io/common/rpc.TCPConnector.DialContextUnencrypted:107\n\tstorj.io/common/rpc.TCPConnector.DialContext:71\n\tstorj.io/common/rpc.Dialer.dialEncryptedConn:220\n\tstorj.io/common/rpc.Dialer.DialNodeURL.func1:110\n\tstorj.io/common/rpc/rpcpool.(*Pool).get:105\n\tstorj.io/common/rpc/rpcpool.(*Pool).Get:128\n\tstorj.io/common/rpc.Dialer.dialPool:186\n\tstorj.io/common/rpc.Dialer.DialNodeURL:109\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:124\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:95\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.

(*Cycle).Run:92\n\tstorj.io/common/sync2.(*Cycle).Start.func1:71\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

 

2021-10-29T20:04:22.694Z INFO contact:service context cancelled {"Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
2021-10-29T20:04:22.694Z ERROR gracefulexit:blobscleaner couldn't receive satellite's GE status {"error": "context canceled"}

 

Error: piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory

Link to comment
  • 1 month later...

I'm trying to take a Graceful Exit, but cant get it to work.

https://support.storj.io/hc/en-us/articles/360039432051-Graceful-Exit-Guide


docker exec -it storagenode /app/storagenode exit-satellite --config-dir /app/config --identity-dir /app/identity

docker exec -it storagenode /app/storagenode exit-status --config-dir /app/config --identity-dir /app/identity

 

I have come up with this, but just getting error.

docker exec -it storagenode-v3 "/mnt/disk16/NetBackup/storj/storagenode/" exit-satellite --config-dir "/mnt/disk1/storj/" --identity-dir "/mnt/disk16/NetBackup/storj/storagenode/"

 

OCI runtime exec failed: exec failed: container_linux.go:367: starting container process caused: exec: "/mnt/disk16/NetBackup/storj/storagenode/": stat /mnt/disk16/NetBackup/storj/storagenode/: no such file or directory: unknown

 

Anyone help?

Link to comment
On 12/5/2021 at 11:46 AM, ChillZwix said:

I'm trying to take a Graceful Exit, but cant get it to work.

https://support.storj.io/hc/en-us/articles/360039432051-Graceful-Exit-Guide


docker exec -it storagenode /app/storagenode exit-satellite --config-dir /app/config --identity-dir /app/identity

docker exec -it storagenode /app/storagenode exit-status --config-dir /app/config --identity-dir /app/identity

 

I have come up with this, but just getting error.

docker exec -it storagenode-v3 "/mnt/disk16/NetBackup/storj/storagenode/" exit-satellite --config-dir "/mnt/disk1/storj/" --identity-dir "/mnt/disk16/NetBackup/storj/storagenode/"

 

OCI runtime exec failed: exec failed: container_linux.go:367: starting container process caused: exec: "/mnt/disk16/NetBackup/storj/storagenode/": stat /mnt/disk16/NetBackup/storj/storagenode/: no such file or directory: unknown

 

Anyone help?

 

 

Found out how to do this:

 

docker exec -it storagenode-v3 /app/storagenode exit-satellite --config-dir /app/config --identity-dir /app/identity

 

docker exec -it storagenode-v3 /app/storagenode exit-status --config-dir /app/config --identity-dir /app/identity

Link to comment
10 hours ago, ChillZwix said:

 

 

Found out how to do this:

 

docker exec -it storagenode-v3 /app/storagenode exit-satellite --config-dir /app/config --identity-dir /app/identity

 

docker exec -it storagenode-v3 /app/storagenode exit-status --config-dir /app/config --identity-dir /app/identity

Thanks for sharing, I'll add it to the description.

Link to comment
On 10/29/2021 at 4:59 PM, Ignatious11 said:

Okay so I've made some progress but now I cannot get into the docker container to authorize the identity because it immediately shuts down ("gracefulexit") after starting. Here's the latest from the log: 

 

2021-10-29T20:04:22.694Z INFO contact:service context cancelled {"Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6"}
2021-10-29T20:04:22.694Z ERROR contact:service ping satellite failed {"Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB", "attempts": 1, "error": "ping satellite: rpc: dial tcp: operation was canceled", "errorVerbose": "ping satellite: rpc: dial tcp: operation was canceled\n\tstorj.io/common/rpc.TCPConnector.DialContextUnencrypted:107\n\tstorj.io/common/rpc.TCPConnector.DialContext:71\n\tstorj.io/common/rpc.Dialer.dialEncryptedConn:220\n\tstorj.io/common/rpc.Dialer.DialNodeURL.func1:110\n\tstorj.io/common/rpc/rpcpool.(*Pool).get:105\n\tstorj.io/common/rpc/rpcpool.(*Pool).Get:128\n\tstorj.io/common/rpc.Dialer.dialPool:186\n\tstorj.io/common/rpc.Dialer.DialNodeURL:109\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:124\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:95\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.

(*Cycle).Run:92\n\tstorj.io/common/sync2.(*Cycle).Start.func1:71\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

 

2021-10-29T20:04:22.694Z INFO contact:service context cancelled {"Satellite ID": "12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB"}
2021-10-29T20:04:22.694Z ERROR gracefulexit:blobscleaner couldn't receive satellite's GE status {"error": "context canceled"}

 

Error: piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory

I'm getting this same exact error while trying to start my node.  Has anyone had any luck fixing this?  

Link to comment
  • 4 weeks later...
On 12/10/2021 at 1:42 PM, craig2583 said:

I'm getting this same exact error while trying to start my node.  Has anyone had any luck fixing this?  

I had this issue.  Its DNS failures.  If you have PiHole installed try adding some more DNS servers under settings.  I have Cloudflare & Google.

Link to comment
  • 2 weeks later...

I got the docker setup, and running. but i am getting an error after docker starts.  any suggestions on how to resolve this?

 

e":"No such container: 0889798417f7"}
e":"No such container: 0889798417f7"}
e":"No such container: 0889798417f7"}
e":"No such container: 0889798417f7"}
and so on....  it just repeats over and over.

Link to comment
4 minutes ago, Squid said:

That's the log window, and since it was opened, you installed an update (or removed and reinstalled) the container and the original reference no longer exists.  Perfectly normal.  Reopen it

Thanks.  I see the log files now.  the docker has shutdown itself.  here are the log files showing what looks like 2 errors: storage directory and satellite failed.

 

2022-01-16T16:02:25.460Z INFO Public server started on [::]:28967
2022-01-16T16:02:25.460Z INFO Private server started on 127.0.0.1:7778


2022-01-16T16:02:25.460Z ERROR services unexpected shutdown of a runner {"name": "piecestore:monitor", "error": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory", "errorVerbose": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1.1:133\n\tstorj.io/common/sync2.(*Cycle).Run:92\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1:130\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}


2022-01-16T16:02:25.461Z ERROR nodestats:cache Get pricing-model/join date failed {"error": "context canceled"}


2022-01-16T16:02:25.461Z ERROR contact:service ping satellite failed {"Satellite ID": "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6", "attempts": 1, "error": "ping satellite: rpc: dial tcp: operation was canceled", "errorVerbose": "ping satellite: rpc: dial tcp: operation was canceled\n\tstorj.io/common/rpc.TCPConnector.DialContextUnencrypted:114\n\tstorj.io/common/rpc.TCPConnector.DialContext:78\n\tstorj.io/common/rpc.Dialer.dialEncryptedConn:220\n\tstorj.io/common/rpc.Dialer.DialNodeURL.func1:110\n\tstorj.io/common/rpc/rpcpool.(*Pool).get:105\n\tstorj.io/common/rpc/rpcpool.(*Pool).Get:128\n\tstorj.io/common/rpc.Dialer.dialPool:186\n\tstorj.io/common/rpc.Dialer.DialNodeURL:109\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:124\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:95\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:92\n\tstorj.io/common/sync2.(*Cycle).Start.func1:71\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}


2022-01-16T16:02:25.461Z INFO contact:service context cancelled {"Satellite ID": "121RTSxxxx.xxxx.....xxxxx1tuvgk3vzoA6"}
2022-01-16T16:02:25.461Z ERROR piecestore:cache error during init space usage db: {"error": "piece space used: context canceled", "errorVerbose": "piece space used: context canceled\n\tstorj.io/storj/storagenode/storagenodedb.(*pieceSpaceUsedDB).Init:73\n\tstorj.io/storj/storagenode/pieces.(*CacheService).Run:75\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2.1:87\n\truntime/pprof.Do:40\n\tstorj.io/storj/private/lifecycle.(*Group).Run.func2:86\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

Link to comment
5 minutes ago, SWN said:

"piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory"

 

Suggest there is an issue with the storage path. Have a look at your config and setup instructions.

Link to comment
10 hours ago, MrChunky said:

 

Suggest there is an issue with the storage path. Have a look at your config and setup instructions.

here are my settings in the docker on Unraid 6.9.2:

--mount type=bind,source="/mnt/user/appdata/storj/identity/storagenode/",destination=/app/identity  --mount type=bind,source="/mnt/disks/d3pjf/storj/",destination=/app/config

 

Error:

Error: piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory

 

I am pointing the storage device to unassigned disks that are mounted and formatted to XFS.

I created a folder on the disk (dev 3, sde, d3pjf) called storj per the instructions from Storj CLI tutorial.  Here is my directory listing at storage destination.

root@CALRISSIAN:/mnt/disks/d3pjf/storj# ls -l
total 44
-rw------- 1 root root  9355 Jan 16 00:29 config.yaml
drwx------ 4 root root    47 Jan 16 09:02 orders/
-rw------- 1 root root 32768 Jan 16 19:48 revocations.db
drwx------ 6 root root  4096 Jan 16 19:48 storage/
-rw------- 1 root root  1374 Jan 16 19:48 trust-cache.json
 

I can see all the files that Storj created during the install.  I am confused as to why it is not able to point here now that docker is installed.

 

Link to comment

Here are more log files.  these are the first lines after a boot of docker:

2022-01-17T03:10:55.851Z INFO Configuration loaded {"Location": "/app/config/config.yaml"}
2022-01-17T03:10:55.852Z INFO Operator email {"Address": "xxxxxx"}
2022-01-17T03:10:55.853Z INFO Operator wallet {"Address": "xxxxxxx"}
2022-01-17T03:10:56.084Z INFO Telemetry enabled {"instance ID": "xxxxxx"}
2022-01-17T03:10:56.150Z INFO db.migration Database Version {"version": 53}

2022-01-17T03:10:56.717Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2022-01-17T03:10:57.240Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2022-01-17T03:10:57.240Z INFO bandwidth Performing bandwidth usage rollups
2022-01-17T03:10:57.240Z INFO trust Scheduling next refresh {"after": "4h58m9.068406927s"}
2022-01-17T03:10:57.240Z INFO Node xxxxxx started
2022-01-17T03:10:57.240Z INFO Public server started on [::]:28967
2022-01-17T03:10:57.240Z INFO Private server started on 127.0.0.1:7778
2022-01-17T03:10:57.241Z ERROR services unexpected shutdown of a runner {"name": "piecestore:monitor", "error": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory", "errorVerbose": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1.1:133\n\tstorj.io/common/sync2.(*Cycle).Run:92\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1:130\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

 

piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory

 

Why is it looking for this file or directory called 'storage-dir-verification'?  it doesn't exist in my storage folder.

 

Link to comment
1 hour ago, SWN said:

Here are more log files.  these are the first lines after a boot of docker:

2022-01-17T03:10:55.851Z INFO Configuration loaded {"Location": "/app/config/config.yaml"}
2022-01-17T03:10:55.852Z INFO Operator email {"Address": "xxxxxx"}
2022-01-17T03:10:55.853Z INFO Operator wallet {"Address": "xxxxxxx"}
2022-01-17T03:10:56.084Z INFO Telemetry enabled {"instance ID": "xxxxxx"}
2022-01-17T03:10:56.150Z INFO db.migration Database Version {"version": 53}

2022-01-17T03:10:56.717Z INFO preflight:localtime start checking local system clock with trusted satellites' system clock.
2022-01-17T03:10:57.240Z INFO preflight:localtime local system clock is in sync with trusted satellites' system clock.
2022-01-17T03:10:57.240Z INFO bandwidth Performing bandwidth usage rollups
2022-01-17T03:10:57.240Z INFO trust Scheduling next refresh {"after": "4h58m9.068406927s"}
2022-01-17T03:10:57.240Z INFO Node xxxxxx started
2022-01-17T03:10:57.240Z INFO Public server started on [::]:28967
2022-01-17T03:10:57.240Z INFO Private server started on 127.0.0.1:7778
2022-01-17T03:10:57.241Z ERROR services unexpected shutdown of a runner {"name": "piecestore:monitor", "error": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory", "errorVerbose": "piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1.1:133\n\tstorj.io/common/sync2.(*Cycle).Run:92\n\tstorj.io/storj/storagenode/monitor.(*Service).Run.func1:130\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

 

piecestore monitor: error verifying location and/or readability of storage directory: open config/storage/storage-dir-verification: no such file or directory

 

Why is it looking for this file or directory called 'storage-dir-verification'?  it doesn't exist in my storage folder.

 

i mounted the folder for storage, deleted the docker, re installed it.  i am not getting this error anymore.

used this tutorial to mount the folder.

https://docs.storj.io/node/resources/faq/linux-static-mount

 

I am having another problem now.  I will post another thread.

Link to comment

2022-01-17T04:27:26.232Z ERROR contact:service ping satellite failed {"Satellite ID": "xxxx removed for security.....", "attempts": 7, "error": "ping satellite: check-in ratelimit: node rate limited by id", "errorVerbose": "ping satellite: check-in ratelimit: node rate limited by id\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:138\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:95\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:92\n\tstorj.io/common/sync2.(*Cycle).Start.func1:71\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:57"}

 

any thoughts on this error?

Not sure how to start troubleshooting this one.

thanks.

Link to comment

It means that your node cannot reach the satellite for some reason. I would recommend to have a look at the threads on storj.io or make a new thread on the forums there. People there have much more experience debugging storj related errors, as networking is most likely not an unraid issue.

Edited by MrChunky
Link to comment
  • 2 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.