hi and thank you for making this tool available in Unraid.
I can't get it working. Today was my first time to try it.
Below is output from the "docker logs hoarder" command:
root@dunraid:~# docker logs hoarder
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service init-db-migration: starting
Running db migration script
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-db-migration successfully started
s6-rc: info: service svc-workers: starting
s6-rc: info: service svc-web: starting
s6-rc: info: service svc-workers successfully started
s6-rc: info: service svc-web successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
▲ Next.js 14.2.21
- Local: http://localhost:3000
- Network: http://0.0.0.0:3000
✓ Starting...
✓ Ready in 372ms
> @hoarder/
[email protected] start:prod /app/apps/workers
> tsx index.ts
2025-01-24T04:45:08.100Z info: Workers version: 0.21.0
2025-01-24T04:45:08.116Z info: [crawler] Loading adblocker ...
(node:120) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
2025-01-24T04:45:09.948Z info: [Crawler] Browser connect on demand is enabled, won't proactively start the browser instance
2025-01-24T04:45:09.948Z info: Starting crawler worker ...
2025-01-24T04:45:09.948Z info: Starting inference worker ...
2025-01-24T04:45:09.949Z info: Starting search indexing worker ...
2025-01-24T04:45:09.949Z info: Starting tidy assets worker ...
2025-01-24T04:45:09.949Z info: Starting video worker ...
2025-01-24T04:45:09.950Z info: Starting feed worker ...
2025-01-24T04:45:09.950Z info: Starting asset preprocessing worker ...
(node:68) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
2025-01-24T04:45:16.003Z info: [Crawler][35] Will crawl "hhttps://www.website.article/article.html" for link with id "e0kkyoioazqbyyqryip8cci6"
2025-01-24T04:45:16.004Z info: [Crawler][35] Attempting to determine the content-type for the url https://www.website.article/article.html.html
2025-01-24T04:45:16.368Z info: [Crawler][35] Content-type for the url https://www.website.article/article.html is "text/html; charset=utf-8"
2025-01-24T04:45:16.369Z info: [Crawler] Connecting to existing browser websocket address: ws://localhost:3000?token=2BR6DgQzZL8md4Bk5rewy3K9k
2025-01-24T04:45:16.425Z error: [Crawler][35] Crawling job failed: [object Object]
undefined
MeiliSearchCommunicationError: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
code: 'INTERNAL_SERVER_ERROR',
name: 'TRPCError',
[cause]: o [MeiliSearchCommunicationError]: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
errno: undefined,
code: undefined
}
}
MeiliSearchCommunicationError: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
code: 'INTERNAL_SERVER_ERROR',
name: 'TRPCError',
[cause]: o [MeiliSearchCommunicationError]: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
errno: undefined,
code: undefined
}
}
MeiliSearchCommunicationError: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
code: 'INTERNAL_SERVER_ERROR',
name: 'TRPCError',
[cause]: o [MeiliSearchCommunicationError]: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
errno: undefined,
code: undefined
}
}
MeiliSearchCommunicationError: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
code: 'INTERNAL_SERVER_ERROR',
name: 'TRPCError',
[cause]: o [MeiliSearchCommunicationError]: fetch failed
at node:internal/deps/undici/undici:13484:13
at process.processTicksAndRejections (node:internal/process/task_queues:105:5) {
errno: undefined,
code: undefined
}
}
2025-01-24T04:45:44.346Z info: [Crawler][36] Will crawl "https://www.website.article/article.html" for link with id "e0kkyoioazqbyyqryip8cci6"
2025-01-24T04:45:44.346Z info: [Crawler][36] Attempting to determine the content-type for the url hhttps://www.website.article/article.html
2025-01-24T04:45:44.572Z info: [Crawler][36] Content-type for the url https://www.website.article/article.html is "text/html; charset=utf-8"
2025-01-24T04:45:44.572Z info: [Crawler] Connecting to existing browser websocket address: ws://localhost:3000?token=2BR6DgQzZL8dm4Bk5rewy3K9k
2025-01-24T04:45:44.577Z error: [Crawler][36] Crawling job failed: [object Object]
undefined
below is the output from the "docker logs meilisearch:"
Config file path: "none"
Database path: "./data.ms"
Server listening on: "http://0.0.0.0:7700"
Environment: "production"
Commit SHA: "0df84bbba7db46680e38843dd6454257161a282f"
Commit date: "2024-06-27T12:47:08Z"
Package version: "1.9.0"
Thank you for using Meilisearch!
We collect anonymized analytics to improve our product and your experience. To learn more, including how to turn off analytics, visit our dedicated documentation page: https://www.meilisearch.com/docs/learn/what_is_meilisearch/telemetry
Anonymous telemetry: "Enabled"
Instance UID: "33c338a4-e12a-438a-b95f-65ceceb7aa6b"
A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
Check out Meilisearch Cloud! https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
Documentation: https://www.meilisearch.com/docs
Source code: https://github.com/meilisearch/meilisearch
Discord: https://discord.meilisearch.com
2025-01-24T04:31:26.380773Z INFO actix_server::builder: starting 4 workers
2025-01-24T04:31:26.380810Z INFO actix_server::server: Actix runtime found; starting in Actix runtime
and browserless-v2:
root@unraid:~# docker logs browserless-v2
browserless.io:limiter:info Concurrency: 10 queue: 10 timeout: 300000ms +0ms
browserless.io:index:info
---------------------------------------------------------
| browserless.io
| To read documentation and more, load in your browser:
|
| OpenAPI: http://0.0.0.0:3000/docs
| Full Documentation: https://docs.browserless.io/
---------------------------------------------------------
█▓▒
████▒
████▒
████▒ ▒██▓▒
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒██████▓▒
████▒ ▒██████████▒
████▒ ▒██████▓████
████▒ ▒█▓▓▒ ▒████
████▒ ▒████
████▒ ▒▓██████
████▒ ▒▓████████▓▒
████▓▓████████▓▒
██████████▓▒
▓███▓▒
+0ms
browserless.io:index:info Running as user "blessuser" +0ms
browserless.io:index:info Starting import of HTTP Routes +0ms
browserless.io:index:info Starting import of WebSocket Routes +61ms
browserless.io:router:trace Registering HTTP POST /content?(/),/chromium/content?(/) +0ms
browserless.io:router:trace Registering HTTP POST /download?(/),/chromium/download?(/) +0ms
browserless.io:router:trace Registering HTTP POST /function?(/),/chromium/function?(/) +0ms
browserless.io:router:trace Registering HTTP GET /json/list?(/) +0ms
browserless.io:router:trace Registering HTTP PUT /json/new?(/) +1ms
browserless.io:router:trace Registering HTTP GET /json/protocol?(/) +0ms
browserless.io:router:trace Registering HTTP GET /json/version?(/) +0ms
browserless.io:router:trace Registering HTTP POST /pdf?(/),/chromium/pdf?(/) +0ms
browserless.io:router:trace Registering HTTP POST /performance?(/),/chromium/performance?(/) +0ms
browserless.io:router:trace Registering HTTP POST /scrape?(/),/chromium/scrape?(/) +0ms
browserless.io:router:trace Registering HTTP POST /screenshot?(/),/chromium/screenshot?(/) +0ms
browserless.io:router:trace Registering HTTP GET /active?(/) +0ms
browserless.io:router:trace Registering HTTP GET /config?(/) +0ms
browserless.io:router:trace Registering HTTP GET /kill/+([0-9a-zA-Z-_])?(/) +0ms
browserless.io:router:trace Registering HTTP GET /meta?(/) +0ms
browserless.io:router:trace Registering HTTP GET /metrics/total?(/) +0ms
browserless.io:router:trace Registering HTTP GET /metrics?(/) +0ms
browserless.io:router:trace Registering HTTP GET /pressure?(/) +0ms
browserless.io:router:trace Registering HTTP GET /sessions?(/) +0ms
browserless.io:router:trace Registering HTTP GET / +0ms
browserless.io:router:trace Registering WebSocket "/devtools/browser/*" +0ms
browserless.io:router:trace Registering WebSocket "?(/),/chromium?(/)" +0ms
browserless.io:router:trace Registering WebSocket "/devtools/page/*" +0ms
browserless.io:router:trace Registering WebSocket "/playwright/chromium?(/),/chromium/playwright?(/)" +1ms
browserless.io:index:info Imported and validated all route files, starting up server. +10ms
browserless.io:server:info Server instantiated with host "0.0.0.0" on port "3000 +0ms
browserless.io:server:info HTTP Server is starting +1ms
browserless.io:server:info HTTP Server is listening on http://0.0.0.0:3000
Use http://0.0.0.0:3000 for API and connect calls +5ms
browserless.io:index:info Starting metrics collection. +6ms
browserless.io:index:info Current period usage: {"date":1737693332810,"error":0,"maxConcurrent":0,"maxTime":null,"meanTime":0,"minTime":null,"rejected":0,"successful":0,"timedout":0,"totalTime":0,"units":0} +5m
browserless.io:index:info Saving metrics to "/tmp/browserless-metrics.json" +0ms
browserless.io:index:info Current period usage: {"date":1737693632823,"error":0,"maxConcurrent":0,"maxTime":null,"meanTime":0,"minTime":null,"rejected":0,"successful":0,"timedout":0,"totalTime":0,"units":0} +5m
browserless.io:index:info Saving metrics to "/tmp/browserless-metrics.json" +0ms
browserless.io:index:info Current period usage: {"date":1737693932853,"error":0,"maxConcurrent":0,"maxTime":null,"meanTime":0,"minTime":null,"rejected":0,"successful":0,"timedout":0,"totalTime":0,"units":0} +5m
browserless.io:index:info Saving metrics to "/tmp/browserless-metrics.json" +0ms
browserless.io:index:info Current period usage: {"date":1737694232872,"error":0,"maxConcurrent":0,"maxTime":null,"meanTime":0,"minTime":null,"rejected":0,"successful":0,"timedout":0,"totalTime":0,"units":0} +5m
browserless.io:index:info Saving metrics to "/tmp/browserless-metrics.json" +1ms
browserless.io:index:info Current period usage: {"date":1737694532900,"error":0,"maxConcurrent":0,"maxTime":null,"meanTime":0,"minTime":null,"rejected":0,"successful":0,"timedout":0,"totalTime":0,"units":0} +5m
browserless.io:index:info Saving metrics to "/tmp/browserless-metrics.json" +0ms
Links do not get processed, when i go to search bar, i get, "Something went wrong."
Any ideas where I went wrong?
Thank you.