Human-126094

🚀 v2.43.0 - NVENC HEVC-unsupported diagnosis + NVIDIA GPU surface is live ghcr.io/masterjb/x265-butler:2.43.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This one is about making NVENC failures on older NVIDIA cards readable instead of cryptic. If your GPU can't do HEVC-NVENC you used to get a bare "-22 / Could not open encoder before EOF" and the diagnostics showed only that an nvidia device existed - never which card. Now both are clear. 🆕 WHAT'S NEW • Dedicated "GPU has no HEVC-NVENC block" diagnosis - the -22 / Could not open encoder failure now maps to a clear (hedged) message on the test-encode surface and the notification bell, distinct from the Pascal API-floor case from v2.41.0. This signature means the card is Kepler / Maxwell-1 era with no HEVC encode block; HEVC-NVENC needs Maxwell-2 / GM206+ (GTX 950/960 or newer). The message points you at a self-check before giving up on NVENC. • NVIDIA GPU model + driver in diagnostics - the diagnostics page and the copy-report now list each NVIDIA card's name + driver version (via nvidia-smi), next to the existing CPU line. So a report can be evidence-checked - confirm a "GeForce GT 730" really is Kepler - instead of a hedged guess. On a host without the NVIDIA runtime it just shows a clean "not present" line and the report still loads fine. No new dependencies, no DB migration, no encode/scan behaviour change - this is diagnostics-surface only. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.43.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.42.0 Keep the reports coming, the cryptic-error cleanups like this one come straight from them.

🚀 v2.42.0 - Clear NVENC "API too new" diagnosis ghcr.io/masterjb/x265-butler:2.42.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This is a small diagnostics release. When NVENC refuses a card with "Required: X.Y Found: A.B", the app now tells you what it actually means, the GPU's NVENC API is older than the encoder was built for (a hardware-generation limit on Pascal/Maxwell, not a driver you can update), instead of the old misleading "update your driver" hint. v2.41.0 already fixed the normal path (NVENC routes to the bundled jellyfin-ffmpeg), so you'll only see this via the FFMPEG_NVENC_PATH=ffmpeg revert lever or a card older than jellyfin's floor. No encode behaviour changed - QSV / VAAPI / libx265 / NVENC / bench / VMAF are all byte-for-byte identical. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.42.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.41.0 Keep the reports coming, the cryptic-error cleanups like this one come straight from them.

🚀 v2.41.0 - Pascal/Maxwell NVENC support is live ghcr.io/masterjb/x265-butler:2.41.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler If you have an older NVIDIA card, Pascal (Tesla P4, GTX 10-series) or Maxwell (GTX 9-series), NVENC now works. Before this release those cards were refused with "Required: 13.1 Found: 13.0" and silently fell back to CPU encoding, because our ffmpeg build (BtbN) has an NVENC API floor of 13.1 compiled in and those cards top out at 13.0. The fix ships a SECOND ffmpeg - jellyfin-ffmpeg - used only for NVENC. It has an older NVENC floor so Pascal/Maxwell cards encode, and modern RTX cards keep working too (the floor is backward-compatible). Everything else is unchanged: the primary BtbN ffmpeg still handles QSV, VAAPI, libx265, cropdetect, ffprobe and the VMAF quality bench. An encode routes to the jellyfin binary only when the encoder is NVENC. Your existing settings carry over untouched. (Trade-off: the image grew ~100 MB for the second binary.) Escape hatch: if you want NVENC back on the old binary, set container env FFMPEG_NVENC_PATH=ffmpeg (this re-triggers the Pascal refusal - it is a revert lever, not a fix). 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.41.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.40.0 🔭 COMING NEXT • A clearer diagnostic message for the NVENC API-floor case (instead of the cryptic "Required: 13.1 Found: 13.0") Plans can shift, but that's the current direction. More of the same rolling bug-stabilisation from your reports, keep them coming in this thread

@obideuce the Pascal NVENC fix is ready for you to test on a dev build 🎉 Quick recap of what changed: instead of swapping our ffmpeg (which would have broken the VMAF quality bench, since the Pascal-capable build ships no libvmaf), the image now carries TWO ffmpeg binaries. The normal BtbN one stays for QSV/VAAPI/x265/bench, and a second jellyfin-ffmpeg binary (older NVENC floor) handles NVENC only. So your Tesla P4 should finally encode instead of falling back to CPU. It's on the dev image, not a release tag yet, I need your P4 to confirm before I ship it. STEP 1 pull the dev image: docker pull ghcr.io/masterjb/x265-butler:dev Confirm you actually got a fresh image (the id should change from before): docker inspect --format '{{.Id}}' ghcr.io/masterjb/x265-butler:dev STEP 2 run the NVENC probe against the NEW binary (note: ffmpeg-nvenc, not ffmpeg). Assumes your container is named x265-butler and is running with --runtime=nvidia + NVIDIA_DRIVER_CAPABILITIES=compute,video,utility: docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20 WHAT I'M HOPING TO SEE: a "frame= 30" line and exit 0, NOT the old "Required: 13.1 Found: 13.0" refusal. That's the GO signal. STEP 3 sanity check in the Butler UI: • Encoder detection now lists NVENC as working (not a silent libx265 fallback) • The "Test encode" button on the Diagnostics page succeeds for NVENC • A real encode with the encoder set to NVENC completes STEP 4 (optional, nice-to-have) 10-bit HEVC on the P4: docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -pix_fmt p010le -c:v hevc_nvenc -f null - 2>&1 | tail -20 Same "frame= 30" + exit 0 = 10-bit works too. If this one fails but Step 2 passed, no problem — 8-bit is the fix, 10-bit would just be a known gap. Paste back the tail output from Step 2 (and Step 4 if you try it). If it's "frame= 30" I'll tag the release; if it's another "Required: … Found: …" line on the fresh dev image, that tells me the floor is still too high and I re-scope. Thanks for being the P4 test bench, no way to validate this without your hardware 🙏

Thanks for the report, the Required: 13.1 Found: 13.0 line pins it exactly. Your Tesla P4 is Pascal, and the bundled FFmpeg is compiled against NVENC headers that demand a driver newer than any Pascal driver branch (580) can provide. The NVENC minimum is baked into the binary at compile time, so there's no setting to flip, it needs a different FFmpeg build. I have a candidate fix (swap the bundled FFmpeg for a build with an older NVENC floor that still runs on modern cards too). Before I rebuild, could you run these 4 commands on your unRAID host and paste the output? Command C is the important one, it proves the fix works on your actual P4 before I ship anything. A — your exact GPU + driver: nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csv B — reproduce the failure with Butler's current FFmpeg (replace x265-butler if your container name differs): docker exec x265-butler ffmpeg -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20 (expected: the Required: 13.1 Found: 13.0 error) C — proof-of-fix: same NVENC test, but with the candidate FFmpeg build, on your P4: docker run --rm --runtime=nvidia --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20 (if this ends with frame= 30 and no error → the fix will work for you. If it shows another Required: … Found: … line → it won't, and I'll need a different approach.) D — encoder parity check on the candidate build: docker run --rm --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -encoders 2>/dev/null | grep -Ei 'nvenc|qsv|vaapi|libx265' And 3 quick questions so I scope it right: 1. Besides the Tesla P4, do you run any other older NVIDIA cards (Maxwell GTX 9xx, other Pascal) you'd want supported? 2. On the P4 you only need HEVC encoding, correct? (Pascal can't do AV1 encode either way.) 3. Once I have a fix, can you pull a :dev image and re-run command B (now expecting success) on the real P4 before I tag the release?

🚀 v2.40.0 - Encoding Profile card is live ghcr.io/masterjb/x265-butler:2.40.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This one is pure UI tidy-up: the three per-encode video toggles - Auto-Crop, Force 10-bit, and Colour/HDR10 passthrough - now sit together under one "Encoding Profile" card in the Encoder settings tab instead of being scattered across separate cards. That delivers the "gather the per-encode toggles into one place" note from the last release. No encoder code changed - your encodes come out byte-for-byte identical to v2.39.0, and your existing toggle states carry over untouched. Nothing to re-configure after the upgrade. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.40.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.39.0

🚀 v2.39.0 - 10-bit toggle, per-encode ETA, and "same as source" colour + HDR10 ghcr.io/masterjb/x265-butler:2.39.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This one is the HandBrake-style "near-passthrough" bundle a few of you asked for: keep the source bit-depth and colour, and show how long an encode has left. Three opt-in features, all default-OFF, so nothing changes on your existing setup until you flip a switch. 🔟 FORCE 10-BIT OUTPUT (new Setting, default OFF) Encodes every output as 10-bit HEVC (Main10) regardless of source depth, good for banding-prone gradients and a uniform archive depth. Works on all four encoders (libx265 / NVENC / QSV / VAAPI). OFF = output byte-identical to before. ⏱️ PER-ENCODE ETA IN THE QUEUE Each running encode now shows a live ETA next to its progress, derived from ffmpeg's own reported speed (no extra flag, no overhead). Refreshes on the existing ~30s progress cadence and reflects the real speed at your chosen preset. 🎨 PRESERVE SOURCE COLOUR + HDR10 (new Setting, default OFF) Carries the source colour signalling onto the re-encode instead of letting it default, the fix for washed-out colour / HDR after re-encode. One toggle covers both: • Colour VUI tags - colour space, primaries, transfer (e.g. BT.2020 / PQ), range. • HDR10 static metadata - mastering-display colour volume + MaxCLL/MaxFALL, so HDR displays tone-map correctly. OFF = output bytes unchanged. HARDWARE-ENCODER NOTE (NVENC / QSV / VAAPI): for the hardware encoders, 10-bit and HDR10 preservation ride ffmpeg's automatic side-data passthrough rather than an explicit per-encoder flag. Validated directly on libx265; on hardware it is best-effort and operator-validated. After a hardware encode of an HDR10 source with "Preserve source colour" ON, confirm it survived: ffprobe -hide_banner -show_streams <output> | grep -iE "mastering|content light" Expect a "Mastering display metadata" line and a "Content light level metadata" line. If you run a hardware encoder, a quick report either way is very welcome. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.39.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.38.2 🔭 COMING NEXT Rough outlook (plans can shift): • A general encoding-options UI: gather the per-encode toggles (10-bit, preserve-colour/HDR10, auto-crop, output mode...) into one place instead of scattered individual Settings, so you can dial in a profile without hunting. This release's three toggles are the first batch that UI will surface.

Thanks for the kind words and the detailed write up. Really helpful to know your exact workflow. 😊 Good news: most of what you describe is already how the encoder works today. It copies audio, subtitles, metadata and attachments untouched, applies no filters, doesn't touch framerate, and keeps the source bit depth (8 bit source stays 8 bit, 10 bit stays 10 bit, exact necessary already ships too (auto crop). So your "almost passthrough" case really only needs two small additions, both now planned: 1. Force x265 10 bit. An opt in toggle to encode even an 8 bit source as 10 bit bit" option, less banding and often better compression). 2. ETA per encode. Bringing the estimated time remaining back onto the running job view. Plus a small hardening so color range and primaries are explicitly carried through as "same as source". The broader "expose all encoding options" idea (deinterlace, denoise, tune, profile dropdowns) is a bigger change and, as you said, not everyone needs it. So I split that into a separate later step rather than holding up the part you actually need. Targeting these for an upcoming release. Will report back here when there's a build to test. Thanks again!

🚀 v2.38.2 - Poll-rate diagnostics now measured, not estimated ghcr.io/masterjb/x265-butler:2.38.2 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler v2.38.1 killed the watcher stat-storm on FUSE/shfs shares (confirmed on real hardware: ~59,000 → ~1,000 statx/sec). But the poll-rate number on the Diagnostics page still under-reported the real rate by about 3x - it printed an ESTIMATE from a fixed "10 paths per watched file" guess that doesn't fit every library shape. v2.38.2 makes that number real: each forced-polling share now reports the chokidar-MEASURED watched-path count (actualWatchedPaths / actualStatsPerSec / actualPathMultiplier) right next to the old estimate, so you can audit the stat-rate straight from /api/diagnostics without an strace. Diagnostics-only: the actual poll cadence and stat-rate on your host do NOT change vs v2.38.1 only the number the page prints for it becomes correct. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.38.2 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.38.1

Thanks, that strace is exactly the smoking gun, and your read is spot on. You nailed it: there WAS a second stat path not represented by pollingShares. It wasn't a separate scanner, it was chokidar's second poll knob. Chokidar's forced-polling mode has two intervals: "interval" for ordinary files, and "binaryInterval" for binary-extension files (.mkv/.mp4/.jpg/.png), which defaults to 300 ms. v2.38.0 only scaled "interval". Since your library is basically all binary extensions (video plus .jpg/.png artwork), every one of those paths kept polling at the 300 ms default. So WATCH_POLL_INTERVAL_MS=60000 was genuinely active (your diagnostics were honest), it just didn't govern the binary knob. That's the 169x gap between the ~350/s estimate and the ~59k/s you measured. This is fixed in v2.38.1 (just shipped): both poll knobs are now pinned to the same resolved interval, so the value you set governs the whole tree, artwork and video included. Please pull and restart: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler Then re-run the same 10s strace with WATCH_POLL_INTERVAL_MS=60000 still set. The ~591k/10s down toward the diagnostics computedStatsPerSec order (a few hundredper second, not tens of thousands). Would really appreciate a fresh strace sample after the upgrade to confirm it on your hardware. Your captures have been exactly what was needed to pin this down.

🚀 v2.38.1 - Watcher poll-storm hotfix is live ghcr.io/masterjb/x265-butler:2.38.1 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler Hotfix on v2.38.0: the file-watcher poll fix only governed one of chokidar's two poll knobs, so on a media library (mostly .mkv/.mp4/.jpg/.png) the binary-file poll kept running at 300 ms and the stat-storm survived even with WATCH_POLL_INTERVAL_MS set. v2.38.1 pins both knobs together, so the interval you set now flattens the whole tree. No config change needed - existing installs benefit automatically. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.38.1 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.38.0

🚀 v2.38.0 - Watcher poll-pool stat-storm fix is live ghcr.io/masterjb/x265-butler:2.38.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler If your library lives on a user-share (FUSE/shfs) and the web UI got sluggish while encodes ran - pages crawling, sometimes never loading, this is the fix. The file-watcher's polling mode was hammering the filesystem with a stat-storm (~62.000 stat calls/s on an 11.685-file share, strace-confirmed) that saturated Node's I/O thread pool, so every page-load queued behind it. v2.38.0 scales the poll interval with library size to cut that stat-rate by 10x or more. inotify shares (ext4/xfs) are unchanged; encodes were never affected. New WATCH_POLL_INTERVAL_MS env knob lets you pin it manually, and /api/diagnostics now shows the per-share polling state so the storm is visible without a shell. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.38.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.37.0

Thanks for the detailed test, that's exactly the data I needed, and your read was on the right track. I shipped v2.37.0 which fixes precisely this. Two things were going on: 1. The 4K "stdout exceeded cap" kill. It wasn't the resolution or the log size, it was duration. ffmpeg's machine-readable progress stream (not the x265 console output) emits ~2 lines/sec, so a ~5.5h 4K encode piled up ~40,000 progress blocks (~8.4 MB) and tripped an 8 MB stdout guard, which killed the job near the end. 1080p/720p finished faster, stayed under the cap, so they succeeded. That's why both 4K logs froze around ~13 MB while the 1080p one reached 24 MB, the 4K jobs were killed mid-stream, the 1080p one ran to completion. 2. The truncated job log. Same root cause, the progress stream was flooding the log, so "copy report" was 99% progress spam. v2.37.0 throttles progress to one update every 30s (the same 5.5h encode now produces ~660 blocks, ~140 KB, nowhere near the cap), and it no longer dumps the progress stream into the job log, so "copy report" now shows the real ffmpeg/x265 output. To update: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler Could you re-run the two 4K files once you're on 2.37.0? They should now run to completion, and the copied job log should be readable. Thanks again for the logs!

🚀 v2.37.0 - Output Stream-Mapping Hardening is live ghcr.io/masterjb/x265-butler:2.37.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler Two encode-output fixes from real footage: iPhone .MOV clips that refused to encode, and long 4K encodes that self-killed hours in. 🎬 IPHONE .MOV NOW ENCODES Every iPhone .MOV failed instantly, exit 234, 0 frames, "Only audio, video, and subtitles are supported for Matroska". It looked like it failed near the end, but the mux header was actually refused before the first frame. Cause: the iPhone attaches Apple timed-metadata (mebx) data streams that Matroska rejects, and we were copying every stream into the MKV. Now the MKV path maps only video/audio/subtitle and attachment streams (your anime ASS fonts are kept) and drops the incompatible ones. MP4 output is unchanged. 🐢 4K ENCODES NO LONGER SELF-KILL If you saw a 4K encode run for hours and then die with "encode:stdout exceeded cap" (cause in the job transition log) while 1080p/720p finished fine, this release fixes exactly that. It was duration-bound, not resolution- or log-size-bound: ffmpeg's progress stream emitted ~2 blocks/sec, so a ~5.5h 4K encode piled up ~40k blocks (~8.4 MiB) and tripped an 8 MiB stdout guard, killing the job near the end. (Your hunch that 4K "generates too much log and saturates stdout" was on the right track, it's the machine-readable progress stream specifically, not the x265 console output.) Progress is now throttled to one update every 30s, so the same encode produces ~660 blocks (~140 KiB) and never gets near the cap. Bonus: the job log "copy report" no longer fills up with progress spam, it now shows the real ffmpeg/x265 output, so the truncated-log problem is gone too. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.37.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.36.1 🔭 COMING NEXT Plans can change, but on the bench right now: • Sharper diagnosis of UI sluggishness during heavy encodes (CPU attribution sampler, waiting on a QSV capture from an affected host) • More robust hardware-encoder detection on multi-GPU / mixed Intel+AMD boxes As always - if something breaks on your hardware, drop the diagnostics copy-report here and I'll dig in.

Thanks a lot for the very detailed captures, the thread samples, the UI-closed test and the diagnostics report together make the picture clear, so I want to confirm the exact cause before changing anything. Your data points strongly at libuv thread-pool saturation from background filesystem work (not JS, not the database). To nail it down, please run this while the problem is happening, i.e. with your 2 QSV encodes running and the web UI feeling sluggish (not on an idle system, otherwise we miss the exact state we're measuring): docker exec x265-butler sh -c 'timeout 10 strace -f -c -p 1 2>&1 | head -30' What it does: attaches to the x265-butler Node process for 10 seconds and prints a summary of which Linux system calls it made and how often. If the top is dominated by newfstatat/statx (file-stat calls), that confirms the file-watcher is the cause; if it's read/pread64, it's a different path. Either way it tells us exactly where to fix it. The command only observes, it changes nothing and doesn't affect your encodes. Paste me the top lines and I'll take it from there.

🚀 v2.36.0 - AMD/VAAPI card no longer lost after a fresh-install restart ghcr.io/masterjb/x265-butler:2.36.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler If your AMD card went missing after restarting on a fresh install - encoding dropping back to CPU until you restarted again - this is the fix. At boot the app was probing for hardware AND starting the queue at the same time, so a real encode could grab the AMD card's single encode session right when the detection probe needed it. The probe failed, the card got marked "broken", and that was cached until the next restart. Detection is now single-flight: the boot probe and the queue share ONE detection run instead of two competing VAAPI probes, and the first encode only starts after detection finishes on an idle GPU. A failed detection is no longer cached either - the next try re-probes fresh. • If a card ever drops out, the "Re-detect hardware" button on the /diagnostics page re-runs detection on demand (with a before/after diff) - no restart needed. No new dependencies, no migrations, encode output byte-identical to v2.35.x. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.36.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.35.0 🔭 COMING NEXT • iPhone / .MOV encodes that fail right away - if an iPhone clip dies before frame 1 ("never finishes, always fails near the end"), it's the Apple timed-metadata track the file carries that the MKV container rejects. Next release teaches the encoder to drop those container-incompatible side tracks so the encode just runs. • Web UI staying snappy under load - better CPU prioritisation so the pages don't get sluggish while several encodes run at once. Plans can change, but that's what's on the bench.

Thanks for the detailed report, and good news, your logs already contain the root cause, so I don't need much more from you. This is not the encoder/hardware path at all (your Test Encode passing and libx265 being functional confirm that). Look at these lines in the small-file log: [matroska @ ...] Only audio, video, and subtitles are supported for Matroska. [out#0/matroska @ ...] Could not write header (incorrect codec parameters ?): Invalid argument [vf#0:0] Task finished with error code: -22 (Invalid argument) encoded 0 frames Conversion failed! Your iPhone .MOV files carry several Apple timed-metadata tracks: Stream #0:2..6 Data: none (mebx / 0x7862656D) x265-butler currently copies every input stream into the output. The default output container is MKV (Matroska), and Matroska cannot hold these mebx data streams, so ffmpeg refuses to write the file header and bails before encoding a single frame. That's why it looks like it "fails near the end": the progress bar runs to 100% in ~1 second (speed=256x, frame=0) because ffmpeg only scanned the timestamps, it never actually encoded anything. The fix is on my side: drop the unsupported data streams when muxing to Matroska. This affects basically all recent iPhone footage (the mebx tracks are standard on iOS). A fix is coming in the next release. One thing that would help me confirm your big file is the exact same bug (its attached log got truncated): could you check whether that log also contains the line Only audio, video, and subtitles are supported for Matroska. If yes, same root cause, same fix, nothing else needed. Thanks again for the great bug reports!

Thanks, that's a clean report and rules out the obvious suspects (renice, DRI permissions, slow queries). With both ffmpeg children already at nice 19 and next-server pinned near 100%, the bottleneck is inside the Node process itself, not the encodes. To find which loop is burning the CPU I need a few data points captured DURING the two active QSV encodes. 1. Per-thread CPU of next-server (which thread is hot, main loop vs libuv pool): ps -eLo pid,tid,ni,pcpu,comm | grep -i next-server | sort -k4 -nr | head -20 2. Run with debug logging for ~60s during the two encodes and attach the output: docker exec <container> sh -c 'LOG_LEVEL=debug' is build-baked, so instead set LOG_LEVEL=debug in the container env and restart, then capture: docker logs --since 60s <container> > x265-debug.log 3. How many concurrent SSE/progress clients are open? Please test with ALL browser tabs to the UI fully closed (not just backgrounded) for 60s and report whether next-server CPU drops. "Closing the tab doesn't help" suggests the work is server-side and tab-independent — confirming that narrows it to the encode progress/poll path, not SSE fan-out. 4. Library size, rough row counts help me judge if a query/scan loop scales with DB size: - number of files in library - number of active + queued jobs 5. Progress granularity: are the two encodes 4K/long sources? Frequent ffmpeg progress lines (per-frame) parsed + persisted per encode can dominate. Knowing source resolution/fps + how often the queue/progress UI updates helps. What I suspect: the per-encode ffmpeg progress parser + per-update SQLite write, multiplied by 2 concurrent encodes, running hotter than expected, or a UI poll endpoint doing more work per request than it should.

🚀 v2.35.0 - Web UI stays responsive while encoding is live ghcr.io/masterjb/x265-butler:2.35.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler If the UI got sluggish while encoding, pages slow to load, or the page-load "crashing" when a couple of encodes ran at once, this is the fix. ffmpeg used to run at the same priority as the web server and starved it on a busy CPU. Now every ffmpeg child is renice'd to the lowest priority, so the UI always wins the scheduler. Encodes still use the full CPU when the UI is idle, so encode speed is unchanged. • Tune it with ENCODE_NICE (default 19). ENCODE_NICE=0 = the old behavior (revert lever). • Bonus: slow request-path DB queries now show up in the /diagnostics copy-report (were silently dropped before). Tune the threshold with SLOW_QUERY_MS (default 100ms). No new dependencies, no migrations, encode output byte-identical to v2.34.x. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.35.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.34.0 Or just set ENCODE_NICE=0 to turn off the renice without rolling back. 🔭 COMING NEXT • Detection robustness on restart - an AMD/VAAPI card that can go missing after a fresh-install restart (a boot-timing race against the first encode). Looking into a re-detect path so the card comes back without a manual restart. Plans can change, but that's the next thing on the bench.

Great to hear it's all working for you now, thanks for the kind words! 🙏 You've actually spotted something that's already on my radar. The sluggish page loads (and the occasional crash) during active encodes come from the encoder processes competing with the web server for CPU, especially noticeable with three encodes running at once. Good news: the encodes themselves keep running in the background regardless, so no work is ever lost. And a fix to keep the UI responsive while encoding is already in planning. The goal is to make sure the web interface stays snappy no matter how many encodes are in flight. I'll post here once it ships. Thanks again for the detailed report, that's exactly the kind of feedback that helps! 👍

Hi @ASimpleSomething thanks for the report, and glad encoding is working now! I'd like to fix the "lose my AMD card after restart" issue properly, but I need a bit of data from your setup to target it instead of guessing. When you get a chance, could you grab the following right after a restart where the AMD card has gone missing? 1) Diagnostics page Open the app's /diagnostics page and take a screenshot (or copy the text). I'm specifically looking at the VAAPI / render-device section, does it say the encoder is "broken" or "missing", and is there any driver / render-node warning shown? 2) Container log around startup From the Docker logs, please copy the lines from right after the restart. The two I care about most are: - "encoder loop started" - "encoder detection complete" The "detection complete" line lists the detected encoders, the outcome per encoder, and a short error excerpt if a probe failed, that excerpt is the key piece. 3) Was anything encoding at the moment you restarted? Were there jobs running or queued when you restarted the container? (yes/no, roughly how many) 4) Does a second restart bring it back? If you restart again with the queue paused or empty, does the AMD card show up correctly? (yes/no) 5) GPU passthrough How is the GPU passed into the container, which /dev/dri device(s), and your --device / PGID settings? Just want to rule out the device dropping out at the Docker level vs. the app mis-detecting it. My current suspicion: at boot the app probes the GPU and starts encoding at the Same time, so a real encode can grab the AMD card's encode session before the probe finishes, the probe then fails, the card gets marked unavailable, and it stays that way until the next restart. The data above will confirm whether that's what's happening before I ship the fix. Thanks a lot, this will let me fix it precisely rather than blind.

🚀 v2.34.0 - Encode-stall fix on high-core-count hosts + auto-crop clarity is live ghcr.io/masterjb/x265-butler:2.34.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This fixes libx265 encodes that hung forever on machines with very many CPU threads (the report: a 128-thread host) - x265's own thread-pool auto-detect over-allocated and stalled before the first frame. v2.34.0 caps the pool to a sane bound so the encode actually runs. 🔧 WHAT CHANGED • libx265 thread-pool cap (the stall fix): we now pass pools=min(cpuCount, 16) to x265, bypassing the NUMA auto-detect that logged "Thread pool created using 21914 threads" on a 128-CPU box and then hung. The cap covers normal encodes, the Test Encode, and the boot detection-probe. Override with X265_POOLS=8 to pin an exact size, or X265_POOLS=0 (or "auto") to fall back to x265 native auto-detect. libx265 only - NVENC/QSV/VAAPI are unaffected (no CPU thread pool). • Auto-crop no-op is now visible: when Auto-Crop is on and the source has NO black bars, the file is encoded full-frame (correct, unchanged). That used to be silent, so it looked broken. Now it logs a clear line and the Settings + onboarding text says a bars-free file is left unchanged on purpose - that is the expected result, not an error. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.34.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.33.0 🔭 COMING NEXT Rolling bug-stabilization continues - keep the diagnostics reports coming, they drive these fixes.

Great, that confirms it, thanks for running the manual test. Short answer: no, the cap does not use all 128 threads, and that's deliberate, not a limitation. Here's the reasoning: - For a single x265 encode, throughput stops scaling well past ~16 threads. A 4K HEVC encode only exposes so much parallelism (wavefront ≈ 34 rows, 3 frame-threads in your log), so threads 17–128 mostly sit idle waiting on each other anyway. The 21914 threads your box tried to spawn wasn't "using all cores" — it was the auto-detect over-allocating on a 64-core/128-thread CPU, getting stuck spawning threads, and never starting the encode. That's the hang we're fixing. - So the fix defaults the pool to min(cores, 16) = 16 on your machine. Your pools=8 manual test (8 threads, encode completed) already proved the pool just needs to be bounded. - The 5–8% CPU you see is expected for one encode on a 128-thread box, that's roughly 10 cores busy. The encode being slow (0.114x in your test) is just 4K 60fps 10-bit HEVC being heavy; the thread cap is not what makes it slow. If you want more cores working, two levers: 1. X265_POOLS=32 (or 64), raises the per-encode pool. It's uncapped above 16, so you can tune it. Expect diminishing returns, and don't go silly-high or you risk the same over-allocation stall. 2. Better on a 64-core box: raise the app's concurrency so it encodes several files at once. One encode won't saturate your CPU no matter the pool size; running 4–8 in parallel will. That's the real way to put those cores to work. The "it worked days ago" isn't you dreaming, if those encodes used the NVIDIA/Intel hardware path, or were shorter/smaller clips, they'd dodge the x265 thread-pool path entirely. Fixed image is on the way, it ships the 16-thread default plus the X265_POOLS override so you can tune it without waiting on me.

Hi, thanks a lot for the detailed report and logs, they were exactly what I needed. I think I've found the cause. Your log shows the x265 encoder trying to create a thread pool with 21914 threads on your 128-core host. That's bad auto-detection on machines with very high core counts (and/or multiple NUMA nodes): x265 over-allocates the thread pool, gets stuck spawning threads, and never actually starts encoding — which is exactly why the CPU sits idle, the queue shows no progress, and the Test Encode times out. It's not the black-bars/auto-crop setting (that part worked correctly and found no bars on your footage), and it's not specific to a version — every image fails because the app never capped x265's threads. The fix: I'll set an explicit x265 thread-pool cap so it can't over-allocate, with an override (X265_POOLS) in case you want to tune it. If you don't mind, two quick things to confirm the diagnosis while I prepare the fix: 1. The output of "lscpu" on your server (so I can see the CPU/NUMA layout). 2. Does a manual encode complete if you cap the pool? For example: ffmpeg -i input.mov -c:v libx265 -x265-params pools=8 -an -t 30 -f null - If that runs (instead of hanging), it confirms the thread-pool theory. I'll post back when the fixed image is up. Thanks again for helping track this down!

🚀 v2.33.0 - Re-Queue, Queue Progress & Cache-Path Fixes is live ghcr.io/masterjb/x265-butler:2.33.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base. Pull / upgrade: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butler This is a plumbing release - three operator-reported fixes in the queue/encode path, no codec or quality changes. 🛠️ WHAT CHANGED • Cache-path EACCES fix for older installs. If you installed before v2.21 you carry a stored cache path of /mnt/cache/x265-butler (the old hard-coded default). Since v2.21 any stored value is treated as a manual override and used as-is with no writability check - so on any host that doesn't map /mnt/cache/x265-butler into the container, every encode failed with cache_pool_unavailable:EACCES. v2.33.0 adds a one-time boot migration that drops ONLY that exact legacy-default value, so it auto-resolves again (writable /mnt/cache -> same path; otherwise falls back to /config/cache). A real custom cache path is never touched. • Central-sidecar source re-queue fix. In central sidecar + suffix mode, a smaller re-encode kept the source but never wrote a source-keyed sidecar for it, so rescans kept re-queueing already-encoded files. The marker is now written on that path - rescans stop redoing finished work. • Per-job queue progress bars. The progress bar could vanish mid-encode, and with parallelism >=2 the concurrent encodes collapsed onto one bar. Each concurrent encode now shows its own live progress bar. 🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions) If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container: Extra Parameters: --runtime=nvidia Variables: NVIDIA_VISIBLE_DEVICES = all NVIDIA_DRIVER_CAPABILITIES = compute,video,utility The "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself. Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line. ♻️ ROLLBACK If v2.33.0 misbehaves on your hardware, roll back to the previous image: docker stop x265-butler && docker rm x265-butler docker pull ghcr.io/masterjb/x265-butler:2.32.0 🔭 COMING NEXT • Route the job-log retention sweep through the cache-path resolver (so log retention keeps running after the cache-path fix above). • More operator-reported stabilization across the queue, encode, and storage surfaces.