June 24Jun 24 Great news, just started Apple MOV + Custom 4K + movie 4K conversion, will let you know tomorrow 😊Big thanks to you !
June 24Jun 24 2 hours ago, Human-126094 said:Thanks a lot for the very detailed captures, the thread samples, the UI-closed test and the diagnostics report together make the picture clear, so I want to confirm the exact cause before changing anything.Your data points strongly at libuv thread-pool saturation from background filesystem work (not JS, not the database). To nail it down, please run this while the problem is happening, i.e. with your 2 QSV encodes running and the web UI feeling sluggish (not on an idle system, otherwise we miss the exact state we're measuring):docker exec x265-butler sh -c 'timeout 10 strace -f -c -p 1 2>&1 | head -30'What it does: attaches to the x265-butler Node process for 10 seconds and prints a summary of which Linux system calls it made and how often. If the top is dominated by newfstatat/statx (file-stat calls), that confirms the file-watcher is the cause; if it's read/pread64, it's a different path. Either way it tells us exactly where to fix it. The command only observes, it changes nothing and doesn't affect your encodes.Paste me the top lines and I'll take it from there.Confirmed. I attached strace from a temporary debug container to the host PID of the actual next-server process while 2 QSV encodes were running and the Web UI was sluggish.Command used:pid=$(docker top x265-butler -eo pid,comm,args | awk '$2=="next-server"{print $1; exit}') docker run --rm -it \ --pid=host \ --cap-add SYS_PTRACE \ --security-opt seccomp=unconfined \ alpine sh -c "apk add --no-cache strace >/dev/null && timeout 10 strace -f -c -p $pid 2>&1 | head -30"Outputstrace: Process 970844 attached with 23 threadsstrace: Process 970844 detached...% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ---------------- 87.90 307.585941 495 620684 statx 8.28 28.979974 61 474333 97810 futex 1.71 5.997531 32 182512 epoll_pwait 1.11 3.883501 34 113574 readSo yes, this strongly confirms that the bottleneck is massive filesystem stat work. During the 10s sample, next-server made about 620k statx calls, roughly 62k/sec, and statx accounted for ~88% of traced syscall time.This matches the previous thread samples where all libuv workers were busy and explains why increasing UV_THREADPOOL_SIZE improves responsiveness somewhat but does not fix the root cause.
June 25Jun 25 Author 🚀 v2.38.0 - Watcher poll-pool stat-storm fix is liveghcr.io/masterjb/x265-butler:2.38.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerIf your library lives on a user-share (FUSE/shfs) and the web UI got sluggish while encodes ran - pages crawling, sometimes never loading, this is the fix. The file-watcher's polling mode was hammering the filesystem with a stat-storm (~62.000 stat calls/s on an 11.685-file share, strace-confirmed) that saturated Node's I/O thread pool, so every page-load queued behind it. v2.38.0 scales the poll interval with library size to cut that stat-rate by 10x or more. inotify shares (ext4/xfs) are unchanged; encodes were never affected. New WATCH_POLL_INTERVAL_MS env knob lets you pin it manually, and /api/diagnostics now shows the per-share polling state so the storm is visible without a shell.🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.38.0 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.37.0
June 25Jun 25 WATCH_POLL_INTERVAL_MS=60000 is now definitely active and diagnostics confirms it.Env:WATCH_POLL_INTERVAL_MS=60000UV_THREADPOOL_SIZE=16Diagnostics:Horror effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=12Serien_Kinder effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=191Serien effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=147So diagnostics now expects roughly:~350 stats/secBut the real 10s strace sample is still essentially unchanged:% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ---------------- 70.71 300.249783 507 591930 statx 26.19 111.215596 243 456980 94509 futex 1.38 5.865397 34 170438 epoll_pwait 0.90 3.837793 35 107325 readThat is still around:591,930 statx calls / 10s~59,000 statx/secSo the actual syscall rate is about 169x higher than the diagnostics estimate.The earlier path trace showed these statx calls are still coming from /media/..., including video and artwork/metadata files across all watched shares:/media/Horror/...fanart.jpg/media/Horror/...clearart.png/media/Horror/...poster.jpg/media/Serien/..._hevc.mkv/media/Serien/...thumb.jpg/media/Serien/...season-poster.jpg/media/Serien_Kinder/..._hevc.mkv/media/Serien_Kinder/...poster.jpgSo the remaining stat storm appears to come from a second filesystem/stat path that is not represented by pollingShares, or from additional watcher/scanner instances not controlled by WATCH_POLL_INTERVAL_MS. Edited June 25Jun 25 by cmeyer86 removed links from diagnostic
June 25Jun 25 Author 🚀 v2.38.1 - Watcher poll-storm hotfix is liveghcr.io/masterjb/x265-butler:2.38.1 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerHotfix on v2.38.0: the file-watcher poll fix only governed one of chokidar's two poll knobs, so on a media library (mostly .mkv/.mp4/.jpg/.png) the binary-file poll kept running at 300 ms and the stat-storm survived even with WATCH_POLL_INTERVAL_MS set. v2.38.1 pins both knobs together, so the interval you set now flattens the whole tree. No config change needed - existing installs benefit automatically.🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.38.1 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.38.0
June 25Jun 25 Author 1 hour ago, cmeyer86 said:WATCH_POLL_INTERVAL_MS=60000 is now definitely active and diagnostics confirms it.Env:WATCH_POLL_INTERVAL_MS=60000UV_THREADPOOL_SIZE=16Diagnostics:Horror effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=12Serien_Kinder effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=191Serien effectiveIntervalMs=60000 intervalSource=env computedStatsPerSec=147So diagnostics now expects roughly:~350 stats/secBut the real 10s strace sample is still essentially unchanged:% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ---------------- 70.71 300.249783 507 591930 statx 26.19 111.215596 243 456980 94509 futex 1.38 5.865397 34 170438 epoll_pwait 0.90 3.837793 35 107325 readThat is still around:591,930 statx calls / 10s~59,000 statx/secSo the actual syscall rate is about 169x higher than the diagnostics estimate.The earlier path trace showed these statx calls are still coming from /media/..., including video and artwork/metadata files across all watched shares:/media/Horror/...fanart.jpg/media/Horror/...clearart.png/media/Horror/...poster.jpg/media/Serien/..._hevc.mkv/media/Serien/...thumb.jpg/media/Serien/...season-poster.jpg/media/Serien_Kinder/..._hevc.mkv/media/Serien_Kinder/...poster.jpgSo the remaining stat storm appears to come from a second filesystem/stat path that is not represented by pollingShares, or from additional watcher/scanner instances not controlled by WATCH_POLL_INTERVAL_MS.Thanks, that strace is exactly the smoking gun, and your read is spot on.You nailed it: there WAS a second stat path not represented by pollingShares. It wasn't a separate scanner, it was chokidar's second poll knob. Chokidar's forced-polling mode has two intervals: "interval" for ordinary files, and "binaryInterval" for binary-extension files (.mkv/.mp4/.jpg/.png), which defaults to 300 ms.v2.38.0 only scaled "interval". Since your library is basically all binary extensions (video plus .jpg/.png artwork), every one of those paths kept polling at the 300 ms default. So WATCH_POLL_INTERVAL_MS=60000 was genuinely active (your diagnostics were honest), it just didn't govern the binary knob. That's the 169x gap between the ~350/s estimate and the ~59k/s you measured.This is fixed in v2.38.1 (just shipped): both poll knobs are now pinned to the same resolved interval, so the value you set governs the whole tree, artwork and video included.Please pull and restart: docker pull ghcr.io/masterjb/x265-butler:latest docker restart x265-butlerThen re-run the same 10s strace with WATCH_POLL_INTERVAL_MS=60000 still set. The ~591k/10s down toward the diagnostics computedStatsPerSec order (a few hundredper second, not tens of thousands).Would really appreciate a fresh strace sample after the upgrade to confirm it on your hardware. Your captures have been exactly what was needed to pin this down.
June 25Jun 25 55 minutes ago, Human-126094 said:Thanks, that strace is exactly the smoking gun, and your read is spot on.You nailed it: there WAS a second stat path not represented by pollingShares. It wasn't a separate scanner, it was chokidar's second poll knob. Chokidar's forced-polling mode has two intervals: "interval" for ordinary files, and "binaryInterval" for binary-extension files (.mkv/.mp4/.jpg/.png), which defaults to 300 ms.v2.38.0 only scaled "interval". Since your library is basically all binary extensions (video plus .jpg/.png artwork), every one of those paths kept polling at the 300 ms default. So WATCH_POLL_INTERVAL_MS=60000 was genuinely active (your diagnostics were honest), it just didn't govern the binary knob. That's the 169x gap between the ~350/s estimate and the ~59k/s you measured.This is fixed in v2.38.1 (just shipped): both poll knobs are now pinned to the same resolved interval, so the value you set governs the whole tree, artwork and video included.Please pull and restart:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerThen re-run the same 10s strace with WATCH_POLL_INTERVAL_MS=60000 still set. The ~591k/10s down toward the diagnostics computedStatsPerSec order (a few hundredper second, not tens of thousands).Would really appreciate a fresh strace sample after the upgrade to confirm it on your hardware. Your captures have been exactly what was needed to pin this down.Confirmed fixed on Unraid with v2.38.1.Version:version=2.38.1gitHash=7c0f01a4committedAtCET=25.06.26, 11:41:13Env:WATCH_POLL_INTERVAL_MS=60000UV_THREADPOOL_SIZE=4Diagnostics:Horror: effectiveIntervalMs=60000, intervalSource=env, computedStatsPerSec=12Serien_Kinder: effectiveIntervalMs=60000, intervalSource=env, computedStatsPerSec=191Serien: effectiveIntervalMs=60000, intervalSource=env, computedStatsPerSec=147So diagnostics estimates about:~350 stats/sec10s strace after v2.38.1:% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ----------------43.57 1.037159 97 10610 statx33.56 0.798994 43 18397 3770 futex7.26 0.172810 13 13211 epoll_pwait1.82 0.043368 6 6467 read1.62 0.038540 5 6468 writeThat is:10,610 statx / 10s~1,061 statx/secBefore v2.38.1 with the same WATCH_POLL_INTERVAL_MS=60000, this was:591,930 statx / 10s~59,193 statx/secSo the statx rate dropped by roughly 56x.The remaining rate is still above the diagnostics estimate, but the original binaryInterval poll storm is clearly gone.
June 25Jun 25 Great news, just started Apple MOV + Custom 4K + movie 4K conversion, will let you know tomorrow 😊Big thanks to you !Everything encoded successfully 😊Thanks again for all your great work !If I may, it would great to :Have an ETA (I think it was there on previous versions)Have access to the encoding options (deinterlace, denoise, tune, profile, etc...). Why ? Because in my own usecase I want to :Reencode files in x265 without any further modifications on my filesSet the encode to x265 10bit (which reverts to 8bit if the source is not 10bit I think) to loose as less quality as possibleManage my framerate (constant, same as source)Set the color range to Same as sourceIf fact, sorry to name it again, but I would like to be able to do exactly what I'm doing today with HandBrake : almost passthrough everything except encoding in x265 10bit and crop when necessary (don't had any filter or other option, don't touch framerate or colors, just don't do anything else than reencoding in x265+)I know it's a big change, which everybody may not need. But in my case, I would like to be able to reencode my entire library in an almost lossless x265 version
June 25Jun 25 Author 🚀 v2.38.2 - Poll-rate diagnostics now measured, not estimatedghcr.io/masterjb/x265-butler:2.38.2 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerv2.38.1 killed the watcher stat-storm on FUSE/shfs shares (confirmed on real hardware: ~59,000 → ~1,000 statx/sec). But the poll-rate number on the Diagnostics page still under-reported the real rate by about 3x - it printed an ESTIMATE from a fixed "10 paths per watched file" guess that doesn't fit every library shape. v2.38.2 makes that number real: each forced-polling share now reports the chokidar-MEASURED watched-path count (actualWatchedPaths / actualStatsPerSec / actualPathMultiplier) right next to the old estimate, so you can audit the stat-rate straight from /api/diagnostics without an strace.Diagnostics-only: the actual poll cadence and stat-rate on your host do NOT change vs v2.38.1 only the number the page prints for it becomes correct.🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.38.2 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.38.1
June 25Jun 25 Author 8 hours ago, tbal82 said:Everything encoded successfully 😊Thanks again for all your great work !If I may, it would great to :Have an ETA (I think it was there on previous versions)Have access to the encoding options (deinterlace, denoise, tune, profile, etc...). Why ? Because in my own usecase I want to :Reencode files in x265 without any further modifications on my filesSet the encode to x265 10bit (which reverts to 8bit if the source is not 10bit I think) to loose as less quality as possibleManage my framerate (constant, same as source)Set the color range to Same as sourceIf fact, sorry to name it again, but I would like to be able to do exactly what I'm doing today with HandBrake : almost passthrough everything except encoding in x265 10bit and crop when necessary (don't had any filter or other option, don't touch framerate or colors, just don't do anything else than reencoding in x265+)I know it's a big change, which everybody may not need. But in my case, I would like to be able to reencode my entire library in an almost lossless x265 versionThanks for the kind words and the detailed write up. Really helpful to know your exact workflow. 😊Good news: most of what you describe is already how the encoder works today. It copies audio, subtitles, metadata and attachments untouched, applies no filters, doesn't touch framerate, and keeps the source bit depth (8 bit source stays 8 bit, 10 bit stays 10 bit, exact necessary already ships too (auto crop).So your "almost passthrough" case really only needs two small additions, both now planned:1. Force x265 10 bit. An opt in toggle to encode even an 8 bit source as 10 bit bit" option, less banding and often better compression).2. ETA per encode. Bringing the estimated time remaining back onto the running job view. Plus a small hardening so color range and primaries are explicitly carried through as "same as source". The broader "expose all encoding options" idea (deinterlace, denoise, tune, profile dropdowns) is a bigger change and, as you said, not everyone needs it. So I split that into a separate later step rather than holding up the part you actually need.Targeting these for an upcoming release. Will report back here when there's a build to test. Thanks again! Edited June 25Jun 25 by Human-126094
June 29Jun 29 Author 🚀 v2.39.0 - 10-bit toggle, per-encode ETA, and "same as source" colour + HDR10ghcr.io/masterjb/x265-butler:2.39.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerThis one is the HandBrake-style "near-passthrough" bundle a few of you asked for: keep the source bit-depth and colour, and show how long an encode has left. Three opt-in features, all default-OFF, so nothing changes on your existing setup until you flip a switch.🔟 FORCE 10-BIT OUTPUT (new Setting, default OFF)Encodes every output as 10-bit HEVC (Main10) regardless of source depth, good for banding-prone gradients and a uniform archive depth. Works on all four encoders (libx265 / NVENC / QSV / VAAPI). OFF = output byte-identical to before.⏱️ PER-ENCODE ETA IN THE QUEUEEach running encode now shows a live ETA next to its progress, derived from ffmpeg's own reported speed (no extra flag, no overhead). Refreshes on the existing ~30s progress cadence and reflects the real speed at your chosen preset.🎨 PRESERVE SOURCE COLOUR + HDR10 (new Setting, default OFF)Carries the source colour signalling onto the re-encode instead of letting it default, the fix for washed-out colour / HDR after re-encode. One toggle covers both:• Colour VUI tags - colour space, primaries, transfer (e.g. BT.2020 / PQ), range.• HDR10 static metadata - mastering-display colour volume + MaxCLL/MaxFALL, so HDR displays tone-map correctly.OFF = output bytes unchanged.HARDWARE-ENCODER NOTE (NVENC / QSV / VAAPI): for the hardware encoders, 10-bit and HDR10 preservation ride ffmpeg's automatic side-data passthrough rather than an explicit per-encoder flag. Validated directly on libx265; on hardware it is best-effort and operator-validated. After a hardware encode of an HDR10 source with "Preserve source colour" ON, confirm it survived:ffprobe -hide_banner -show_streams <output> | grep -iE "mastering|content light"Expect a "Mastering display metadata" line and a "Content light level metadata" line. If you run a hardware encoder, a quick report either way is very welcome.🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.39.0 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.38.2🔭 COMING NEXTRough outlook (plans can shift):• A general encoding-options UI: gather the per-encode toggles (10-bit, preserve-colour/HDR10, auto-crop, output mode...) into one place instead of scattered individual Settings, so you can dial in a profile without hunting. This release's three toggles are the first batch that UI will surface.
June 29Jun 29 Tesla P4 on NVIDIA 580-series driver.GPU is visible inside the container and nvidia-smi works, but Butler's bundled FFmpeg requires NVENC API 13.1 / driver 610:Driver does not support the required nvenc API version.Required: 13.1 Found: 13.0The Tesla P4 is Pascal and NVIDIA driver 580 is the final branch supporting Pascal. Could Butler use an FFmpeg build compiled against Video Codec SDK 13.0, or provide a legacy-NVENC image/tag for Pascal GPUs?
Tuesday at 11:37 AM5 days Author 🚀 v2.40.0 - Encoding Profile card is liveghcr.io/masterjb/x265-butler:2.40.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerThis one is pure UI tidy-up: the three per-encode video toggles - Auto-Crop, Force 10-bit, and Colour/HDR10 passthrough - now sit together under one "Encoding Profile" card in the Encoder settings tab instead of being scattered across separate cards. That delivers the "gather the per-encode toggles into one place" note from the last release. No encoder code changed - your encodes come out byte-for-byte identical to v2.39.0, and your existing toggle states carry over untouched. Nothing to re-configure after the upgrade.🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.40.0 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.39.0
Tuesday at 12:03 PM5 days Author 16 hours ago, obideuce said:Tesla P4 on NVIDIA 580-series driver.GPU is visible inside the container and nvidia-smi works, but Butler's bundled FFmpeg requires NVENC API 13.1 / driver 610:Driver does not support the required nvenc API version.Required: 13.1 Found: 13.0The Tesla P4 is Pascal and NVIDIA driver 580 is the final branch supporting Pascal. Could Butler use an FFmpeg build compiled against Video Codec SDK 13.0, or provide a legacy-NVENC image/tag for Pascal GPUs?Thanks for the report, the Required: 13.1 Found: 13.0 line pins it exactly. Your Tesla P4 is Pascal, and the bundled FFmpeg is compiled against NVENC headers that demand a driver newer than any Pascal driver branch (580) can provide. The NVENC minimum is baked into the binary at compile time, so there's no setting to flip, it needs a different FFmpeg build.I have a candidate fix (swap the bundled FFmpeg for a build with an older NVENC floor that still runs on modern cards too). Before I rebuild, could you run these 4 commands on your unRAID host and paste the output? Command C is the important one, it proves the fix works on your actual P4 before I ship anything.A — your exact GPU + driver:nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csvB — reproduce the failure with Butler's current FFmpeg (replace x265-butler if your container name differs):docker exec x265-butler ffmpeg -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20(expected: the Required: 13.1 Found: 13.0 error)C — proof-of-fix: same NVENC test, but with the candidate FFmpeg build, on your P4:docker run --rm --runtime=nvidia --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20(if this ends with frame= 30 and no error → the fix will work for you. If it shows another Required: … Found: … line → it won't, and I'll need a different approach.)D — encoder parity check on the candidate build:docker run --rm --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -encoders 2>/dev/null | grep -Ei 'nvenc|qsv|vaapi|libx265'And 3 quick questions so I scope it right:1. Besides the Tesla P4, do you run any other older NVIDIA cards (Maxwell GTX 9xx, other Pascal) you'd want supported?2. On the P4 you only need HEVC encoding, correct? (Pascal can't do AV1 encode either way.)3. Once I have a fix, can you pull a :dev image and re-run command B (now expecting success) on the real P4 before I tag the release?
Tuesday at 08:14 PM5 days 8 hours ago, Human-126094 said:Thanks for the report, the Required: 13.1 Found: 13.0 line pins it exactly. Your Tesla P4 is Pascal, and the bundled FFmpeg is compiled against NVENC headers that demand a driver newer than any Pascal driver branch (580) can provide. The NVENC minimum is baked into the binary at compile time, so there's no setting to flip, it needs a different FFmpeg build.I have a candidate fix (swap the bundled FFmpeg for a build with an older NVENC floor that still runs on modern cards too). Before I rebuild, could you run these 4 commands on your unRAID host and paste the output? Command C is the important one, it proves the fix works on your actual P4 before I ship anything.A — your exact GPU + driver:nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csvB — reproduce the failure with Butler's current FFmpeg (replace x265-butler if your container name differs):docker exec x265-butler ffmpeg -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20(expected: the Required: 13.1 Found: 13.0 error)C — proof-of-fix: same NVENC test, but with the candidate FFmpeg build, on your P4:docker run --rm --runtime=nvidia --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20(if this ends with frame= 30 and no error → the fix will work for you. If it shows another Required: … Found: … line → it won't, and I'll need a different approach.)D — encoder parity check on the candidate build:docker run --rm --entrypoint /usr/lib/jellyfin-ffmpeg/ffmpeg jellyfin/jellyfin:latest -hide_banner -encoders 2>/dev/null | grep -Ei 'nvenc|qsv|vaapi|libx265'And 3 quick questions so I scope it right:1. Besides the Tesla P4, do you run any other older NVIDIA cards (Maxwell GTX 9xx, other Pascal) you'd want supported?2. On the P4 you only need HEVC encoding, correct? (Pascal can't do AV1 encode either way.)3. Once I have a fix, can you pull a :dev image and re-run command B (now expecting success) on the real P4 before I tag the release?A: name, driver_version, compute_cap Tesla P4, 580.159.04, 6.1 B: [hevc_nvenc @ 0x564ef338e000] Driver does not support the required nvenc API version. Required: 13.1 Found: 13.0 [hevc_nvenc @ 0x564ef338e000] The minimum required Nvidia driver for nvenc is 610.00 or newer C:fb72fd6b365b: Pull complete 19bd9c9a3a70: Pull complete Digest: sha256:aefb67e6a7ff1debdd154a78a7bbb780fd0c873d8639210a7f6a2016ad2b35db Status: Downloaded newer image for jellyfin/jellyfin:latest Input #0, lavfi, from 'testsrc=size=320x240:rate=30': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr, 30 tbn Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_nvenc)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf61.7.103 Stream #0:0: Video: hevc (Rext), gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 2000 kb/s, 30 fps, 30 tbn Metadata: encoder : Lavc61.19.101 hevc_nvenc Side data: cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A [out#0/null @ 0x14b5a4a7c040] video:55KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown frame= 30 fps=0.0 q=10.0 Lsize=N/A time=00:00:01.00 bitrate=N/A speed=2.92xD: V....D av1_nvenc NVIDIA NVENC av1 encoder (codec av1) V..... av1_qsv AV1 (Intel Quick Sync Video acceleration) (codec av1) V....D av1_vaapi AV1 (VAAPI) (codec av1) V....D h264_nvenc NVIDIA NVENC H.264 encoder (codec h264) V..... h264_qsv H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264) V....D h264_vaapi H.264/AVC (VAAPI) (codec h264) V....D libx265 libx265 H.265 / HEVC (codec hevc) V....D hevc_nvenc NVIDIA NVENC hevc encoder (codec hevc) V..... hevc_qsv HEVC (Intel Quick Sync Video acceleration) (codec hevc) V....D hevc_vaapi H.265/HEVC (VAAPI) (codec hevc) V..... mjpeg_qsv MJPEG (Intel Quick Sync Video acceleration) (codec mjpeg) V....D mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg) V..... mpeg2_qsv MPEG-2 video (Intel Quick Sync Video acceleration) (codec mpeg2video) V....D mpeg2_vaapi MPEG-2 (VAAPI) (codec mpeg2video) V....D vp8_vaapi VP8 (VAAPI) (codec vp8) V....D vp9_vaapi VP9 (VAAPI) (codec vp9) V..... vp9_qsv VP9 video (Intel Quick Sync Video acceleration) (codec vp9)Just the P4 in my build. Yes, just HEVCYes, happy to do so once it's there.
Wednesday at 10:06 AM4 days Author 13 hours ago, obideuce said:A:name, driver_version, compute_cap Tesla P4, 580.159.04, 6.1 B:[hevc_nvenc @ 0x564ef338e000] Driver does not support the required nvenc API version. Required: 13.1 Found: 13.0 [hevc_nvenc @ 0x564ef338e000] The minimum required Nvidia driver for nvenc is 610.00 or newer C:fb72fd6b365b: Pull complete 19bd9c9a3a70: Pull complete Digest: sha256:aefb67e6a7ff1debdd154a78a7bbb780fd0c873d8639210a7f6a2016ad2b35db Status: Downloaded newer image for jellyfin/jellyfin:latest Input #0, lavfi, from 'testsrc=size=320x240:rate=30': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr, 30 tbn Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_nvenc)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf61.7.103 Stream #0:0: Video: hevc (Rext), gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 2000 kb/s, 30 fps, 30 tbn Metadata: encoder : Lavc61.19.101 hevc_nvenc Side data: cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A [out#0/null @ 0x14b5a4a7c040] video:55KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown frame= 30 fps=0.0 q=10.0 Lsize=N/A time=00:00:01.00 bitrate=N/A speed=2.92xD: V....D av1_nvenc NVIDIA NVENC av1 encoder (codec av1) V..... av1_qsv AV1 (Intel Quick Sync Video acceleration) (codec av1) V....D av1_vaapi AV1 (VAAPI) (codec av1) V....D h264_nvenc NVIDIA NVENC H.264 encoder (codec h264) V..... h264_qsv H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264) V....D h264_vaapi H.264/AVC (VAAPI) (codec h264) V....D libx265 libx265 H.265 / HEVC (codec hevc) V....D hevc_nvenc NVIDIA NVENC hevc encoder (codec hevc) V..... hevc_qsv HEVC (Intel Quick Sync Video acceleration) (codec hevc) V....D hevc_vaapi H.265/HEVC (VAAPI) (codec hevc) V..... mjpeg_qsv MJPEG (Intel Quick Sync Video acceleration) (codec mjpeg) V....D mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg) V..... mpeg2_qsv MPEG-2 video (Intel Quick Sync Video acceleration) (codec mpeg2video) V....D mpeg2_vaapi MPEG-2 (VAAPI) (codec mpeg2video) V....D vp8_vaapi VP8 (VAAPI) (codec vp8) V....D vp9_vaapi VP9 (VAAPI) (codec vp9) V..... vp9_qsv VP9 video (Intel Quick Sync Video acceleration) (codec vp9)Just the P4 in my build.Yes, just HEVCYes, happy to do so once it's there.@obideuce the Pascal NVENC fix is ready for you to test on a dev build 🎉Quick recap of what changed: instead of swapping our ffmpeg (which would have broken the VMAF quality bench, since the Pascal-capable build ships no libvmaf), the image now carries TWO ffmpeg binaries. The normal BtbN one stays for QSV/VAAPI/x265/bench, and a second jellyfin-ffmpeg binary (older NVENC floor) handles NVENC only. So your Tesla P4 should finally encode instead of falling back to CPU.It's on the dev image, not a release tag yet, I need your P4 to confirm before I ship it.STEP 1 pull the dev image:docker pull ghcr.io/masterjb/x265-butler:devConfirm you actually got a fresh image (the id should change from before):docker inspect --format '{{.Id}}' ghcr.io/masterjb/x265-butler:devSTEP 2 run the NVENC probe against the NEW binary (note: ffmpeg-nvenc, not ffmpeg). Assumes your container is named x265-butler and is running with --runtime=nvidia + NVIDIA_DRIVER_CAPABILITIES=compute,video,utility:docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20WHAT I'M HOPING TO SEE: a "frame= 30" line and exit 0, NOT the old "Required: 13.1 Found: 13.0" refusal. That's the GO signal.STEP 3 sanity check in the Butler UI:• Encoder detection now lists NVENC as working (not a silent libx265 fallback)• The "Test encode" button on the Diagnostics page succeeds for NVENC• A real encode with the encoder set to NVENC completesSTEP 4 (optional, nice-to-have) 10-bit HEVC on the P4:docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -pix_fmt p010le -c:v hevc_nvenc -f null - 2>&1 | tail -20Same "frame= 30" + exit 0 = 10-bit works too. If this one fails but Step 2 passed, no problem — 8-bit is the fix, 10-bit would just be a known gap.Paste back the tail output from Step 2 (and Step 4 if you try it). If it's "frame= 30" I'll tag the release; if it's another "Required: … Found: …" line on the fresh dev image, that tells me the floor is still too high and I re-scope.Thanks for being the P4 test bench, no way to validate this without your hardware 🙏
Wednesday at 03:01 PM4 days 4 hours ago, Human-126094 said:@obideuce the Pascal NVENC fix is ready for you to test on a dev build 🎉Quick recap of what changed: instead of swapping our ffmpeg (which would have broken the VMAF quality bench, since the Pascal-capable build ships no libvmaf), the image now carries TWO ffmpeg binaries. The normal BtbN one stays for QSV/VAAPI/x265/bench, and a second jellyfin-ffmpeg binary (older NVENC floor) handles NVENC only. So your Tesla P4 should finally encode instead of falling back to CPU.It's on the dev image, not a release tag yet, I need your P4 to confirm before I ship it.STEP 1 pull the dev image:docker pull ghcr.io/masterjb/x265-butler:devConfirm you actually got a fresh image (the id should change from before):docker inspect --format '{{.Id}}' ghcr.io/masterjb/x265-butler:devSTEP 2 run the NVENC probe against the NEW binary (note: ffmpeg-nvenc, not ffmpeg). Assumes your container is named x265-butler and is running with --runtime=nvidia + NVIDIA_DRIVER_CAPABILITIES=compute,video,utility:docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20WHAT I'M HOPING TO SEE: a "frame= 30" line and exit 0, NOT the old "Required: 13.1 Found: 13.0" refusal. That's the GO signal.STEP 3 sanity check in the Butler UI:• Encoder detection now lists NVENC as working (not a silent libx265 fallback)• The "Test encode" button on the Diagnostics page succeeds for NVENC• A real encode with the encoder set to NVENC completesSTEP 4 (optional, nice-to-have) 10-bit HEVC on the P4:docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -pix_fmt p010le -c:v hevc_nvenc -f null - 2>&1 | tail -20Same "frame= 30" + exit 0 = 10-bit works too. If this one fails but Step 2 passed, no problem — 8-bit is the fix, 10-bit would just be a known gap.Paste back the tail output from Step 2 (and Step 4 if you try it). If it's "frame= 30" I'll tag the release; if it's another "Required: … Found: …" line on the fresh dev image, that tells me the floor is still too high and I re-scope.Thanks for being the P4 test bench, no way to validate this without your hardware 🙏Outputs for you:root@unraid:~# docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -c:v hevc_nvenc -f null - 2>&1 | tail -20 Input #0, lavfi, from 'testsrc=size=320x240:rate=30': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr, 30 tbn Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_nvenc)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf61.7.103 Stream #0:0: Video: hevc (Rext), gbrp(pc, gbr/unknown/unknown, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 2000 kb/s, 30 fps, 30 tbn Metadata: encoder : Lavc61.19.101 hevc_nvenc Side data: cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A [out#0/null @ 0x562117606300] video:55KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown frame= 30 fps=0.0 q=10.0 Lsize=N/A time=00:00:01.00 bitrate=N/A speed=2.86x root@unraid:~# docker exec x265-butler ffmpeg-nvenc -hide_banner -f lavfi -i testsrc=size=320x240:rate=30 -frames:v 30 -pix_fmt p010le -c:v hevc_nvenc -f null - 2>&1 | tail -20 Input #0, lavfi, from 'testsrc=size=320x240:rate=30': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: wrapped_avframe, rgb24, 320x240 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr, 30 tbn Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_nvenc)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf61.7.103 Stream #0:0: Video: hevc (Main 10), p010le(tv, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 2000 kb/s, 30 fps, 30 tbn Metadata: encoder : Lavc61.19.101 hevc_nvenc Side data: cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A [out#0/null @ 0x55b02e7243c0] video:20KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown frame= 30 fps=0.0 q=10.0 Lsize=N/A time=00:00:01.00 bitrate=N/A speed=2.85x
Thursday at 09:20 AM3 days Author 🚀 v2.41.0 - Pascal/Maxwell NVENC support is liveghcr.io/masterjb/x265-butler:2.41.0 and :latest are up. amd64-only image on a Debian 13 (Trixie) base.Pull / upgrade:docker pull ghcr.io/masterjb/x265-butler:latestdocker restart x265-butlerIf you have an older NVIDIA card, Pascal (Tesla P4, GTX 10-series) or Maxwell (GTX 9-series), NVENC now works. Before this release those cards were refused with "Required: 13.1 Found: 13.0" and silently fell back to CPU encoding, because our ffmpeg build (BtbN) has an NVENC API floor of 13.1 compiled in and those cards top out at 13.0.The fix ships a SECOND ffmpeg - jellyfin-ffmpeg - used only for NVENC. It has an older NVENC floor so Pascal/Maxwell cards encode, and modern RTX cards keep working too (the floor is backward-compatible). Everything else is unchanged: the primary BtbN ffmpeg still handles QSV, VAAPI, libx265, cropdetect, ffprobe and the VMAF quality bench. An encode routes to the jellyfin binary only when the encoder is NVENC. Your existing settings carry over untouched. (Trade-off: the image grew ~100 MB for the second binary.)Escape hatch: if you want NVENC back on the old binary, set container env FFMPEG_NVENC_PATH=ffmpeg (this re-triggers the Pascal refusal - it is a revert lever, not a fix).🟢 NVIDIA NVENC - HOST SETUP RECAP (applies to all versions)If you run an NVIDIA card, NVENC needs the host driver (unRAID NVIDIA-Driver-Plugin or nvidia-container-toolkit) plus these on the container:Extra Parameters:--runtime=nvidiaVariables:NVIDIA_VISIBLE_DEVICES = allNVIDIA_DRIVER_CAPABILITIES = compute,video,utilityThe "video" capability is the one that bites you: the runtime default is compute,utility and WITHOUT video the NVENC session fails to init even though nvidia-smi works fine. utility = the detection probe, compute = CUDA filters, video = NVENC itself.Verify: docker exec x265-butler nvidia-smi -L → expect a "GPU 0: ..." line.♻️ ROLLBACKIf v2.41.0 misbehaves on your hardware, roll back to the previous image:docker stop x265-butler && docker rm x265-butlerdocker pull ghcr.io/masterjb/x265-butler:2.40.0🔭 COMING NEXT• A clearer diagnostic message for the NVENC API-floor case (instead of the cryptic "Required: 13.1 Found: 13.0")Plans can shift, but that's the current direction.More of the same rolling bug-stabilisation from your reports, keep them coming in this thread
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.