foz

Ah, shows how long I've been working on the latest round of changes with this page open - I hadn't refreshed and seen these new messages! Re: the diagnostics bundle @gandalf15 thanks, it helps. The bundle is from build 261, so it shows how things were before the fixes: The service died once a day, about 19 hours apart. That’s the file-handle leak from GitHub issue iamfoz/backblaze-64-personal-wine-container#13. After each death the service couldn’t start again for 30 to 60 minutes, until the watchdog cleared the stuck pass. The “no selection entry” warnings for E: to V: were the false warning fixed in build 279. That’s the one you saw disappear. The current beta fixes the leak itself, so you shouldn’t see those daily stops any more. It also found a file under appdata/Notifiarr the container can’t read. If you don’t need appdata backed up, an exclusion for it is the simplest fix. Re: orphaned bztransmit and the failed file check @rogman, sorry, both of these are down to my bugs. Between 6th and 8th October the watchdog had a rule that stopped Backblaze processes it thought had been left behind. Under Wine every process looks that way, so it was stopping the live upload process about 24 minutes into each pass. That’s the hourly pattern you’re seeing. The rule was removed on 8th October, so rather than a timeout setting, please update to the latest beta (build 301) and the messages will stop. I think the failed file check came from the same thing. The file it names, bz_done_20261007_1.bzff, is the client’s record of what it uploaded on 7th October. Backblaze’s servers check the size of each one, and wrong_size_bz_comb means the copy they hold doesn’t match. That’s what you’d expect if the upload of that record was cut off part way. The client keeps a log of these and sends the file again on later passes. I’m seeing the same failure on my own server since the 7th October kills, so I’m watching to see how it clears and will report back. I’d leave bzdc_filecheck.xml alone. It’s the server’s verdict written down locally, so editing it only hides the message, and the next check rewrites it. In the meantime, you can switch that warning off under Settings > Warnings and events. Dismissing it should keep it hidden until you press Reset warnings. If it comes back after a plain dismiss, can you tell me which build you’re on? The file you pasted also includes your hguid, Backblaze’s ID for your computer. It’s best kept out of public posts, so you may want to edit it out.

It’s been a busy few days, mostly on the beta. I started work on the issues raised here, then found more bugs to squash, and a filed issue on GitHub also needed attention. The file-handle leak Client 10.0.3.1075’s service opens its Backup.sql database every few seconds. Under Wine, none of those handles ever closed, so they built up until Wine hit its 40,960 open-file limit. On my server that took about six hours, and on the reporter’s about nineteen. After that, nothing in the container could open a file. Backup passes hung, and eventually the service died. If your bzserv has been stopping about once a day since 1075, this is why. The cause was in Wine. SQLite keeps its database locked, and Wine was holding every closed handle on a locked file open because of how Linux file locks work. The beta (build 299 onwards) is now on Wine 11.19, with a patch that uses a different kind of Linux lock, so it releases closed handles. On my server, the count for that file now holds at 4. On the old build, it passed 400 within twelve minutes. 11.19 also fixes the 11.18 start-up fault I found when it was briefly packaged with the beta image. Both images also have a fallback: With Automatic recovery on, the watchdog restarts the Backblaze service when Wine’s open files reach 85% of the limit (FD_RESTART_PCT). bb-doctor now reports the count against the limit. The stable images stay on standard Wine, so they rely on that restart. If you’re using /config on /mnt/user/appdata/..., consider moving it to /mnt/cache/appdata/.... Every file Wine holds open is also held open by shfs, which shares one limit with everything else on /mnt/user. In the issue, the leak ran shfs out, and a PostgreSQL container started crashing. A bug in autorecovery Between 6th and 8th October, the watchdog had a rule meant to stop stray Backblaze processes. It decided a process was stray when its parent was the container’s init process. Wine starts every Windows process that way, so the rule stopped scans and, once, a live upload process, and the pass hung waiting for it. Nothing was lost, because the client redoes the work, but if you pulled an image in that window and have Automatic recovery on, please update; the code is gone. Other changes in the beta An Exclusions panel on the Settings tab (more on this below). bb-doctor names any directory the scanner couldn’t open, with the reason. Windows rejects names containing ", and names ending in a dot, and the container user can’t read some ownerships. When the scanner hits one of these, it skips the folder silently, and the only sign was “producing file lists” for hours. On my server, every scan for three days had been stopping at one folder with quotes in its name. The Monitor shows a warning while such a folder exists, and it can be turned off in Settings. bb-doctor warns when a mapped drive’s folders are separate filesystems. The client doesn’t cross mount points. On Unraid, each share on a ZFS pool is its own dataset, so mapping /mnt/cache as one drive backs up nothing. Map /mnt/cache/<share> instead. bb-doctor’s drive selection check now reads the client’s own volume list (see the reply on drive N: below). The watchdog now starts the Backblaze service itself if it has stayed down for more than one check. Two people had it sit down for hours. Diagnostic bundles now include the recovery log and the end of the service’s own log. The Status tab’s drive bars are in drive-letter order and line up. They no longer jump around when the ETA changes width. The stable images moved to the latest base image (v4.14.0) and picked up the watchdog and bb-doctor changes. They’re still on standard Wine 11.0. Re: mapping disks, the XML exclusions file and the strace results @nick5429, thanks for taking the measurements, which found something I’d missed. Every time Wine fails to find a file under the exact name it was given, it checks whether the folder is case sensitive. On a FUSE filesystem such as shfs, part of that check is a statfs call. shfs answers it by adding up free space across every disk the share spans, so each call takes about 21 ms. Your trace had it at nearly half the wall time of a pass. The beta (build 299 onwards) now carries a patch that makes the decision once per device and caches it. I’ve moved my own server to disk paths, so I haven’t been able to measure it on user shares yet - I'll be adding them back after my backup completes. If you still have the user-share container, I’d be glad to see a before and after. Mapping /mnt/diskN and the pool shares is still the faster setup, since shfs is slower at everything else too. I’ll be raising the Wine side upstream: there’s an existing Wine bug for slow FUSE filesystems that this adds weight to. On the exclusions file: you were right that it’s the sensible way to exclude a folder across twenty disks. The Settings tab now has an Exclusions panel that manages the rules in bzexcluderules_editable.xml. It only touches its own rules between two marker comments, so your hand-written rules stay exactly as they are. It keeps the previous file alongside, and its rules are included in the settings export. One tip for anyone switching to per-disk drives: the client abandons a drive selection saved while it’s uploading (there’s just a line in its log). Pause the backup, untick the drive and press OK, tick it and press OK again, then resume. Before mapping /mnt/user alongside the disks, check that /mnt/user/.bzvol doesn’t show one of the disks’ stamps, because the client recognises drives by that stamp. Re: drive N: not showing on Status, and bb-doctor saying nothing is selected @rogman, glad it sorted itself out. I suspect two things lined up: The Status tab only shows a drive once the client has written a file list for it, and after a switch to per-disk drives the client rescans each disk, which can take a while. bb-doctor’s selection check was looking in the wrong place. It looked for a per-drive entry the client only writes when you change a drive from its default, so a drive could be selected and still read as “nothing selected”. From build 279, it reads the client’s volume list, the list the client works from. If it comes back on a current beta, a bb-report bundle would help me pin it down.

Yes, this is an issue I had discovered during bug fixing. The beta build of 28 September, build 257, the one your bundle shows, has a bug in the part of the container that restarts the Backblaze service when it dies. The first time the service stopped after that build, the restart routine called itself in a loop instead of restarting anything. The watchdog saw the service was down every half hour and handed the job to a routine that was no longer working, which is the pattern in your timeline. The fix has been in :beta since 30 September, and today's beta adds a second line of defence: if the service is still down half an hour after the watchdog first reports it, the watchdog starts it itself instead of waiting on the service watch. Pull the latest beta and recreate the container from the Unraid template: docker pull iamfoz/backblaze-64-personal-wine-container:betaThe broken routine in your running container is also putting load on a CPU core, and has been since the service died. Recreating the container ends that as well. Your pass had been waiting on its upload children for 37 minutes before everything stopped, the same as on my own host the day before. I am looking at what ends those waits. A diagnostic bundle from the new build after it has run for a day would help, since from this build the bundle includes the recovery log and the service's own log. Hopefully the changes have squashed this bug!

Thanks for the bundle, it helped me narrow down the bug quickly. Your backup is fine: the latest pass went all the way through and recorded its files, and nothing in the transfer logs mentions an inherit. What happened is that your client never wrote the final "done" stage into its progress file; it stopped at "after files swap, 60%", most likely because the container was restarted during the last step, and my code treated anything without a "done" message as still running. That is why the notice, the terminal monitor and bb-doctor all kept reporting the apparently still in-progress state. The next beta now reads it the way the client behaves: a progress file that hasn't changed for an hour while passes have run since counts as finished. The notice goes away, and bb-doctor notes that the inherit finished earlier and where its file stopped. Once you update to the latest release, these messages should clear automatically. My own backup is only about a quarter of the way through its first backup (it’s quite large!) and I couldn’t inherit as my prior client was a Mac, so this scenario hadn’t come up for me; I’m glad for reports like yours - the software should work well for everyone!

Interesting little rabbit hole to go down, but relatively straightforward to correct: The two numbers are both right; they just count different things. The container figure is the kernel's memory charge for the container, and that includes the page cache for every file the client has read. Your dedup scan reads every file on those disks, so the cache grows into the tens of gigabytes and gets booked against the container. Unraid's dashboard counts cache as free, which is why it shows 14 GiB used and 43 GiB free at the same moment. The client's processes themselves are holding a few gigabytes, and the cache gives way to anything else that needs the memory. So nothing to worry about, and it will shrink once the scan is through. The next beta shows this on the Monitor: the processes' own memory on the gauge, with the cache figure beside it, so the figures are clearer. It also has a cleaner bb-doctor check on Swap - if the host has a reasonable amount of RAM (I went with 16GB), the line will report as info only if swap is still not configured. However, if repeated OOM events occur, this will turn back into a warning to hopefully keep the diagnostics relevant and useful.

Hi @rogman, I happened to be doing some bug fixing (there seem to be lots of edge cases where the backup can hang to squash) and the drive check was hard-coded to 3 levels, which will easily miss files if you've got things laid out with deeper categorisation. I've left the 3-level setting as the default, as this should do for most and is a good balance for speed, but in Settings in the next release (beta 247), you will be able to alter the depth yourself or select unlimited search with a 60-second timeout per drive. Edit: The swap message is for OS swap, not the temporary drive setting in Backblaze. If you've got plenty of RAM, then this shouldn't be an issue. The problem is that Backblaze can peak at some extremely high usage, so it's hard to say what a reasonable cutoff is. I'll have a think and maybe make this some sort of setting too. If there's a reasonable calculation I can make with regard to historic usage, then it might be something I can automatically mark as OK.

Thanks for confirming. The ten-second idle is the parent process reading the next chunk. Your log from before the change shows the client had sixteen upload slots open, and each child finished its 10 MB chunk in about three seconds, but the parent only handed out a new chunk every two seconds. Before each launch it reads the next 10 MB from the source drive, hashes it and stages it, and two seconds for 10 MB is a 5 MB/s read. However many threads you allow, the parent cannot feed them faster than it can read. Your drives are mapped through /mnt/user, so every read goes through Unraid's shfs layer, and long sequential reads like these are what it is slowest at. Change the container's drive mappings to the disk or pool path underneath, /mnt/cache/share or /mnt/diskN/share rather than /mnt/user/share, and do the same for the temporary drive. That takes shfs out of the read path. Then watch the chunks per minute on the Monitor; the parent's cadence is what should change. Before changing anything, pull :beta again to update to build 236 and run bb-doctor. The Source drives section now reads 64 MB from a large file on each drive and prints the speed, with how many chunks a minute the pass can stage from it. Under 20 MB/s on the drive your big files live on and the mapping should fix it. If it comes out fast, the bottleneck is somewhere else. In this case, if you could give me another diagnostic bundle, I'll investigate further.

Turns out it’s not a problem with your drives; it's a bug in the check I wrote, sorry. Your client keeps one volume record for every drive that has ever been assigned a particular letter, and the check compared your stamp against the first record instead of all of them, then found it under D: anyway and printed nonsense. The ID in that warning is the one your own client log reports as attached, so D: is fine. The fix is building now; pull :beta again in half an hour and bb-doctor should read every drive properly. Good to know D: is cache-only. Then the write failures at 03:11 weren't the mover on that share, and the thing to look at next is where the client's own data folder lives, which is C: inside appdata, and whether anything moved or locked files under /config at that time.

@rogman A settings export and import function has been added to the system. It allows you to specify an encryption password to export and import API tokens and notification service strings as well. It also stores Backblaze's own configuration and will restore that to a fresh container too, but this will not store or restore the backup state itself - that remains in appdata and/or the drive .bzvol directories and must be backed up separately or you need to use the inherit state function. @gandalf15 Thanks for the bundle. It has two issues, and the second is the one that needs to be fixed first. Your backup has been stopped since 03:11 this morning, not just slow. At that moment, the client failed to write to its own data folder and to the scratch folder on D:, lost its four-hour lock, and no pass has run since. Those errors are between 03:01 and 03:11, which looks like the Unraid mover moving files out from under the client. Check the mover schedule, and make sure the appdata share and whatever share holds Backblaze's scratch folder are set to cache only. Then restart the container, and update to the current :beta while you're at it: it now notices a dead service or a stuck pass and restarts them, whereas build 199 sat there saying all was OK. The slowness is the scratch folder. Backblaze's large-file path deletes and rewrites every chunk through that folder on D:, an shfs user share, before each upload child starts, and that is why only two or three threads are ever busy and "Preparing" takes so long. Point the client's temporary data drive at a cache-only share, or better a disk path, and the parent stops being the bottleneck. On the D: warning: that one was my check's fault, and it's fixed in the current :beta. bb-doctor looked for a volume ID incorrectly, so it warned about D: on a stray token and said nothing about the drives with a real problem. Your own client log lists F: and G: as not attached, with no volume ID, which means those two are not being backed up whatever the settings window shows. Run bb-doctor on the new build and it will say, per drive, which of four states it is in: the client knows the ID under another letter, the client has a different ID for that letter, the drive is stamped for a different computer identity, or the client has no record of the drive at all. For the middle two there is a value the client itself wrote elsewhere, and bb-doctor --fix restores it into the drive's .bzvol stamp, keeping the old stamp beside it. Restart the container afterwards so the client re-reads it. For the last one there is nothing to restore: untick and re-tick the drive in the client's settings, and expect that drive's upload to start over. Whatever it says, never delete the .bzvol folder on a drive. Backblaze's own README in there says the drive's backed-up files go with it. While you have it running, --fix also now starts the Backblaze service if it has died, stops a pass that is stuck waiting on a lost upload child so a fresh one starts, and rewrites the supportedOS manifest if it is missing. Plain bb-doctor reports all of those without changing anything.

New beta build with the things reported over the last day or so, and some other odds, ends and bugfixes. @rogman, the Status tab went blank in build 207 due to a stray apostrophe in one of the notices that broke the page's script, that's been fixed. As per your request, client update notifications are in. Warnings on the Monitor now have an X to dismiss them, one per warning, and warnings dismissed on the Status tab no longer show on the Monitor band either. Two can't be hidden, because both mean nothing is being backed up: a safety freeze, and passes losing their four-hour lock. Dismissed warnings stay listed in Settings until you press Reset warnings. Settings now has one "Warnings and events" list in place of the Events list under Notifications and the separate Warnings box. Each warning has a Show box and, where there's a matching notification, a Notify box. Reset warnings puts dismissed warnings back without touching those choices. On a phone, the "Preparing" and "Finishing" line in Uploading now wraps instead of pushing the panel off the screen. The Status panel title no longer sits on the top bar, and the warnings list is one card per warning rather than a stack of loose lines. The Backup client section in Settings fills in once the first client reading arrives. It used to ask once when the page opened and then give up, so a page opened in the first minute after a restart stayed empty until you reloaded it. @anna, on the timestamps being two hours behind: Backblaze writes its own log files in UTC, whatever TZ is set to. The dashboard converts to local time where it shows log events, and the container's own logs follow TZ, so it matters where you saw it. If it was in the raw Backblaze logs, through the Tools tab, bb-report or a shell, that's expected. If it was on a dashboard screen, in the terminal monitor or in one of the bb-* tools, tell me which one and what time it showed against the time it should have been, and I'll fix it. All the 'pretty' user-facing UX should show you everything in your local timezone (or whatever you've set your TZ to). Pull :beta and it's all there. As usual, any issues or requests, let me know. As I mentioned previously, I've pushed a fix for the client issue to stable, but if all of this work in the beta is holding up, I'll look to do a proper release soon, as there's now quite a gap between the two now.

Follow-up on the "producing file lists" problem: I've found why client 10.0.3.1075 fails under Wine, fixed it in the beta, and the beta now runs 1075 with passes completing. Stable has a fix of its own. Details and what to do below. What 1075 does differently. Its service, bzserv, checks on each backup pass every ten seconds by opening the bztransmit process. The pass locks its own process down so that nobody can open it. On Windows, the service gets in anyway, because it runs as the system account, which holds a privilege that overrides that lock. Wine didn't honour that privilege, so the check failed, bzserv decided the pass had died, deleted the four-hour lock and started a new pass, and the old one, still running, gave up when it found its lock gone. 1069's service never does the check, which is why it always worked. So one correction to my last post: the lock file was not on disk the whole time. bzserv was deleting it, and tracing which process did that is what led to the cause. The beta's Wine is built from source, so I've added the missing behaviour there. On my server, a 1075 pass now runs from STARTBACKUP through to TransmitFilesInOneTodoFile with no lock error, and the pass-check command from my last post shows the healthy sequence. The current :beta has that fix, Wine 11.18, a newer base image, and no client pin, so it follows Backblaze's newest client again. Stable uses Wine from WineHQ's packages, which I want to receive the patches via inclusion in the source, so stable v10.2.2 (out today) pins the client to 10.0.1.1069 and stops the updater. If you're on stable and restarted since mid-September, you have 1075 too, and updating the container puts 1069 back on its own. No template changes needed. If you're on the beta: If you did the manual rollback and set Disable Auto-Update to true: pull :beta, then either leave that setting to stay on 1069, or set it back to false and restart to move to 1075. Both work on the new beta. If you'd rather hold one exact version, the BACKBLAZE_VERSION variable does that and wins over the update settings. The template you installed from doesn't have it, so add it by hand with "Add another Path, Port, Variable, Label or Device" on the container's edit page. Leave it empty to let the image decide. @anna, the lock error after an unclean shutdown is different from this one. A pass that dies without releasing its lock leaves the next passes failing to grab it until the client clears it, which it does by itself after a while. The repeating one, every pass, every minute, was 1075. If passes still don't complete after updating, run the pass check command from my last post and paste the result, along with a diag bundle.

Hey all, busy as usual so apologies for the delay, but I've got to the bottom of the "producing file lists" problem, with thanks to everyone who posted logs here and on GitHub. The cause is the Backblaze client itself, version 10.0.3.1075, installed by the auto-update function. Backblaze released 1075 in the same week as beta 181, and the container's updater installs the newest client on every start, so anyone who restarted since about 14 September got it. Under Wine, 1075 loses its own four-hour lock on every backup pass. The lock file is on disk; the client's check seems to miss it. The pass aborts at the transmit step, bzserv starts another a few minutes later, and that one does the same. Meanwhile, large files keep uploading through the chunk path, so the rate looks fine and bb-health says OK. The GUI stays on "Producing file lists" as that's the stage every pass restarts from. To @rogman's question about the coincidence: it was both at once, and I've separated them. I ran 1075 on a container with the monitor unable to touch Wine at all, and it failed the same way. 10.0.1.1069 on the same image completes passes. The GitHub reporter's bundle shows the same thing from the other side: every push line before their switch carries 10.0.1.1069, and their startapp.log shows the container installing 1075 at the moment they pulled the new image. @Shadowrunner was close. The file stays on disk throughout, and the client fails its own check for it. Creating it by hand doesn't help for the same reason, and the "Failed to grab fourHourLock" lines that follow are the next passes finding the lock the dead one left. @anna, I'd expect yours to come back; a lucky pass doesn't necessarily indicate the client is working. The fix is in the beta that's building now: The container pins the client to 10.0.1.1069 by default and installs it over whatever is there, in place. Pull :beta, restart, and the next pass should complete. There's a new variable, BACKBLAZE_VERSION; set it empty when you want the updater back. The Status tab, the terminal monitor, bb-doctor, the metrics and notifications now warn when passes are losing the lock, so this can't hide behind a healthy-looking upload rate again. If you don't want to wait, this does the same thing on the image you have. It copies the 1069 files over the install and runs Backblaze's installer. Backblaze64 is the template's default container name; substitute yours if you changed it: U="$(docker exec Backblaze64 stat -c '%u:%g' /config)" docker exec -u "$U" -e HOME=/config -e WINEPREFIX=/config/wine/ Backblaze64 sh -c 'cd /config/wine/dosdevices/c:/ && curl -fsSL "https://secure.backblaze.com/api/install_backblaze?file=bzinstall-win32-10.0.1.1069.exe" -o install_1069.exe && rm -rf bzextract1069 && mkdir bzextract1069 && 7z x -y -obzextract1069 install_1069.exe >/dev/null && d="$(dirname "$(find bzextract1069 -iname bzdoinstall.exe | head -1)")" && b="/config/wine/drive_c/Program Files/Backblaze" && cp "$d"/bz*.exe "$d"/*.dll "$d"/*.xml "$d"/*.gif "$d"/*.ico "$d"/*.txt "$b/" 2>/dev/null; cd "$d" && wine bzdoinstall.exe -doinstall "$(winepath -w "$d")"; echo "exit=$?"'Then set Disable Auto-Update to true in the container template before it restarts, or the updater puts 1075 straight back. Please don't delete Program Files\Backblaze to force a reinstall. The machine identity lives in bzinstall.xml in that directory. Without it the client treats your server as a new computer and you have to inherit your backup state, which I learned the slow way tonight. Copying over the top keeps it. To check a pass is completing, this shows the last few pass events from the transmit log: docker exec Backblaze64 sh -c 'cd "/config/wine/dosdevices/c:/ProgramData/Backblaze/bzdata/bzlogs/bztransmit" && grep -h -E "STARTBACKUP|Finished B reading|TransmitFilesInOneTodoFile|Lost four hour lock" "$(ls -t *.log | head -1)" | tail -8 | cut -c1-120'Healthy is STARTBACKUP, then Finished B reading, then TransmitFilesInOneTodoFile, and no Lost four hour lock after it. This will keep everything working alongside the most current beta, I will of course be investigating what is causing trouble with the new client and seeing if its something we can ameliorate. If you're still having problems, let me know, and I'll investigate further.

Thank you, and it is my pleasure. It's gratifying to see people getting value from my work. I think you've already noticed the update, but I added some specific templates for the various popular notification tools as well as the option of building your own output - my fault for half-assing it a bit in the first place. A few other bug fixes and cleanups as well.

@rogman, not selfish at all. A very useful request, and one that broke me out of my (slightly nostalgic) CLI reverie. Beta 156 has a Tools tab. It runs bb-doctor, bb-health and bb-version from the page and shows what they print and allows you to interact with options and outputs as you'd expect. Notes are on the page, but let me know if there are questions. A bug. docker exec enters the container as root, and root passes every permission test. bb-doctor decides whether a skipped file is readable with a permission test, and whether /config is writable with another, so from the console both always said yes. If your share had the wrong owner, which is the most common fault this container has, running bb-doctor the way the README told you to said your files could be read. The Tools tab runs as the container user, which is the user whose permissions matter, so it gives the true answer. And bb-doctor itself now re-runs as that user when it is started as root, so the console gives the true answer as well. If you ran bb-doctor from the console before 150 and it said your files were fine, it may have been wrong. Status tab. Skipped Files is now Status, and it leads with everything the client says needs attention: warnings, a pause with the reason, a first backup running or finished, and then the skipped list. The pause part is the one I care about. The client writes a reason code with every pause and we were showing a flag. Now it says "Paused from here" when you or the API paused it, and "Paused by the client" with the cause when the client chose to, for example when Backblaze's cluster authority is not answering, which is usually their maintenance and resumes on its own. One is a button, the other is a wait, and the page says which. A Copy status summary button gives five lines of plain text for a post like this one: build, state, progress, health, today's uploads. Nothing in it names a file. Notifications. The API tab has a Notifications section. Point it at an ntfy topic or a webhook that takes JSON, tick the events, and the container tells your phone when the backup needs you: a safety freeze, files skipped past a threshold, no completed backup within the client's own limit, a stall, a pause the client chose, first backup complete, a milestone. Once when it starts, once when it clears, never on every poll, and nothing that names a file is ever sent. A Test button checks the endpoint. Quiet hours. Also in the API tab. Windows by day of the week with a start and end; the container asks the client to pause at the start and starts the backup at the end. The client's own pause lasts about two hours, so inside a window the container renews it. If you start the backup yourself during a window it stays running until the window ends. bb-doctor. Two new checks. It samples each mapped drive as the container user and names anything it cannot read, with the owner and the chown line, before the client has to give up on files. And it compares the volume ID the client wrote under .bzvol with the volumes the client knows about; a drive it no longer recognises is the "No files are selected" state after an inherit, which the last thread hit and nothing named. bb-health reports FROZEN when Backblaze has frozen the backup, so the Docker tab goes unhealthy for that state. API. A new switch allows you to turn it off without revoking keys; off, everything returns 404 and the keys are kept. Revoked and expired keys have a Delete button. New endpoints: /summary, /metrics in Prometheus format, /events as a server-sent stream on state changes, /tools behind new diagnose and diagnose:repair permissions, and /openapi.json. The OpenAPI document is also in the repository. Smaller things. All thirteen themes now apply to every tab; before, the Status, Tools and API pages used the default. Text uses the width of the window instead of wrapping at about 375 pixels. Keys: 1 to 5 switch tabs, s opens settings, p pauses or resumes. The browser tab title carries a mark for uploading, paused, or a warning. The desktop pane shows a Reconnect button when noVNC drops its connection, which after a container restart used to be a blank pane. A Today line under the seven-day chart, and a progress-over-time line in About.

Hi @YourNekoWaifu, welcome - this is normal when the client restarts and is most noticeable on large backup sets. The client will scan the set, which involves putting a file list into memory. My own set is 90TB, and my memory usage will hit 10GB easily during this phase. As @rogman said, the best way to manage this is to either have a large amount of RAM on hand or configure Swap. I'm currently running off of an N100 stick computer with 12GB of RAM and Proxmox, so the Backblaze process can easily overwhelm the host, which I've resolved with a Swap partition. After the file processing and the backup begins, memory use will fall back. Thread numbers will also contribute to this pressure. Backblaze will prioritise files by size, so each run will always start with new and changed small files, then go on to larger files. After about 100MB, they are chunked by 10MB pieces. Each piece will use a thread to upload, and this is when users with decent upload bandwidth will find the most effective use of that bandwidth comes into play. Hi @DevanteWeary, welcome. Good news first: nothing is deleted. A Safety Freeze is Backblaze protecting a backup when something looks wrong with the drive underneath it. Your data is still on their side, and the freeze stops any sudden catastrophic changes. Backblaze have a page on clearing one yourself: https://www.backblaze.com/computer-backup/docs/resolve-a-safety-freeze Read it before you touch anything. The last step is re-licensing with Inherit Backup State, and that's where it's possible to pick the wrong computer record and lose the link to the backup you're trying to save. You've been through that screen once already, so you know it. Just be careful which entry you choose. The uninstall and reinstall step is easier here than on Windows. Delete /config/wine/drive_c/Program Files/Backblaze/ and restart the container. It downloads the client again and reinstalls it on the next start, which is the same thing it does when a version update comes through. Your appdata and the client's own data are in a different place and aren't touched by that, so the licence and the inherit history survive. What I can't tell you is whether that's enough. Backblaze's uninstall step may be meant to clear the client's local state as well, and the container's reinstall keeps it. If the freeze is still there afterwards, say so before you go further, because the next thing anyone would reach for is wiping that state, and that's the part with real risk attached. Before any of that, I'd fix the thing that probably caused it. The first two symptoms were a red herring. "No files are selected" and "Initial Backup Paused" is what the window says while the client is still building its file list, which on a large share takes hours. You saw that resolve itself: correct size, correct count, then "preparing files". Nothing was wrong there. The permissions skips are the part that matters. New container, new appdata, so User ID and Group ID are back at their defaults of 99 and 100. If your shares are owned by something else, the container can't read them, and a large slice of an existing backup suddenly going unreadable is the kind of thing I'd expect to look like an altering event from Backblaze's side. Hypothetical rather than diagnostic, but it fits both things you've reported. On the latest beta, run this: docker exec backblaze-personal-wine bb-doctorIt reads the client's own skipped-file list and works out, for each file, whether it's actually gone or just unreadable from here. Where it can't read something, it prints the directory with its owner and mode, next to the UID the container runs as. If it comes back with a list of directories, that's the answer. Fix ownership on the host with the command it prints. Do it on the host rather than inside the container, and don't blanket-chown the whole share. Get that clean first, then work through Backblaze's page. On .bzvol: today's date on those files is normal, the client updates them. The 2023 date on bzscratch and the empty bzcurrentlargefile are normal too. That's the staging area for files over 100 MB, and it's empty because nothing large is in flight. @gandalf15 I think yours is a different thing. "Initial Backup Paused" with a Continue button is the client's own pause rather than a freeze, and clicking Continue is the right answer. If it starts happening regularly, I can take a look, but it is normal for the client to cycle through these states by itself from time to time. And as noted, I've been working on some updates and bugfixes on the beta channel, here's what's changed: The memory gauge now names the program using the memory. A container partway through its first backup announced that the backup was complete, and latched that to disk for a week. The completion check shared a helper with the staleness warning, and that helper answers "yes" when there are no totals to judge against. Right for a warning, where a missing figure should not raise a false alarm. Wrong for declaring something finished, and during a file-list scan the totals are absent. It now needs positive evidence. ETA sanity took a pass. A backup starts on its small files, and a handful of those gives a per-byte rate you can extrapolate into geological time. One run read "4259966d", about 11,600 years, and the trend feature then compared today's nonsense against yesterday's. Estimates beyond a century are withheld now, and the trend ignores anything resting on fewer than three completed transfers. A grim estimate still shows if it is real: 100 TB on a 1 Mbit/s uplink is about 25 years, and that is accurate. With nothing usable, it reads "not yet". The failure counter no longer says "failed" as this was misleading. The number it showed is attempts a storage vault turned away for being busy or full. The client retries against another vault and the file goes up. It reads "retried" now, without the alarm colour, and the API carries the breakdown by reason. The count that means data is not backed up is the skipped-file count, which is reported separately and loudly. Pause reporting got two corrections, and one of them was me being wrong. The monitor said "Paused" the moment the request landed, while the client was still finishing the transfers already in flight. Backblaze's own window looked like it had not noticed, because it was finishing up. A pause reads "Pausing" until it takes. The monitors also said "Preparing <file>" next to a state of Paused, which reads as stalled; the client parks on the file it was about to take, so while a pause is set that is the next file rather than the current one. This may be what you were experiencing with the API @rogman - let me know if this is still a problem. Backblaze keeps a list of files it has given up on, with a reason against each one. It does not queue them, it does not retry them, and nothing in the desktop window tells you they exist. A backup can look finished while a directory's worth of files was never sent. The beta surfaces that list in three places now: a warning in both monitors, a Skipped Files tab in the web interface with a filter and a breakdown by reason, and bb-doctor, which goes further and works out why. bb-doctor reads a sample of the list and sorts it into three groups. Files that no longer exist, which need nothing done. Files that are readable again and will clear at the next scan. And files this container still cannot read, where it names the directory, prints its owner and mode next to the UID the container runs as, and gives you the command to run on the host. It repairs nothing, even with --fix. These are your files on a mounted share, often thousands of them, and other software on your host may depend on who owns them. Two bugs found and fixed in that check: the list is written by a Windows program, so every line ends in CRLF, and the shell left the carriage return on the end of each path. The same check counted the report's own header as a skipped file, so a clean list reported two. The monitor was never affected, because Python strips line endings where the shell does not. Smaller things. Buttons and chips in the dark themes had near-black text hardcoded on them, around 1.5:1 against dark grey, so they were effectively invisible. bb-monitor run without a terminal gracefully exits and indicates the issue instead of dying in a curses traceback, which is what docker exec without -it gets you. There is a seven-day upload chart, a breakdown of what the backup is made of by category, and a "backing up since" date.

Sorry I've been away for a bit, life likes to keep me busy! The Issues tab won't show them, which is why the monitor reads them itself. The client keeps its own list that nothing in the GUI surfaces: docker exec Backblaze64 sh -c 'F="/config/wine/dosdevices/c:/ProgramData/Backblaze/bzdata/bzreports/bzlist_skipped_files.txt"; echo "=== reasons ==="; cut -f2 "$F" | sort | uniq -c; echo "=== files ==="; cut -f2,3 "$F"' Substitute your container name if it isn't Backblaze64. With five files it's worth reading the actual reasons rather than assuming, but under this container the common one is BZ_FILE_PERMANENT_BAD_PERMISSIONS, which means the container user couldn't read the file. That points at ownership or permissions on the mounted share rather than anything Backblaze has done. Mine was 770 files in a single directory that had ended up owned by root with 0700; a chown fixed it, and they cleared on the next scan. The thing worth knowing either way is that skipped files aren't queued and aren't retried. They're just not backed up, and until the monitor started reading that file there was nothing telling you so. Post the reasons column and I'll tell you what yours are.

Beta 132 is up and the big thing in it is an HTTP API. @rogman Your dashboard plugin looks awesome! I have had a dashboard plugin of my own on the roadmap, but it has sat well down the list behind the Wine work and the monitor, and I also knew that it would hit a wall: there is no sensible way for a plugin to get data out of this container. The monitor's own feed is behind the web login, its shape is whatever the page needs that week, and I would be changing it under anyone building against it. An API was always the part that mattered, and the plugin was the easy bit afterwards. Then you posted yours, and finishing the API properly became a thing worth doing. That is what this build is: it exists so your plugin has something solid to talk to. I will still finish mine at some point, but it is the smaller job of the two and there is no hurry, after all the API is designed for others to be able to make cool stuff with. That goes for anyone, not only rogman. If you want to build something that talks to this container, the API is there for it, and I would rather you used it than scraped a page - it should be considerably more stable this way. Your tile polls /monitor/api/status, which sits behind whatever Web Authentication the container is set to, so it only works with that turned off. That is not a limitation of your plugin, it is that there was nothing else to point at. There is now: /api/v1/status authenticates with a key instead of the browser login, so it works whether authentication is on or off, and it is stable rather than a page's internals. Same port, no extra ports to open. It also carries a lot more than the page feed does, in raw numbers rather than the strings the dashboard draws, so you can format for yourself: bytes and seconds rather than "19.5 TB" and "3 days". Turning it onIt is off until you make a key, and answers 404 until then, so nothing changes for anyone who does not want it. There is a new API tab in the web interface, or from a terminal: docker exec backblaze-personal-wine bb-apikey create --label "dashboard" --scope readThe key is shown once. Then: curl -H "Authorization: Bearer <key>" https://<host>:<port>/api/v1/statusPermissions are granted per operation. A status display gets read and nothing else: not the names of your files, and no ability to touch the backup. If you want something that can start or pause a backup, that is a separate permission again, and starting is separate from pausing. Full reference, every field and what it means, is in the repository at docs/api-v1.md. What anyone might do with itNot just dashboards. Some things it would support today: A Home Assistant sensor, with an automation that tells you when the backup has not completed in a week. A phone notification through ntfy or Gotify when files start getting skipped, which is the one that would have actually saved me something. A Grafana panel, if you already run one, since everything comes out as numbers. A cron job that starts a backup after your array finishes a parity check. A Stream Deck button that pauses the backup while you are gaming, and starts it again after. Another Unraid plugin, if someone wants a different take on the tile. The point is that the container should not care what is on the other end. If you build something with it and hit a field that is missing or wrong, say so and I will add it. FixesMulti-part totals were wrong again, in a new way: a 221 MB file with 10 MB parts showing as "22/236". The part size is fixed for a file, but one bad reading could set a row's total for good. It cannot now. Also in: the monitor says when a backup is paused rather than reading "Uploading" with nothing moving, a key created on the command line is visible to the service rather than silently invisible, and the dashboard reuses its connection instead of opening a new one every two seconds. @gandalf15 on threads - setting 100 and seeing 7 or 8 is expected: that setting is a ceiling, not a target, and the client only opens what it has work for. Chunks of one file go out in parallel, but the client is not queueing a hundred at once. Where you are setting threads, this must mean you have turned the auto throttle off. Try moving the slider to faster backups if you haven't already, and maybe using the loading trick to see if Backblaze is causing odd things to happen due to lack of system pressure. My own system is currently uploading ~200MB files, which get about 20 threads at 90 Mbit/s. I've set Backblaze to manual throttle, faster backups and max 30 threads. For your 16 MB/s at 17 ms, the arithmetic that matters is per connection rather than overall. Each connection is limited to roughly its send buffer divided by the round trip, and under Wine that buffer is 64 KB, which at 17 ms is about 3.8 MB/s per thread. Eight threads gives about 30 MB/s as a ceiling, so 16 is short of it but the same order. More threads would help you more than anything else, which is the opposite of what you would expect, and it is the send buffer rather than your line. Some of this may change in the future with the work I'm doing with Wine. As always this is beta only. Stable is unchanged at 10.2.1. Quote the build number if you report anything.

Beta 127 is up. There has been a few changes since 110, most of it from going through the files the Backblaze client keeps in bzdata to see what it already knows and does not show you. The monitor reports what the client is doing rather than guessing from which processes are running. The client writes its own status out, and it turns out the field I was reading first (cur_state) says "transmitting" no matter what, including while it is building file lists. The useful one names the file in hand, and for a large file it says which phase it is in: preparing, part 12 of 21, then finishing. A file-list scan now shows a real percentage, directories indexed out of the total, with a running count of files found, rather than just the word "Scanning". Chunked files can now be shown in a compact view. Every chunk of the large file being split has an index, a byte offset and a SHA-1 on disk, and that same SHA-1 turns up in each transfer's own record, so a thread can be matched to the exact chunk it is carrying. Chunks fill in where they actually sit in the file, out of order, with the ones in flight marked differently. Chunks that finished before you opened the monitor stay unmarked, because there is no way to tell those from ones not yet started. It is off by default, next to the theme picker. Files Backblaze has given up on are now listed. The client keeps a list of files it has skipped with a reason, and it neither queues nor retries them, so nothing else tells you they are unprotected. When I turned this on against my own machine it found 770 skipped files. All of them were in one directory that had ended up owned by root with 0700 permissions, so the container user could not read it. About 28 GB I believed was backed up and was not. Under this container, the cause is nearly always ownership of the mounted source rather than anything Backblaze has done, and it is invisible until something asks for the list. If you run nothing else from this build, look at that number. A first backup is still working through the set now says so, with how long it has been going and how far it has got, instead of looking like something is stuck. Mine reads day 67 and 12.9%. There is no "initial backup finished" flag anywhere, so this is worked out from how much of the set has never been sent. Also in: the count of uploads for the day with failures broken out, the bytes compression has saved, and Backblaze's own measured throughput figure. It reports about 3.5 Mbit/s per connection for files over a megabyte here, and 244 kbit/s for small ones, which is a much worse case dominated by a round trip per file rather than by the socket buffer. The web dashboard is mobile-friendly now, so it is usable from a phone. FixesThe "21/6594" part counts are fixed. The count was coming from a field that describes the part a thread happens to be carrying rather than the file's part size, and small readings turned filesize-divided-by-part into tens of thousands. It reads the configured part size instead, which is constant for a file. A total that wrong also meant the record never reached completion, so every later upload of the same file kept piling into it. The compact chunk view showed a film at 0/21 while the client was busy producing file lists, for the same reason: the directory describing the large file being split is not cleared when that file completes. It also drew one row per thread rather than per file, so a dozen threads on one film gave a dozen identical rows. File names with an apostrophe or an ampersand were coming out as raw XML, so "Mike Judge's" instead of an apostrophe. And the compact view toggle did nothing, because the strip it was meant to control was being drawn regardless of the setting. Small files were never appearing in Recently Completed. They are gone before a poll can catch a thread carrying one, and Backblaze pushes them in bundles rather than one at a time, so there is no per-file record of them in the log either. They are now taken from the client naming each one as it works through them, and listed without a thread, size or rate because none of those exist for a file handled that way. That tells you the client moved on from the file, not that a transfer was confirmed. @rogman Those Homarr layouts look cool, having some templates for various dashboards might be nice - something to go on the backlog. Glad the 8099 clash turned out to be it, and good call moving yours to 8098 and writing up the conflict for anyone else who had the earlier version installed. On the credit: it is staying where it is - the web dashboard is in the container because of your initial idea, I think its important to show credit where due! @gandalf15 Thanks for the restore test info, data is always useful! As always, everything above is beta only. Stable is unchanged at 10.2.1 and stays there until this has had a decent run. Quote the build number if you report anything, bottom right of both monitors and from bb-version.

Beta 110 is up with two fixes from this thread. @gandalf15 the clipped number was due to the size formatter stopping at gigabytes, so your 251 TB backup came out as "257524.9 GB" and ran off the end of the label. It goes up to petabytes now, so you should see "19.5 TB/251.5 TB" instead. Thanks for the screenshot. The ETA of 171 days is doing what it should, incidentally. It weights your last ten completed transfers rather than the live sample, so it will settle as more finish. A first pass over a quarter of a petabyte really does take that long. @rogman the dark theme is now your requested values, black rather than the dark grey I had. You were right that it was not really dark. On the monitor tab sticking on "Waiting for the monitor service", I have a suspicion and one thing I would like you to check. Your earlier setup ran your script on port 8099 through a cont-init.d entry in the container template. The built-in service uses that same port, and cont-init.d runs before services start, so if those entries are still in your template your script takes 8099 first, the built-in one cannot bind, and nginx has nothing to proxy to. That page is what you get when the service is not listening, and it will sit there forever because it keeps polling and never gets an answer. Run this and paste the output: docker exec Backblaze64 sh -c 'pgrep -af bb-monitor-web; ss -ltnp 2>/dev/null | grep 8099'If it shows something pointing at /config/bb-monitor-web.py, that is it, and the fix is removing the old Port and Path entries from your template. The container provides both now, so nothing is lost by dropping them. If that is not it, this runs the same code in the foreground so any startup error shows up instead of disappearing into the supervisor: docker exec Backblaze64 /usr/local/bin/bb-monitor-web --dump-rttI do not think SECURE_CONNECTION is involved. nginx talks to the service over plain HTTP inside the container regardless of what it serves to you, so TLS should not reach that far. @DayspringGaming looking good, there is clearly an issue with the data paths in Wine that need sorting out, hopefully it won't be too long before I get a shape of the code that needs changing and I can submit something that fixes it. My original patches were designed to be small incisions; the change that might be coming will be a much bigger piece of work. Still, while that plays out, everyone can get good network speeds from the beta.

Beta build 107 is up, and my implementation of the web dashboard is now part of the container. Click the WebUI button and you land on a tabbed page: the Wine desktop via VNC on one tab, the upload monitor on the other. Switching between them does not drop your VNC session, and the monitor tab does not start polling until you open it. It is on the port you already use, so there is no second port to map and nothing new exposed on your LAN. The terminal bb-monitor has caught up too. It now has the same overall progress and ETA, files remaining, uptime, the assigned upload server, the sparkline, and a settings dialog with the same thirteen themes. Press s to open it, Tab moves between Preferences and About. Both monitors read from one shared data layer now, so a feature will appear in both rather than one trailing the other. docker pull ghcr.io/iamfoz/backblaze-personal-wine:beta and you are on 107 (or if already running, you can just update in Unraid). It'll make its way to stable shortly, assuming we don't find any bugs. @rogman The HTML escaping you added was a great catch. File names went into the page unescaped, so a file whose name contained markup could inject it into the dashboard. Fixed in this beta, and credited in the changelog. On the ping work, where you said "Is this useful? Probably not." - it is. Per-connection throughput is the send buffer divided by the round-trip time, so RTT is the other half of the ceiling I have been going on about. With the 64 KB buffer, my 149 ms path works out at about 3.5 Mbit/s per connection, which is why thread count moves my numbers and line speed does not. The dashboard now shows RTT next to the transfer rate for that reason. I did rework how it is measured though; rather than opening a connection to time it, it reads the figure the kernel already keeps. Every established connection carries a smoothed round-trip time, and there is an interface that exposes it for sockets the process does not own. There is no traffic at all when nothing is uploading, and the number describes the actual upload connections rather than a separate probe alongside them. Your instinct to prefer the live upload connection over a hostname was the right one and I kept it. On the password and self-signed certificate: for a standalone script those were the correct call, and doing them properly with PBKDF2 and a constant-time comparison is good. The container does it differently only because it can; serving through the existing web interface means it inherits whatever WEB_AUTHENTICATION and SECURE_CONNECTION you already have, so there is one set of credentials rather than two. No change on the Wine front since Friday. The maintainers have my reply and I am waiting. The 64 KB lead is still a lead: I have now confirmed the arithmetic on my own hardware, but I still have not measured what a real Windows box reports, and until I do it stays a suspicion rather than a finding. The dashboard works on a phone, incidentally. I hadn't planned on working on it, but it turned out to be about an hour's work once the layout was already there. As ever: if something in this is confusing, wrong, or annoying, say so. Quote the build number from the corner of the monitor when you do.

A quick update: the upstream work has moved on rather than stalled. I have been going back and forth with the Wine maintainers about how the proper fix should be built, and I have just sent them a long reply conceding most of their objections to my original approach and asking them to pick the design. That is the right order. The workaround in the beta stays until whatever they choose lands, and then it goes. While digging into that I found something else, and I want to be careful about it because I have not finished checking. There is a Windows call that asks the system how much data a connection can usefully hold in flight, and Windows works that out per connection. Wine answers it with a fixed 64 KB every time, and the Backblaze client appears to size its send buffer from that answer. If that holds up it would put a ceiling on each connection that gets worse the further you are from the datacentre, and it would be a separate fix from the one under discussion. I need to measure what real Windows returns before I say anything stronger than that, so treat it as a lead rather than a result for now. @gandalf15 thank you, that is exactly the kind of before-and-after I was hoping for. 8 to 10 MB/s up to 12 to 18 MB/s is a bigger jump than I expected to see reported. On the thread count: that number is Backblaze's decision, not the container's. Your setting is an upper bound, and the client picks how many to actually run based on what it has queued and how the transfers are going. I see it swing between 3 and 17 here across a single evening on the same settings, so 6 is not necessarily a problem. What your numbers do tell me is that you are close to Backblaze in network terms. Six threads at 12 to 18 MB/s is roughly 16 to 24 Mbit/s per connection, and per-connection throughput is governed by the send buffer divided by round-trip time. To get that per thread you must have a fairly short path to their datacentre. Mine is 144 ms and I see about a fifth of that per thread. So your 10 gigabit line is not the limiting factor and neither is file size at 2.2 GB. If the thread count climbs on its own, expect the total to climb with it. @rogman the version I have now is your final one. Compact mode was a good call, I'm looking to build this out further into an API so all sorts of consumers will be able to fetch status and other information on backups. 20 MB/s on 200 MB files sounds right for what the client does with larger files. It splits them further and runs more threads, which is the lever that actually moves the total. More when I have the measurements. Thanks both.

OK, a few updates: two releases today. The stable one is a pair of fixes. The beta one roughly triples my upload speed. v10.2.1 fixes some little bugs: bb-monitor showed completion times in UTC while its own clock showed local time, so if you'd set TZ, the two disagreed by an hour in the same panel. Backblaze stamps its own logs in UTC no matter what the container's timezone is, and I'd been passing those straight through. Every image now also stamps a build number. The :beta tag is mutable, so I figured it would become useful to know which beta. bb-version prints the number, and the monitor shows it in the corner. If you report anything, quote it so I know what you're running. Beta build 103 carries a second Wine patch. On my connection, it takes aggregate upload from around 13 Mbit/s to 41. First, a correction. When I posted about the writability fix I said that once Wine reported writability correctly, the speed should all be there. I've now measured that properly, and I was wrong. Same host, same thread count, same files, ten-minute windows, alternating builds: my patch alone landed at 13.0, 13.9 and 13.9 Mbit/s across three runs, and with the second patch it went to 40.9. Wine tells an application a socket is writable exactly once and then latches that fact. While the latch is set, it stops asking the kernel to watch the socket for writability at all. If the application learns "not writable" from a poll rather than from a send that failed, it then waits for a notification that can never arrive, because nothing is watching any more. It sleeps until its own timeout expires, every single time around the loop. My first patch fixed the answer Wine gives to select(). It never touched the notification path, and it turns out Backblaze's client uses both: I counted 44 WSAEventSelect and 46 select calls in a single short capture. So I was fixing one of the two paths it depends on, and not the one that governs its throughput. This second patch is temporary. It clears that latch when a poll observes that the socket cannot accept a send. Real Windows does not do that. I tested it on Server 2022 and 2025 before writing any of this, and Windows only re-arms after a send actually fails. So the patch makes Wine diverge from Windows on purpose, which is why it is in the beta and not in the stable images, and why I am not submitting it upstream. CI now fails the build if a patched Wine ever turns up in a stable image, so that separation is enforced rather than something I have to remember. The other similar design from the upstream of this project is a reasonable piece of work which found a real defect that mine did not address. Where mine differs is the trigger. Theirs infers "cannot accept a send" from the absence of POLLOUT, and Linux withholds POLLOUT well before the buffer is actually full, so that fires in cases where a send would have gone through fine. Mine asks the kernel directly whether it would accept the data. It also drops a redundant reselect, because the function already reselects every socket before it returns. Narrower trigger, one fewer moving part. Both of us are papering over the same gap, though, and neither version belongs upstream in wine as written. The proper fix is a larger redesign that I'm working through over at WineHQ now. My measurements turned up some things that are not yet handled, so that discussion has somewhere useful to go. When it lands, the container patches can disappear and the stable images will operate as expected. One other thing turned up along the way, and it applies to whichever build you run. The Backblaze client asks for a 64 KB socket send buffer itself. At my round-trip time that caps a single connection near 4 Mbit/s, no matter how clean your line is, and no patch changes it, because it is the client's own setting, and a Windows machine would hit exactly the same ceiling. What that means in practice: Backblaze scales by connection count, not by connection speed. If you are on a fast line and a long path, your thread count is the lever that matters. Mine is on 30. Be careful with this setting as wine can get funny with lots of threads. Big thanks to @rogman for his initial feature request for bb-monitor to have a web version, and then the python build that followed, because the console has a habit of dying and he wanted to watch a session overnight. He also added overall backup progress with an ETA, and a theme picker, neither of which had occurred to me. I have taken his concept and I am building it into the container proper: it will be served through the existing web UI on port 5800 rather than needing a second port, which means it will inherit whatever authentication and HTTPS you have already configured instead of exposing your file names to anyone on your LAN. The plan will be a tabbed view so the VNC desktop and the monitor sit side by side, with a settings dialog for the themes and whatever else accumulates. That is going into the 10.3 line, and it will appear in a beta before it goes stable. Thank you rogman, this is a better project for it. I've also noted the 99% 'stickiness' - I suspect this is where the client is waiting for confirmation of a successful upload, while I've been poking around, I've been making lots of notes and believe that there are numerous improvements and features that can be teased out of the client. As always, if you have an idea, an itch, or something that annoys you about this container, please say so. It does not need to be code. "This number is confusing" and "why does it do that" are both useful, and rogman's started as a question about a terminal window closing. Update or switch to the beta channel if you want to try the speed patch. Stable is unchanged in behaviour beyond the two fixes above.

@gandalf15 No problem, if its the same account and an inherited state, it should be able to figure our what is already uploaded from hashing. @YourNekoWaifu That sounds like deduplication and or hash matching - if it still feels off, feel free to run bb-report and send me your logs, I can take a look. @rogman Looking good - I'm currently getting my older, slower speeds for some reason, but after a lot of bug hunting, I think there's some trouble between me and Backblaze, Telia's transatlantic link is showing congestion and loads of retransmission, so hopefully my own low numbers are not related to any of the code. There's movement on the Wine side of things, which may well be good in terms of overall future quality but probably not so great in terms of speedy deployment, I will update on this later.

Following up on my last post, where I'd said testing had turned up a problem. It turned into quite a bit more than one, and the short version: the beta image is now published and carries the reworked fix, it's faster here than anything I'd measured before, and the answer to combining the two fixes turned out to be more interesting than just yes or no. The problem my testing found was in my own patch. The way it measured send-buffer room could, right at the point where the buffer fills, report a socket as writable when a send would actually block. Backblaze never hits that path, which is why months of real backups never showed it. Wine's own socket test suite does hit it, and deadlocks on it, every time. Not shippable. Digging into why led somewhere better. Wine disagrees with itself about what "writable" means: the answer it gives applications comes from one place, and the decision about whether to even attempt a blocking send comes from another, using a stricter rule. The reworked fix uses the kernel's own definition (a socket is writable exactly when a send would be accepted) and applies it in both places. It heads back to the Wine reviewers next: I'll be updating my open submission once it has been through their test service across the Windows versions I can't spin up myself. On correctness, the numbers are already solid: Wine's full socket suite passes with fewer failures than an unpatched build, and the same tests pass on Windows Server 2022 and 2025 with zero failures. Once the updated submission is up I'll post the links here for anyone who wants the nuts and bolts of the actual changes. On combining it with the other fix, I tested that properly, including building the second half and running the test on real Windows. When a socket stops being writable without a send failing, Windows never re-signals it, so making Wine do so would be a divergence, and it's not going in. In principle, once writability is reported correctly the speed should all be there without it. In practice, that's exactly what I want your help confirming. Which brings me to the ask. On my connection, the new beta sustains around 79 Mbit/s, the best I've measured from any version. That's one line, one host, one backlog. It's also one workload shape: what's uploading here right now is 160 MB files, which Backblaze splits into 16 chunks of 10 MB, so per-chunk round-trip times may be what's capping my number rather than the socket path. Once the multi-GB files in my backlog start moving, I'd expect it to climb. The new :beta is published now: same tag, pull again and restart (on Unraid, the Repository field is ghcr.io/iamfoz/backblaze-personal-wine:beta). If you run it, I'd like numbers: rough speed before and after (bb-monitor in the container console shows it live), your thread count, your connection speed, and roughly what size files you're uploading. Especially if you'd compiled my earlier patched Wine yourself and swapped it in, or built an image with the other fix. If any setup is faster with the re-arm than with this, that's important data, and I want to see it rather than argue with it. The update also picks up a container fix: a stale lock left by a hard restart could wedge uploads in a way that looked healthy to every check. The container now detects that, and with the watchdog enabled, clears it itself. If you'd rather not run the beta, v10.2.0 is out today as a stable release and carries everything else from this round. The highlights: bb-monitor: a live upload dashboard in the container console showing speed, per-thread file progress, recently completed files, and memory gauges Backup-aware health: the Unraid Docker page now shows unhealthy when the backup is actually stalled, not just when a process dies Self-healing (opt-in): set ENABLE_WATCHDOG=true and the container clears stale locks and restarts deadlocked upload threads on its own bb-doctor: checks the install against every problem this project has hit, and --fix repairs the safe ones bb-report: builds a diagnostic bundle for bug reports with file names hashed, so nothing private leaves your box The beta only adds the Wine speed fix on top. The full list is in the changelog on the GitHub releases page.

Quick update while I'm mid-testing: the deeper test run has turned up a problem, so this is taking longer than planned. While putting both halves of the fix through Wine's full socket test suite, I found a case where my writability change can leave a blocking send waiting forever. Backblaze itself doesn't use sockets that way, which is why weeks of real backup testing never showed it; the client polls rather than blocks, and that path is fine. But it's a real defect, and I'm not shipping a combined beta until I understand it fully and have it fixed properly. Currently narrowing down exactly which conditions trigger it and whether it was present from the start or exposed by a newer Wine. I'll report back with the fix, and the Wine reviewers will get the full write-up either way. Short version: the speed fix works as well as ever for backups, I've caught something in testing that nobody else has hit yet, and I'd rather be slow and right than fast twice.