8 hours ago, YourNekoWaifu said:

Updated to the latest beta version and now its doing what the other available client does where it eats all my available RAM and then stops uploading until I manually start it again. A few versions ago it just sat around 8GB consistently

I could be wrong, but I think as you get to larger and larger file sizes being backed up, your memory usage grows. That might be contributing to your issue and why you haven't ran into it before. It might be beneficial to setup and use swap space on an SSD or HDD to make up for the limited RAM availability. It also makes it worse when the scan is running that list of all your 2.3M files gets loaded into memory. If you are using encryption, that also increases memory usage. Lowering your threads would reduce some memory usage. Maybe try automatic, or set it to 8-10?

The swap file would be what you have mapped for "Temporary Data Drive" on the settings tab in the GUI. You could map a folder on an SSD to your container as a new drive letter and try setting it to that.

I might be way off on this, but it wouldn't hurt to try.

1 hour ago, rogman said:

I could be wrong, but I think as you get to larger and larger file sizes being backed up, your memory usage grows. That might be contributing to your issue and why you haven't ran into it before. It might be beneficial to setup and use swap space on an SSD or HDD to make up for the limited RAM availability. It also makes it worse when the scan is running that list of all your 2.3M files gets loaded into memory. If you are using encryption, that also increases memory usage. Lowering your threads would reduce some memory usage. Maybe try automatic, or set it to 8-10?

The swap file would be what you have mapped for "Temporary Data Drive" on the settings tab in the GUI. You could map a folder on an SSD to your container as a new drive letter and try setting it to that.

I might be way off on this, but it wouldn't hurt to try.

Lowering threads did fix it. I just thought backblaze would have some sort of memory management.

Encryption is turned on and i did set a temporary drive but it doesn't seem to want to use the SWAP from it

But its seems like when its doing multiple small files its the problem, now when its chunking its working fine.

I restarted it and it backed up a bunch of smaller files I had, so I think that's where I went wrong

Backblaze does manage the memory but I don't think it spills over to the swap space until your ram is full or near full. I don't believe there is a way to force it to use it. Maybe if you limited your container to only use X GB RAM it could force it over to swap before your system runs out of memory. I know @foz has swap in use in some of his screenshots so he would be in the know on this issue.

Hey guys,

So I disabled tessypowder and installed iamfoz.

I set a different appdata folder and mounted the exact same mount for /drive_d/ (which would be /mnt/user/). I've tried both with and without trailing slashes.

I inherited backup and let it scan. I came back after a couple hours and there was simply no window. Just the tabs at the top. So I had to restart.

Checking the web dashboard looks like the inherit worked and the container stopped showing trial and shows my 1 year plan.

However, when I start the container up, I get a box that says "Not Backing Up: No files are selected." and the main window says "Initial Backup Paused." When I check the settings, the D drive is definitely selected.

If I try inherit backup again, it only lets me select the trial backup so it seems inherit worked correctly.

I just can't get it to pick up my local files I guess? Going into the console and going to cd /drive_d/ shows the correct files. And there is indeed a .bzvol at /mnt/user/.

The files in the .bzvol folder have today's date, but the bzscratch folder has 2023 (when I first started using Backblaze). bazscratch has a folder called bzcurrentlargefile and bazcurrentlargefile has nothing in it.

Did I just... magically lose my backup or something? Also, wouldn't it still be trying to scan the files in my /mnt/user/ folder?

Thank you!

Edited September 2Sep 2 by DevanteWeary

8 hours ago, DevanteWeary said:

Hey guys,

So I disabled tessypowder and installed iamfoz.

I set a different appdata folder and mounted the exact same mount for /drive_d/ (which would be /mnt/user/). I've tried both with and without trailing slashes.

I inherited backup and let it scan. I came back after a couple hours and there was simply no window. Just the tabs at the top. So I had to restart.

Checking the web dashboard looks like the inherit worked and the container stopped showing trial and shows my 1 year plan.

However, when I start the container up, I get a box that says "Not Backing Up: No files are selected." and the main window says "Initial Backup Paused." When I check the settings, the D drive is definitely selected.

If I try inherit backup again, it only lets me select the trial backup so it seems inherit worked correctly.

I just can't get it to pick up my local files I guess? Going into the console and going to cd /drive_d/ shows the correct files. And there is indeed a .bzvol at /mnt/user/.

The files in the .bzvol folder have today's date, but the bzscratch folder has 2023 (when I first started using Backblaze). bazscratch has a folder called bzcurrentlargefile and bazcurrentlargefile has nothing in it.

Did I just... magically lose my backup or something? Also, wouldn't it still be trying to scan the files in my /mnt/user/ folder?

Thank you!

Suddenly I have, without an update - the same message on the screen.

After clicking continue it took work up again. indeed strange.

50 minutes ago, gandalf15 said:

Suddenly I have, without an update - the same message on the screen.

After clicking continue it took work up again. indeed strange.

Well I checked a little bit ago and noticed it seemed to have my correct size/file amount on the main window and said it was preparing files.

It also said it skipped a few files due to permissions. So I figured it started working.

Well now I checked again and I have this: Computer's backup has been Safety Frozen.

Hi @YourNekoWaifu, welcome - this is normal when the client restarts and is most noticeable on large backup sets. The client will scan the set, which involves putting a file list into memory. My own set is 90TB, and my memory usage will hit 10GB easily during this phase. As @rogman said, the best way to manage this is to either have a large amount of RAM on hand or configure Swap. I'm currently running off of an N100 stick computer with 12GB of RAM and Proxmox, so the Backblaze process can easily overwhelm the host, which I've resolved with a Swap partition. After the file processing and the backup begins, memory use will fall back. Thread numbers will also contribute to this pressure. Backblaze will prioritise files by size, so each run will always start with new and changed small files, then go on to larger files. After about 100MB, they are chunked by 10MB pieces. Each piece will use a thread to upload, and this is when users with decent upload bandwidth will find the most effective use of that bandwidth comes into play.

Hi @DevanteWeary, welcome. Good news first: nothing is deleted. A Safety Freeze is Backblaze protecting a backup when something looks wrong with the drive underneath it. Your data is still on their side, and the freeze stops any sudden catastrophic changes.

Backblaze have a page on clearing one yourself: https://www.backblaze.com/computer-backup/docs/resolve-a-safety-freeze

Read it before you touch anything. The last step is re-licensing with Inherit Backup State, and that's where it's possible to pick the wrong computer record and lose the link to the backup you're trying to save. You've been through that screen once already, so you know it. Just be careful which entry you choose.

The uninstall and reinstall step is easier here than on Windows. Delete /config/wine/drive_c/Program Files/Backblaze/ and restart the container. It downloads the client again and reinstalls it on the next start, which is the same thing it does when a version update comes through. Your appdata and the client's own data are in a different place and aren't touched by that, so the licence and the inherit history survive.

What I can't tell you is whether that's enough. Backblaze's uninstall step may be meant to clear the client's local state as well, and the container's reinstall keeps it. If the freeze is still there afterwards, say so before you go further, because the next thing anyone would reach for is wiping that state, and that's the part with real risk attached.

Before any of that, I'd fix the thing that probably caused it.

The first two symptoms were a red herring. "No files are selected" and "Initial Backup Paused" is what the window says while the client is still building its file list, which on a large share takes hours. You saw that resolve itself: correct size, correct count, then "preparing files". Nothing was wrong there.

The permissions skips are the part that matters. New container, new appdata, so User ID and Group ID are back at their defaults of 99 and 100. If your shares are owned by something else, the container can't read them, and a large slice of an existing backup suddenly going unreadable is the kind of thing I'd expect to look like an altering event from Backblaze's side. Hypothetical rather than diagnostic, but it fits both things you've reported.

On the latest beta, run this:

docker exec backblaze-personal-wine bb-doctor

It reads the client's own skipped-file list and works out, for each file, whether it's actually gone or just unreadable from here. Where it can't read something, it prints the directory with its owner and mode, next to the UID the container runs as.

If it comes back with a list of directories, that's the answer. Fix ownership on the host with the command it prints. Do it on the host rather than inside the container, and don't blanket-chown the whole share. Get that clean first, then work through Backblaze's page.

On .bzvol: today's date on those files is normal, the client updates them. The 2023 date on bzscratch and the empty bzcurrentlargefile are normal too. That's the staging area for files over 100 MB, and it's empty because nothing large is in flight.

@gandalf15 I think yours is a different thing. "Initial Backup Paused" with a Continue button is the client's own pause rather than a freeze, and clicking Continue is the right answer. If it starts happening regularly, I can take a look, but it is normal for the client to cycle through these states by itself from time to time.

---

And as noted, I've been working on some updates and bugfixes on the beta channel, here's what's changed:

The memory gauge now names the program using the memory.

A container partway through its first backup announced that the backup was complete, and latched that to disk for a week. The completion check shared a helper with the staleness warning, and that helper answers "yes" when there are no totals to judge against. Right for a warning, where a missing figure should not raise a false alarm. Wrong for declaring something finished, and during a file-list scan the totals are absent. It now needs positive evidence.

ETA sanity took a pass. A backup starts on its small files, and a handful of those gives a per-byte rate you can extrapolate into geological time. One run read "4259966d", about 11,600 years, and the trend feature then compared today's nonsense against yesterday's. Estimates beyond a century are withheld now, and the trend ignores anything resting on fewer than three completed transfers. A grim estimate still shows if it is real: 100 TB on a 1 Mbit/s uplink is about 25 years, and that is accurate. With nothing usable, it reads "not yet".

The failure counter no longer says "failed" as this was misleading. The number it showed is attempts a storage vault turned away for being busy or full. The client retries against another vault and the file goes up. It reads "retried" now, without the alarm colour, and the API carries the breakdown by reason. The count that means data is not backed up is the skipped-file count, which is reported separately and loudly.

Pause reporting got two corrections, and one of them was me being wrong. The monitor said "Paused" the moment the request landed, while the client was still finishing the transfers already in flight. Backblaze's own window looked like it had not noticed, because it was finishing up. A pause reads "Pausing" until it takes. The monitors also said "Preparing <file>" next to a state of Paused, which reads as stalled; the client parks on the file it was about to take, so while a pause is set that is the next file rather than the current one. This may be what you were experiencing with the API @rogman - let me know if this is still a problem.

Backblaze keeps a list of files it has given up on, with a reason against each one. It does not queue them, it does not retry them, and nothing in the desktop window tells you they exist. A backup can look finished while a directory's worth of files was never sent. The beta surfaces that list in three places now: a warning in both monitors, a Skipped Files tab in the web interface with a filter and a breakdown by reason, and bb-doctor, which goes further and works out why.

bb-doctor reads a sample of the list and sorts it into three groups. Files that no longer exist, which need nothing done. Files that are readable again and will clear at the next scan. And files this container still cannot read, where it names the directory, prints its owner and mode next to the UID the container runs as, and gives you the command to run on the host. It repairs nothing, even with --fix. These are your files on a mounted share, often thousands of them, and other software on your host may depend on who owns them.

Two bugs found and fixed in that check: the list is written by a Windows program, so every line ends in CRLF, and the shell left the carriage return on the end of each path. The same check counted the report's own header as a skipped file, so a clean list reported two. The monitor was never affected, because Python strips line endings where the shell does not.

Smaller things. Buttons and chips in the dark themes had near-black text hardcoded on them, around 1.5:1 against dark grey, so they were effectively invisible. bb-monitor run without a terminal gracefully exits and indicates the issue instead of dying in a curses traceback, which is what docker exec without -it gets you. There is a seven-day upload chart, a breakdown of what the backup is made of by category, and a "backing up since" date.

@foz The dashboard plugin with API Pause/Resume is fixed and working in current version

@foz I have a somewhat selfish request. Is it possible (with minimal work) to add a BB-Doctor tab on the Web GUI that can run the doctor and output results on the HTML page directly? If that was viewable from the Web it would make everything reachable all in one spot, like the skipped files is now (Thank you for this!!)

@rogman, not selfish at all. A very useful request, and one that broke me out of my (slightly nostalgic) CLI reverie.

Beta 156 has a Tools tab. It runs bb-doctor, bb-health and bb-version from the page and shows what they print and allows you to interact with options and outputs as you'd expect. Notes are on the page, but let me know if there are questions.

A bug. docker exec enters the container as root, and root passes every permission test. bb-doctor decides whether a skipped file is readable with a permission test, and whether /config is writable with another, so from the console both always said yes. If your share had the wrong owner, which is the most common fault this container has, running bb-doctor the way the README told you to said your files could be read. The Tools tab runs as the container user, which is the user whose permissions matter, so it gives the true answer. And bb-doctor itself now re-runs as that user when it is started as root, so the console gives the true answer as well. If you ran bb-doctor from the console before 150 and it said your files were fine, it may have been wrong.

Status tab. Skipped Files is now Status, and it leads with everything the client says needs attention: warnings, a pause with the reason, a first backup running or finished, and then the skipped list. The pause part is the one I care about. The client writes a reason code with every pause and we were showing a flag. Now it says "Paused from here" when you or the API paused it, and "Paused by the client" with the cause when the client chose to, for example when Backblaze's cluster authority is not answering, which is usually their maintenance and resumes on its own. One is a button, the other is a wait, and the page says which. A Copy status summary button gives five lines of plain text for a post like this one: build, state, progress, health, today's uploads. Nothing in it names a file.

Notifications. The API tab has a Notifications section. Point it at an ntfy topic or a webhook that takes JSON, tick the events, and the container tells your phone when the backup needs you: a safety freeze, files skipped past a threshold, no completed backup within the client's own limit, a stall, a pause the client chose, first backup complete, a milestone. Once when it starts, once when it clears, never on every poll, and nothing that names a file is ever sent. A Test button checks the endpoint.

Quiet hours. Also in the API tab. Windows by day of the week with a start and end; the container asks the client to pause at the start and starts the backup at the end. The client's own pause lasts about two hours, so inside a window the container renews it. If you start the backup yourself during a window it stays running until the window ends.

bb-doctor. Two new checks. It samples each mapped drive as the container user and names anything it cannot read, with the owner and the chown line, before the client has to give up on files. And it compares the volume ID the client wrote under .bzvol with the volumes the client knows about; a drive it no longer recognises is the "No files are selected" state after an inherit, which the last thread hit and nothing named. bb-health reports FROZEN when Backblaze has frozen the backup, so the Docker tab goes unhealthy for that state.

API. A new switch allows you to turn it off without revoking keys; off, everything returns 404 and the keys are kept. Revoked and expired keys have a Delete button. New endpoints: /summary, /metrics in Prometheus format, /events as a server-sent stream on state changes, /tools behind new diagnose and diagnose:repair permissions, and /openapi.json. The OpenAPI document is also in the repository.

Smaller things. All thirteen themes now apply to every tab; before, the Status, Tools and API pages used the default. Text uses the width of the window instead of wrapping at about 375 pixels. Keys: 1 to 5 switch tabs, s opens settings, p pauses or resumes. The browser tab title carries a mark for uploading, paused, or a warning. The desktop pane shows a Reconnect button when noVNC drops its connection, which after a container restart used to be a blank pane. A Today line under the seven-day chart, and a progress-over-time line in About.

@foz What you've turned this container/project into is nothing short of amazing. I feel bad for the people who don't know about what you've created and how useful and polished it is! My backup finally finished fairly recently, and daily backups are now a breeze. I would still be uploading if not for your hard work and pure intellect that you have demonstrated, finding/fixing/improving the core upload speed issue, and now the plethora of nice features this project has with the API, web dashboard, diagnostics, etc. Thank you again for sharing your smarts with all of us users!

I have been trying to get notifications to work via Pushbullet, which uses a bearer token and a JSON object, but I'm not getting the test to work. It looks like it should work with the settings you provided, but I can't get it to send.

Here is what the dev console shows

Pushbullet documentation provides an example template which reads as

-- HTML

POST /v2/pushes HTTP/1.1

Host: api.pushbullet.com

Authorization: Bearer <your_access_token_here>

Content-Type: application/json

{

"type": "note",

"title": "App Alert",

"body": "This is a notification from your custom application!"

}

-- Python

import requests

import json

def send_push_notification(title, body):

token = "YOUR_ACCESS_TOKEN"

url = "https://api.pushbullet.com/v2/pushes"

headers = {

"Authorization": f"Bearer {token}",

"Content-Type": "application/json"

}

payload = {

"type": "note",

"title": title,

"body": body

}

response = requests.post(url, data=json.dumps(payload), headers=headers)

if response.status_code == 200:

print("Notification sent successfully!")

else:

print(f"Failed to send notification: {response.text}")

# Usage

send_push_notification("Server Alert", "CPU usage is above 90%!")

Edited September 6Sep 6 by rogman
notification via pushbullet

@foz You are amazing, it's working now!

Thank you, and it is my pleasure. It's gratifying to see people getting value from my work.

I think you've already noticed the update, but I added some specific templates for the various popular notification tools as well as the option of building your own output - my fault for half-assing it a bit in the first place. A few other bug fixes and cleanups as well.

I was going to look into bbnotify.py, and when I pulled it up, it said just modified. Saw the Pushbullet info inside and ran to check for an update

Great feature adding the manual entry field. At this point, it's not what this container does, but rather what it does not do?! I can't think of anything else.

10/10

Hey, I was the one having the problem earlier with migrating TO iamfoz and getting a safety lock.

Well let's just say I've tried various things to get it working and it wouldn't work.

Then I tried one more time to wipe out the container completely and re-deploy it and let it run over night.

I didn't do any inheriting or anything. I forgot to. Then I woke up to it just working and seems like it auto-inherited somehow.

So... I think it's all working fine. The only issue and difference is that I forgot to make it the nightly/beta version (maybe that has something to do with it?).

So I'm not getting those tabs on top nor does the widget work anymore.

Is it safe to simply switch over to the nightly now?

@DevanteWeary If you are synced and backing up, it should be safe to switch to beta. You can run bb-doctor from the Tools tab when you're up and see if any warnings/errors show up.

@foz Got a question for you and a tip for others. I removed a drive map from my Docker template because I no longer needed that folder backed up. I ran bb-doctor, and it said I had an unmapped drive that showed as a warning. I couldn't see this drive because I removed it from the template. However, I looked into the AppData folder structure and found where drives are mapped and simply deleted that old one from the list, and the warning cleared. It was found here

That's the tip for everyone. My question is: I never mapped the Z: drive, and I don't remember when I first noticed it. It is not a folder I'm backing up, but when you look in it, you see the container file structure. Is it safe to delete this folder from the \dosdevices\ folder, or is it supposed to be there?

The folder shows root as owner

You're almost at the point the wine web GUI will be obsolete and everything can be done on the Web pages. I truly thought there was nothing else you could think of and every update it's fun to see what's changed. The push update comes in that an update was installed and it's like Christmas opening to see 🤣

Edit: reading your changes answered my question about the Z: mapping, thank you!

I'm curious if others have had that issue of removing or changing mapped drives and they are removed from the container settings but the symlinks still exist in the app data folder, can that be fixed by running bb-doctor --fix and it safely remove those non used links?

Edited yesterday at 07:40 PM1 day by rogman