Since the backup only sends the specific files (config, docket, etc) and not the OS files (bz* files).
When downloading the backup the server has to get your files + the relevant OS files, pack all that in a zip file for DL.
From personal tests, it can take a few seconds indeed.