iCloud Picture Downloader Docker Template


Womabre

Recommended Posts

Trying to get the new keyring system to work and I'm receiving this issues after the keyring file is created

2020-10-22 20:20:14 INFO     Check for new files using password stored in keyring...
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 209, in authenticate
    req = self.session.post(
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 578, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 100, in request
    self._raise_error(code, reason)
  File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 122, in _raise_error
    raise api_error
pyicloud_ipd.exceptions.PyiCloudAPIResponseError: Unknown reason

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/icloudpd", line 11, in <module>
    load_entry_point('icloudpd==1.6.0', 'console_scripts', 'icloudpd')()
  File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/icloudpd-1.6.0-py3.8.egg/icloudpd/base.py", line 255, in main
  File "/usr/lib/python3.8/site-packages/icloudpd-1.6.0-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate
  File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in __init__
    self.authenticate()
  File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 216, in authenticate
    raise PyiCloudFailedLoginException(msg, error)
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseError('Unknown reason'))
2020-10-22 20:20:17 INFO     Two factor authentication cookie generated. Sync should now be successful.

I'm still getting the 

2020-10-22 20:22:32 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes

in the logs.

 

Link to comment
  • 2 weeks later...

Hi, is there any way for someone to write up instructions on setting this up? I downloaded the docker and put the template information. 

Apple ID:  myapple@email

This is the Apple ID for the account you want to download files for

AppData Config Path: /mnt/user/appdata/icloudpd

Synchronisation Interval (seconds): 86400

Default is once every 24h.

Data: /mnt/user/icloud/

Container Path: /home/user/iCloud

 

Docker is up and running and whats next? Im a little lost.

 

Thanks

Link to comment

Hi,

 

I am also running into the same issues as others...

 

I was able to issue the command:

docker exec -it icloudpd /usr/local/bin/sync-icloud.sh

and got the successful cookie generated message.

 

In the web log page I still get the error:

Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes

and

WARNING Using Apple ID password from variable. This password will be visible in the process list of the host. Please add your password to the system keyring instead

How do you setup the run container interactively and how do I store the password in the keychain?

 

Thanks,

 

H.

Link to comment
  • 2 weeks later...

I set up my icloudpd this evening. It was a little bit trial and error but for now I guess it is working. It's currently downloading my more than 20.000 photos and videos. Although I used the parameter to convert HEIC to jpg, it downloads all the HEIC files - guess they will be converted to jpg afterwards and 'not on the fly'.

 

At the very first start, it opened the icloudpd console by using the unraid docker web if and started the sync-icloud.sh manually. As I did not realize that there was no .mounted file in my mounted folder, I tried several things including executing the icloud and icloudpd file in /usr/bin/ which focusses on generating the keychain and 2FA cookie. This was all by accident, but then the cookie has been generated ;-) Anyway, I executed the sync-icloud.sh again and again because I still have not realized that I don't have a .mounted file. Took another 30min to realize and touch this file ;-)

 

Regardless of the parameters, it seems that the tool analyses all pictures / videos uploaded to iCloud. In my case this took about 30min and then the download started.

 

I also added a few parameters. Looks good so far, let's see whether the HEIC to jpg conversion will be executed after the download has been finished (in approx. 10 hours).

 

Attached my docker image settings. Of course there are also a username / password, I've deleted these entries just for the screenshots.

 

 

 

 

 

 

 

 

 

2020-11-17 22_35_20-Heimdall_UpdateContainer.png

2020-11-17 22_35_50-Heimdall_UpdateContainer.png

Edited by Coke84
  • Like 2
Link to comment
  • 1 month later...
On 2/3/2020 at 3:53 PM, Womabre said:

Hi all!

In search of a way to get my iCloud pictures on my array and keep them in sync with iCloud (pull only).

I ended up writing a docker template for the boredazfcuk/icloudpd docker container.

This is my first template so bear with me. If someone has suggestions or comments about it please let me know!

 

The template is located here.

https://github.com/Womabre/unraid-docker-templates

 

Known issues

 

- Your iCloud password cannot contain special characters (I know of * and & to be problematic).

 

Thank you! Seems like it's working pretty good once you run the authentication script and create the .mounted file! Thanks!

Edited by bigbangus
Link to comment
  • 3 months later...
On 5/15/2020 at 2:22 AM, Womabre said:

It also requires a named volume mapped to /config. This is where is stores the authentication cookie. Without it, it will lose the cookie information each time the container is recreated. It will download the photos to the "/home/${user}/iCloud" photos directory. You need to create a bind mount into the container at this point

 

Total noob to unraid and dockers so appreciate the kindness from the group!  I am lost on creating the bind mount.  Can someone ELI5 on how to execute this please?

Link to comment
5 minutes ago, Twinkie0101 said:

Total noob to unraid and dockers so appreciate the kindness from the group!  I am lost on creating the bind mount.  Can someone ELI5 on how to execute this please?

 

The "AppData Config Path" and the "Data" path need to be mapped to a folder outside the container.

For example, the data path is where all the photos are downloaded. You want to save these somewhere on you array.

So this should be something like "/mnt/user/SHARE/iCloudPics"

The container is like a small OS running inside it's own bubble. So the folder "/config" in the container redirects to the "/mnt/user/SHARE/iCloudPics" folder on your array.

This is called a bind mount.

Link to comment
  • 1 month later...

Hi,

 

First of all thank you for providing this container, it fills exactly my needs when it comes to preserving my photos.

 

Since the last 2 updates (as far as I noticed) the container goes into "unhealthy" modus after starting it. It seems to be working OK, but I have never had any container in this modus (running around 20 containers 24/7). I have two instances of the iCloud Picture Downloader running (one for my account one for my girlfriend's).

 

The only info I can find about this unhealthy state is that it takes too long for some components or processes to respond.

Any idea what might be causing this, or how to fix it?

 

thank you!

 

Tom

Link to comment
1 hour ago, TOMMY-V said:

Hi,

 

First of all thank you for providing this container, it fills exactly my needs when it comes to preserving my photos.

 

Since the last 2 updates (as far as I noticed) the container goes into "unhealthy" modus after starting it. It seems to be working OK, but I have never had any container in this modus (running around 20 containers 24/7). I have two instances of the iCloud Picture Downloader running (one for my account one for my girlfriend's).

 

The only info I can find about this unhealthy state is that it takes too long for some components or processes to respond.

Any idea what might be causing this, or how to fix it?

 

thank you!

 

Tom

Hi Tom,

 

I'm not the provider of this container. I only created the template file for use with unRAID.

Try checking the logs if you can find something that might cause the container to become unhealthy.

Link to comment

Thanks for this. I just installed it this week and is working great.

 

Quick question, does it download the whole library each time or does it only download the latest after the first download? I did see a recent settings. I am wondering if I need to use that if it downloads the whole library each time.


Thanks!

Link to comment

Never mind. I am a dork. I looked at the logs and it looked like it was downloading the same files again this morning but it was yesterday and I saw it connected this morning and didn't have anything to download. I thought yesterdays downloads were todays date but they weren't....... DOH!

Link to comment
  • 3 months later...

Hey,

 

since yesterday, I've experienced the issue that icloudpd does not let me convert heic to jpegs anymore.

 

It properly downloads all files but when it comes to converting the files to jpeg, it brings me the issue that heif-convert does not exist. Sth changed in the sync-icloud.sh script in the last update? Or anybody with an idea how to fix?

 

I've already tried to completely set up the docker from the scratch, but the issue remained. First attempt downloads and converts the files, after that the issue comes up again and conversion fails.

 

Thank you so much!

 

Update: was able to fix it. I did not update the new shell scripts in my dockerconfig folder which have been updated recently. As a consequnce, the healthcheck as well as the HEIC conversion was faulty. Update 2: see above. worked for the first try, second attempt and the error comes up again.

 

update 3: the issue has been fixed and an updated Docker template has been published. Thanks for your effort!

 

2021-09-12 18:51:58 INFO Converting /home/user/iCloud/2021/08/IMG_6511.HEIC to /home/user/iCloud/2021/08/IMG_6511.JPG
2021-09-12 18:51:58 INFO Timestamp of HEIC file: Sat Aug 7 09:44:35 2021
2021-09-12 18:51:58 INFO Setting timestamp of /home/user/iCloud/2021/08/IMG_6511.JPG to Sat Aug 7 09:44:35 2021
2021-09-12 18:51:58 INFO Converting /home/user/iCloud/2021/08/IMG_6510.HEIC to /home/user/iCloud/2021/08/IMG_6510.JPG
/usr/local/bin/sync-icloud.sh: line 504: heif-convert: not found
2021-09-12 18:51:58 INFO Timestamp of HEIC file: Sat Aug 7 09:44:32 2021
2021-09-12 18:51:58 INFO Setting timestamp of /home/user/iCloud/2021/08/IMG_6510.JPG to Sat Aug 7 09:44:32 2021
2021-09-12 18:51:58 INFO Converting /home/user/iCloud/2021/08/IMG_6509.HEIC to /home/user/iCloud/2021/08/IMG_6509.JPG
/usr/local/bin/sync-icloud.sh: line 504: heif-convert: not found
2021-09-12 18:51:58 INFO Timestamp of HEIC file: Sat Aug 7 09:43:52 2021
2021-09-12 18:51:58 INFO Setting timestamp of /home/user/iCloud/2021/08/IMG_6509.JPG to Sat Aug 7 09:43:52 2021
2021-09-12 18:51:58 INFO Converting /home/user/iCloud/2021/08/IMG_6507.HEIC to /home/user/iCloud/2021/08/IMG_6507.JPG
/usr/local/bin/sync-icloud.sh: line 504: heif-convert: not found

 

Edited by Coke84
Link to comment
  • 1 month later...
  • 2 weeks later...
  • 5 months later...
  • 4 weeks later...
21 minutes ago, rfrye said:

Just got this setup and its running. I've been looking for something just like this for awhile.

I'm getting  error messages, but the files are downloading and working. I have this running on an unraid machine, but storing files to a network SMB share (Synology NAS)

I'm not sure why every file throws an error message.

https://imgur.com/ezGicq0

Are you using the Unassigned Devices plugin to mount the NAS to your unraid system?

Link to comment
  • 4 weeks later...

I am so lost.

I do not understand how to create or where to place this .mounted file everyone is talking about. I am scouring all of the comments and the official information but I am missing something completely.

 

I have a share named iCloud on the array. Does the .mounted file go in the share? 

 

Again, I feel like I am missing something. Sorry if I have overlooked the information I am looking for.

Link to comment
1 hour ago, Okrah Wimpey said:

I am so lost.

I do not understand how to create or where to place this .mounted file everyone is talking about. I am scouring all of the comments and the official information but I am missing something completely.

 

I have a share named iCloud on the array. Does the .mounted file go in the share? 

 

Again, I feel like I am missing something. Sorry if I have overlooked the information I am looking for.

I got it working now. Not sure how I did it but the data is downloading to the share correctly.

 

Link to comment
  • 1 month later...

Hi guys, 

I am a newbie to Unraid and I get a warning:

 

2022-07-09 14:20:44 WARNING Failsafe file /home/user/iCloud/.mounted file does not exist. I am waiting for the failsafe file to be created....

 

I do not manage to create a mounted.file. 
thanks for your help. 

Link to comment
2 hours ago, meruem said:

Hi guys, 

I am a newbie to Unraid and I get a warning:

 

2022-07-09 14:20:44 WARNING Failsafe file /home/user/iCloud/.mounted file does not exist. I am waiting for the failsafe file to be created....

 

I do not manage to create a mounted.file. 
thanks for your help. 

 

Create the file, you can do this via the terminal for example using the command "nano /pathtofile/.mounted", where the 'pathtofile' is the path you entered when creating the container. When the editor opens, you can directly edit (or enter a random word or so) and press Ctrl + X to exit. Typ Y to save and it should work directly after.

Link to comment
  • 2 weeks later...

Hi! First off: the docker works. 

I have two questions: My original library size is about 200GB. When it downloads everything (99% photos), it ends up being 19GB. All Fotos are saved as png files, and seem to be compressed (300KB per file). 

I am thinking that maybe one of the parameters is not set correctly in the config, but I do not have as many variable options as every one else has in the screenshots (such as downloading Live Photos, converting heic etc).

So my question: how can I access the additional variables? Any ideas as to why the photos end up being so small?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.