Extracting Video from Unifi Protect


LimeB

Recommended Posts

I currently have a Unifi UDM Pro but am swapping it out for a UDM SE. It is my understanding that I can't just move the HDD from the Pro to the SE and keep my current recordings for Protect. And Unifi UI within the UI is not very good on downloading recordings and the files are for Unfi are a proprietary so they can't just be accessed from another system. But one tool that was suggested to me is Unifi Protect Remux.

 

 

Aside from this tool, does anyone have any suggestions either using unRAID or another on extracting the videos from the UDM Pro?

 

And does anyone have any suggestions on how to get Unifi Protect Remux to run off of unRAID? While I work in IT, i don't work with Linux and am very basic, needing to look everything up and figure it out. I'm just not sure how to get Remux to work on unRAID which seems to likely be an easy solution on getting the videos off of the HDD.

 

Or maybe any other suggestions/guides that could help me? For legal reasons, I need to keep the videos for some period and would prefer not keeping the UDM Pro laying around.

 

  • Upvote 1
Link to comment

I have the UDM SE but while I've been tempted to pull the drive to look at the files via Unraid, I haven't done so yet. The problem I can see happening is if there is a database that keeps track of the recordings, if so, there's not much you can do without reverse engineering that DB if it's not in some standard DB. If it's a file based DB where it just uses the file names, you might be in luck. Plug in a drive in SE and have it start recording, then pull it to see how the directory & file structure compares to the one from Pro. If it's the same, just copying the files over might work.

 

In theory.

Link to comment

I think going forward I’ll try that with the SE. The situation I’m in I actually may need to often keep a duplicate drive offsite as there is a chance the SE would get damaged, or stolen. But for the moment I need to make sure I can unload the files from my Pro as I want to sell off the Pro and not have it laying around.

Link to comment

UniFi seems to use a non-standard partition format. It creates 3 partitions of varying small sizes and then 1 large partition to fill up the drive. This partition layout is similar to other Unix based OS's but neither unraid or my DiskSpeed 3 beta could identify them.

 

I tried mounting the large partition as different partition types but none of them were successful. It is marked as a RAID member but with only 1 drive in the system, that shouldn't have been an issue mounting known types.

Link to comment

Sounds like it may not be a simple way to just have it in the Unraid system and extract the video. I was really hoping to use pop the drive in, run say Unifi Protect Remux, and would be good to go.

 

I ended up shipping my Pro and for now put in a different drive into my SE. The chances are pretty slim that I will need the prior recordings. I have had to use some before for a legally issue but I think for the most part, as of about a week or so ago when a judge made an oral ruling, I don't think any prior recordings could be used in court if I needed. Or it might be after July 14th when the judge actually signs the orders.


Seems like such a big issue not to have a way to extract the video. You can't just hand the drive to someone, say the police. Or what if there was a fire and the hardware died but the drive was intact?

Link to comment

Figured out how to access the files.

 

SSH into your Unraid server with the UniFi Protect drive attached. I would recommend not using the browser based shell option.

 

Connect to the DiskSpeed docker - enter in

docker exec -it DiskSpeed bash

 

You should see something like

root@64648c9d4f98:/usr/local/tomcat

 

Install the mdadm and mc tools

apt install mdadm mc

 

From the UNRAID GUI, identify the sdx of your UniFi drive. This is easy if you have Unassigned Devices installed

image.png.2c00738a8cb42386b68520bd85614c07.png

 

If you do not have Unassigned Devices installed, you can enter "fdisk -l" and look for a drive with 4 partitions under type "Linux filesystem" and take note of the last/largest partition.

Disk /dev/sdg: 5.5 TiB, 6001175126016 bytes, 11721045168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 54DBDD5B-2319-43A8-9507-45FDBF81FDE7

Device        Start         End     Sectors  Size Type
/dev/sdg1      2048     1050623     1048576  512M Linux filesystem
/dev/sdg2   2097152     6291455     4194304    2G Linux filesystem
/dev/sdg3   6291456     8388607     2097152    1G Linux filesystem
/dev/sdg5  16777216 11721045134 11704267919  5.5T Linux filesystem

 

Based on the "5" from Unassigned Devices and the "sdg" drive ID, (or shown from fdisk), the video partition location is /dev/sdg5

 

Verify the partition's RAID status

mdadm --examine /dev/sdg5

/dev/sdg5:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 069a3dee:c842797d:ada2dca8:4f1f206a
           Name : Dream-Machine-SE:3
  Creation Time : Tue Jun  7 13:22:55 2022
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 11704005775 (5580.90 GiB 5992.45 GB)
     Array Size : 5852002304 (5580.90 GiB 5992.45 GB)
  Used Dev Size : 11704004608 (5580.90 GiB 5992.45 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=1167 sectors
          State : clean
    Device UUID : 96a0229f:8b61ec1f:166c0bd3:3f28ac1c

Internal Bitmap : 8 sectors from superblock
    Update Time : Fri Jul  8 09:27:22 2022
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 85d860a0 - correct
         Events : 1474479


   Device Role : Active device 0
   Array State : A. ('A' == active, '.' == missing, 'R' == replacing)

 

Take note of the "Data Offset" value.

 

Create a directory to mount the partition

mkdir /mnt/unifi

 

Mount the partition

mount -o ro,loop,offset=$((<data offset value>*512)) /dev/sdg5 /mnt/unifi/

 

Now you can use MC to browse the directory tree /mnt/unifi/.srv and optionally copy files to one of your UNRAID shares/drives on path /mnt/UNRAID

 

Directories of note:

.srv/unifi-protect/video

.srv/unifi-protect/cv/analytics-v1

 

Based on the backup directory dump files, UniFi does have some kind of database it uses.

 

I don't know at this time if simply copying the video files to another UniFi protect HD will bring it online on the new drive with its existing files.

 

 

  • Like 1
Link to comment

I am impressed. I don't really know much using Linux so this for sure not something I would have figured out. It would be interesting if you could move the video files over to an existing drive would work. I did read there is a DB and would assume that helps with having multiple cameras.

Link to comment
  • 2 weeks later...

I had trouble duplicating these mounting steps today but I passed through the /dev/sdx ID as a drive to a Ubuntu VM and then used the "Disks" app to mount the "RAID-1 Array" under /dev/md

 

These steps assume you will not be mounting the drive back into your UniFi Protect system without formatting it.

  • Launch the "Disks" app and locate the UniFi OS video partition. For my 8TB drive, it was displayed as "8 TB RAID 1 Array" under /dev/md/3 (your last digit may differ).
  • Select the drive "8 TB RAID 1 Array" and click on the Gear icon under the "Filesystem" bar and select "Edit mount options"
  • Disable "User Session Defaults" and change the mount point to where you want it. For me, it's "/home/john/UniFi" and ensure "Mount at system startup" and "Show in user interface" are checked and click "OK"
  • Click on the "Play" icon to mount (or reboot)
  • Open a Terminal window. Change to "~" directory if not already there. "cd ~"
  • Enter "ls -l" to see what user/group you're logged in as. For me, it was "john:john"
  • CD to the UniFi drive mount location. Since I mounted it in my user directory, I just need to type "cd UniFi"
  • You will need to update the owners of the UniFi Protect files. Enter in "sudo chown -R john:john .srv" - replace your user/group you're logged in as.

Now if you use the File Manager to navigate to the UniFi directory with hidden directories enabled, you can navigate the directory tree and copy the files to another system.

 

 

Link to comment

In short, yes, but you'll still have to jump through a few hoops. I spent some time trying to create a UniFi Protect VM to do the conversion but it didn't pan out. I ended up using my Dream Machine SE as a conversion host after finding out Ubiquity added a MP4 export tool to their OS.

 

After using the Ubuntu VM to copy the UBV files off to an unraid share, I used Putty to SSH to the DM and then from there I SFTP'ed the files to the DM, converted them, and SFTP'ed them back to unraid.

 

More detailed instructions

Enable SSH on UniFi OS - > System > SSH > Enable
Telnet into the UniFi Protect server.
Local video is hosted under "/srv/unifi-protect/video" by year, month, day. July 20, 2022 will be located in "/srv/unifi-protect/video/2022/07/20"

 

If you don't have much space available, you'll need to edit the UniFi Protect settings to only archive x number of days (take how much you have and subtract a day or two) so it can free up space.

 

Create directory for hosting conversions: /srv/video
CD into /srv/video
SFTP to remote server to download ubv files
The video files start with the Mac ID of the camera.

D021F991680A_0_rotating_1658291482261.ubv
Mac ID      |Q| Type   |    Epoch

Q is video quality. "0" is full, each number higher is lower in quality.

 

Transfer all the files for the day & camera in question from your remote host to your UniFi system. "mget D021F9924ABB_0_rotating_*"
Quit SFTP.

 

Create a single MP4 file. Use a different -d option for each file you're converting. "vid0", "vid1" for example. Use multiple -s options to specify the files in sequence. Using Putty makes it easy as you can double-click on the file name and then right-click to paste it.
/usr/share/unifi-protect/app/node_modules/.bin/ubnt_ubvexport -d ./vid0 -s f1.ubv -s f2.ubv -s f3.ubv

 

It'll also create one zero-byte MP4, you can delete it.

 

SFTP back to the remote server and upload the mp4 files.
The files will be owned by root on the remote server. You will need to "chmod 666 *.mp4" the files to be able to view them over the network.

 

Don't forget to delete your working files in /srv/video.

Edited by jbartlett
Link to comment
  • 4 months later...
11 hours ago, DanielPT said:

Is it possible to backup the video to Unraid (When it is in the UDM pro) and then backup it up to like Onedrive or something? :)

You can remote in as instructed in my previous post and SFTP the files out. The problem is the files are in a proprietary format so they won't be of much use to you without a tool to convert them. I read of someone doing so but it was with an older version of Unifi Protect so it might not be viable.

  • Like 1
Link to comment
  • 1 year later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.