Samba, logs, XBMC and its library

[NAS]

This may be impossible but...

 

it occurred to me that if samba could log file upload files then it would be a simple matter of watching this log and creating a daemon/script that could tell XBMC to update it library when new data is uploaded.

 

My library probably like alot of users here has grown to the point where XBMC is quite slow to update. However the XBMC api is easy to call to update just one specific media file. Doing  this 20 or even 100 times would still be speedy.

 

Thoughts?

[WeeboTech]

This may be impossible but...

 

it occurred to me that if samba could log file upload files then it would be a simple matter of watching this log and creating a daemon/script that could tell XBMC to update it library when new data is uploaded.

 

My library probably like alot of users here has grown to the point where XBMC is quite slow to update. However the XBMC api is easy to call to update just one specific media file. Doing  this 20 or even 100 times would still be speedy.

 

Thoughts?

 

Do you know the API call to do this?

[BRiT]

There's no need to recreate this functionality as it already exists. XBMC shouldn't need to POLL for changes if it uses a INotify API,  where it can monitor changes to files and directories.

 

where its functionality permits reindexing of changed files without scanning the filesystem for changes every few minutes, which would be very inefficient. By being told that a file has changed directly by the kernel, rather than actively looking, Beagle and such utilities can achieve change-to-reindexing times of only about a second.

 

It can also be used to automatically update directory views, reload configuration files, log changes, backup, synchronize, and upload.

[WeeboTech]

There's no need to recreate this functionality as it already exists. XBMC shouldn't need to POLL for changes if it uses a INotify API,  where it can monitor changes to files and directories.

 

Does inotify work across a network mount?

[BRiT]

There's no need to recreate this functionality as it already exists. XBMC shouldn't need to POLL for changes if it uses a INotify API,  where it can monitor changes to files and directories.

 

Does inotify work across a network mount?

 

Oh. I haven't had my morning caffeine yet and didn't think of that. 

 

Does XBMC's API support network invocations? Perhaps running a program using INotify on the unRAID server and making calls out to the XBMC Clients might work?

[NAS]

SOunds like a good direction to go. The trick will be that most people XBMC wont be on 24*7 so some sort of alive polling and storing until then would need to be done.

 

WeeboTech I do have the API call but not at hand. Its fairly obvious and can be done with wget/curl. The trick is you need to then do another call to until it returns OK complete. This was 6 months ago though things might have changed.

 

Edit:

 

http://xbmc.local:8080/xbmcCmds/xbmcHttp?command=ExecBuiltIn(UpdateLibrary(video,smb://TOWER/TV/W/Weeds))

 

Specific path

 

http://xbmc.local:8080/xbmcCmds/xbmcHttp?command=ExecBuiltIn&parameter=XBMC.updatelibrary(video)

 

Whole dbase

 

 

[BRiT]

For scripting access (instead of C API usage), there is 'inotifywait'.

 

inotifywait --daemon --outfile XBMC_Events.log --recursive /mnt/disk#/Media/

 

Then parse the XBMC_Events.log for the events generated and call out to XBMC for new files/directories created.

 

The --recursive options comes with the following notes:

 

Warning: If you use this option while watching the root directory of  a large  tree,  it  may  take quite a while until all inotify watches are established, and events will not be received in this time.  Also, since  one  inotify watch will be established per subdirectory, it is possible that the maximum amount of inotify watches per user  will  be  reached.  The  default  maximum  is  8192;  it  can  be  increased  by writing to /proc/sys/fs/inotify/max_user_watches

[WeeboTech]

For scripting access (instead of C API usage), there is 'inotifywait'.

 

inotifywait --daemon --outfile XBMC_Events.log --recursive /mnt/disk#/Media/

 

Then parse the XBMC_Events.log for the events generated and call out to XBMC for new files/directories created.

 

The --recursive options comes with the following notes:

 

Warning: If you use this option while watching the root directory of  a large  tree,  it  may  take quite a while until all inotify watches are established, and events will not be received in this time.  Also, since  one  inotify watch will be established per subdirectory, it is possible that the maximum amount of inotify watches per user  will  be  reached.  The  default  maximum  is  8192;  it  can  be  increased  by writing to /proc/sys/fs/inotify/max_user_watches

 

I would not be keen on doing it this way. That's allot of extra kernel memory.

It would probably cause issues on my system with 20 disks and over 400000 files.

Watching the samba log and parsing it is feasible and efficient.

 

An inotify watch on samba log with a parser to send messages to XBMC would work pretty well.

[WeeboTech]

Just for the record, doing an updatedb from the slocate pacakge on my system crashes the kernel with an OOM condition.

 

I had to adjust the vm_cache_pressure=200 to alleviate the out of memory conditions.

[BRiT]

So something like this in the samba/smb config file to generate the events?

 

[global]
log level = 0 vfs:2
log file = /var/log/samba.log.%m
max log size = 50
debug timestamp = yes

vfs objects = full_audit
full_audit:prefix = %u|%I
full_audit:success = mkdir rmdir write unlink rename

[kizer]

Are you updating.

Tv shows

movies

mp3's

 

all at the same time? I have mine setup to rescan on startup, which I'm sure a lot used to do when the librarys are/where smaller. That cache script seems to really speed up the scan since my drives don't have to spin up, but you seem to be onto something.

[NAS]

tv + movie but invariably one-many of each gets added every day. once your XBMC dbase gets large it starts to get slow. not really slow but its very inefficient to scan 400k files to find the 3 that are new... hence this idea

 

until xbmc supports musicbrainz it wont be used for my catalog

 

worth considering that this log could also be potentially  useful in a business environment later for audit purposes but thats a nice to have only and OT.

 

 

 

 

[NAS]

Does anyone have a dev machine this now to see if they can capture some samba logs?

[BRiT]

Has anything further been developed on this realm?

[NAS]

Not that I have heard of but the idea still has legs as far as I am concerned