sparklyballs Posted May 18, 2015 Share Posted May 18, 2015 DEPRECATED, USE THIS VERSION FROM NOW ON http://lime-technology.com/forum/index.php?topic=44606.0 The MusicBrainz Server is an open source software application that is written in Perl with web pages constructed using Catalyst and Template Toolkit. The MusicBrainz Server provides the primary interface to the MusicBrainz Database by providing a website for users to browse and edit the data on, and a web service to power client applications like MusicBrainz Picard. The MusicBrainz Server is licensed under the GPLv2 This docker requires a code from musicbrainz so that the database can be updated from their servers. BEFORE ADDING THIS DOCKER Go the metabrainz site and click non-commercial. Then create an account like so. Go through the pages, filling in as appropriate.. Then generate your access token like so... Select, and copy this access token, you'll need it for the next bit. NOW ADD THE DOCKER You'll need to click the advanced tab and paste your access token into the BRAINZCODE section under Environmental Variables like so. Map your two folders to appropriate locations as normal and then click create. The docker will then generate a blank postgres data structure, download the latest data dump from musicbrainz and then import the data. THE DOWNLOAD MAY BE UP TO 5GB AND THE PROCESS OF DOWNLOADING AND SUBSEQUENT IMPORT CAN TAKE A LONG TIME. The docker may appear unresponsive and the logs can look stuck on BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED This is normal, just go off and start another civilisation on a small uninhabited island, or drink a metric ton of coffee and be patient. Every hour or so check the log an eventually it should show something like this IMPORT IS COMPLETE, MOVING TO NEXT PHASE *** Running /etc/my_init.d/004-import-databases--and-or-run-everything.sh... *** Running /etc/my_init.d/00_regen_ssh_host_keys.sh... *** Running /etc/rc.local... *** Booting runit daemon... *** Runit started as PID 609 May 23 12:34:49 1cd29f5762de syslog-ng[614]: syslog-ng starting up; version='3.5.3' May 23 12:59:01 1cd29f5762de /USR/SBIN/CRON[695]: (root) CMD (/bin/bash /root/update-script.sh) May 23 13:00:01 1cd29f5762de /USR/SBIN/CRON[718]: (root) CMD (/bin/bash /root/update-script.sh) May 23 13:17:01 1cd29f5762de /USR/SBIN/CRON[873]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) May 23 13:59:01 1cd29f5762de /USR/SBIN/CRON[1266]: (root) CMD (/bin/bash /root/update-script.sh) May 23 14:00:01 1cd29f5762de /USR/SBIN/CRON[1283]: (root) CMD (/bin/bash /root/update-script.sh) May 23 14:17:01 1cd29f5762de /USR/SBIN/CRON[1444]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) May 23 14:59:01 1cd29f5762de /USR/SBIN/CRON[1622]: (root) CMD (/bin/bash /root/update-script.sh) May 23 15:00:01 1cd29f5762de /USR/SBIN/CRON[1634]: (root) CMD (/bin/bash /root/update-script.sh) May 23 15:17:01 1cd29f5762de /USR/SBIN/CRON[1694]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Voilà MusicBrainz by Sparklyballs is complete.... Now in my main repository... thanks to eric on team LT for some coding help and CHBMB for this writeup. Link to comment
heffe2001 Posted May 18, 2015 Share Posted May 18, 2015 I'm eager to see this up and running. Wonder how hard it'd be to just move my database from my VM to the docker? Link to comment
sparklyballs Posted May 18, 2015 Author Share Posted May 18, 2015 I'm eager to see this up and running. Wonder how hard it'd be to just move my database from my VM to the docker? i couldn't say... Link to comment
sparklyballs Posted May 18, 2015 Author Share Posted May 18, 2015 I'm eager to see this up and running. Wonder how hard it'd be to just move my database from my VM to the docker? it runs from the command line in docker at the moment, i imported the latest database dump from their ftp and browsed it in the included web interface. i couldn't get it to work with nginx, dunno whether that's cos i'm shite with nginx or nginx is just shite in general. got to script it and decide on how to go about handling the initial data import, i'm loathe to have it on some kind of variable, because if someone doesn't reset it, it's a whopping big download, in excess of 5gb. Link to comment
CHBMB Posted May 18, 2015 Share Posted May 18, 2015 I'm eager to see this up and running. Wonder how hard it'd be to just move my database from my VM to the docker? it runs from the command line in docker at the moment, i imported the latest database dump from their ftp and browsed it in the included web interface. i couldn't get it to work with nginx, dunno whether that's cos i'm shite with nginx or nginx is just shite in general. got to script it and decide on how to go about handling the initial data import, i'm loathe to have it on some kind of variable, because if someone doesn't reset it, it's a whopping big download, in excess of 5gb. Sparkly, do you ever sleep? Link to comment
sparklyballs Posted May 20, 2015 Author Share Posted May 20, 2015 Could soon be at the point to say it's coming soon, lol. Link to comment
CHBMB Posted May 20, 2015 Share Posted May 20, 2015 Could soon be at the point to say it's coming soon, lol. What? Sleep? or the container? lol Link to comment
alexricher Posted May 20, 2015 Share Posted May 20, 2015 Very interesting! I'll definitely give this a try once it's released! :-D Link to comment
sparklyballs Posted May 21, 2015 Author Share Posted May 21, 2015 pushed into my beta repo for you to try. Link to comment
heffe2001 Posted May 21, 2015 Share Posted May 21, 2015 Giving me an error on first startup. I verified that I had all the data downloaded from the musicbrainz server, but it looks like the import routines are not finding some of the files that are there (verified twice, downloaded twice). Log of the offending errors attached.. musicbrainz.txt Link to comment
sparklyballs Posted May 21, 2015 Author Share Posted May 21, 2015 Giving me an error on first startup. I verified that I had all the data downloaded from the musicbrainz server, but it looks like the import routines are not finding some of the files that are there (verified twice, downloaded twice). Log of the offending errors attached.. it is supposed to miss those files in the initial download. they are not neccessary, one is just documentation that gets updated with first cron pull (which cron is not working for some reason, i've got to sort) , one is huge (3gb +) and the other i'm not sure but isn't needed. the error that is really odd and worrying is Can't locate DBDefs.pm in @INC (you may need to install the DBDefs module) i'll have to look at that. may have to do a completed breakdown and restart from scratch of my own system to remedy it. Link to comment
sparklyballs Posted May 21, 2015 Author Share Posted May 21, 2015 I need you to confirm, but i'm betting you either mapped all 3 of the folders to the one place or at the very least, the import and the config folder to the same place. in the import script the first line wipes out anything that may be there (to counteract possible remnants of broken downloads), if they're mapped together instead of separately, it will throw up errors. Link to comment
sacretagent Posted May 21, 2015 Share Posted May 21, 2015 ok running the docker on my ubuntu server docker installation only question is what we need to put here for value ? <Environment> <Variable> <Name>BRAINZCODE</Name> <Value></Value> </Variable> </Environment> i started the docker like this docker run -d --name="musicbrainz" -v /mnt/cache/musicbrainz-docker/data:/data -v /mnt/cache/musicbrainz-docker/import:/import -v /mnt/cache/musicbrainz-docker/config:/config -p 5000:5000 --restart=always sparklyballs/musicbrainz and he is downloading the database dump now .. but i was wondering what value that environment tag needs yeah i know it is crazy but i have not enough power on my unraid machines to run all dockers.... so i had a powerfull server which i use to run our dockers now unraids have only crashplan running now and hamachi.... for the rest i mounted their shares on the ubuntu server and before you ask the server is a HP with a SA controller that unraid doesn't recognizes.... Link to comment
sparklyballs Posted May 21, 2015 Author Share Posted May 21, 2015 ok running the docker on my ubuntu server docker installation only question is what we need to put here for value ? <Environment> <Variable> <Name>BRAINZCODE</Name> <Value></Value> </Variable> </Environment> i started the docker like this docker run -d --name="musicbrainz" -v /mnt/cache/musicbrainz-docker/data:/data -v /mnt/cache/musicbrainz-docker/import:/import -v /mnt/cache/musicbrainz-docker/config:/config -p 5000:5000 --restart=always sparklyballs/musicbrainz and he is downloading the database dump now .. but i was wondering what value that environment tag needs yeah i know it is crazy but i have not enough power on my unraid machines to run all dockers.... so i had a powerfull server which i use to run our dockers now unraids have only crashplan running now and hamachi.... for the rest i mounted their shares on the ubuntu server and before you ask the server is a HP with a SA controller that unraid doesn't recognizes.... you need to go here https://metabrainz.org/login?next=%2Fprofile and register, and follow links through to get a server replication code (shortened in my template to BRAINZCODE) then enter it like any other variable. Link to comment
sparklyballs Posted May 21, 2015 Author Share Posted May 21, 2015 Giving me an error on first startup. I verified that I had all the data downloaded from the musicbrainz server, but it looks like the import routines are not finding some of the files that are there (verified twice, downloaded twice). Log of the offending errors attached.. MY BAD, i didn't do a full rip down and build from scratch like i would normally. i'd spent so long on this container that i was so eager to get something out there. eric on team LT put in a pull request that i merged that should hopefully fix this. Link to comment
heffe2001 Posted May 21, 2015 Share Posted May 21, 2015 Attempting it again now. Hopefully it'll finish it's run this time. It's installed, and I can bring up the Webui, but not sure if it's working correctly with Headphones or not yet. I've gotten a few musicbrainz errors logged (can't find artist in this category or something to that effect, I'll keep an eye and see if I can get an exact message. Bad thing about using Mediabrainz with headphones, not much in the way of logging of errors. Does the system automatically generate the indices, or do we need to force that? Same goes for replication, is it automatic, or do we need to start it? **EDIT** Found a mention of the musicbrainz error in Headphones logs, not much info here, lol: 21-May-2015 15:41:38 - INFO :: Thread-36 : [skid Row] There was either an error pulling data from MusicBrainz or there might not be any releases for this category Link to comment
sacretagent Posted May 22, 2015 Share Posted May 22, 2015 works fine for me only one little thing i had issues with was that i needed to put localhost in headphones instead of the ip address.... headphones and musicbrainz running on the same docker server Link to comment
sparklyballs Posted May 22, 2015 Author Share Posted May 22, 2015 works fine for me only one little thing i had issues with was that i needed to put localhost in headphones instead of the ip address.... headphones and musicbrainz running on the same docker server you're an edge case (in terms of this forum) and i'd like to remind unraid users that because our docker implementation uses a bridge by default that putting localhost in headphones to get access to musicbrainz will not work, you must put the ip address. Link to comment
sparklyballs Posted May 22, 2015 Author Share Posted May 22, 2015 Does the system automatically generate the indices In the initial data import it generates some indexes, do you have specific indexes in mind ? Same goes for replication, is it automatic, or do we need to start it? the first few iterations of this , the cron entry wasn't working as i didn't close out the last entry in one of the start files as a background operation, but the latest version does. look in the webui at the very bottom left corner, there is a message that tells you the last replication packet. as for headphones issues, can you give settings in headphones you're using to access this server. Link to comment
heffe2001 Posted May 22, 2015 Share Posted May 22, 2015 As far as indexing, I can remember having to do it with the VM they supplied, which will a full database and index was larger than the supplied VM's size (had to expand it to 60gb just to fit everything). If it generates the index files while doing the import that would most likely be sufficient. Mainly I just need the index that would work with the headphones searches, nothing else really. Same goes for replication, if it's on, that's all I need to know (I hated having to remember to log in and turn on replication on the VM version). For headphones settings: Musicbrainz Mirror: Custom Host: 192.168.0.5 Port: 5000 Requires Authentication (unticked) Sleep Interval: 1 Everything default aside from the host. Sometimes you can click a artist name, and it comes up with the album info for them, sometimes not. I had this problem frequently with the real musicbrainz lookups set, very infrequently with the headphones mirror, and very infrequently with the VM (if the VM was doing something intensive, it'd happen more often). It could be that I'm just doing too much on my unraid box, causing timeouts. I might look at the sleep interval and see if upping that at bit would help. I wish the 'musicbrainz lookup' buttons in Headphones actually used the mirror site you have set up, instead of going to the mail live site... Link to comment
sparklyballs Posted May 23, 2015 Author Share Posted May 23, 2015 pushed to my main repo now Link to comment
heffe2001 Posted May 23, 2015 Share Posted May 23, 2015 Just a FYI, it seems to be working much better with my headphones installation. Wonder if it was still working on setup when I was trying yesterday.. Should I re-install, or leave it as-is? I'm guessing it won't matter if I leave it as it is now since they don't update their stuff very often anyway.. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.