sparklyballs Posted May 23, 2015 Author Share Posted May 23, 2015 Just a FYI, it seems to be working much better with my headphones installation. Wonder if it was still working on setup when I was trying yesterday.. Should I re-install, or leave it as-is? I'm guessing it won't matter if I leave it as it is now since they don't update their stuff very often anyway.. i would leave it as is, there is some mention of an update coming in june, case of wait and see though. Link to comment
Critical Posted May 24, 2015 Share Posted May 24, 2015 Two things: 1. I dont see where you get the access token on the musicbrainz site. I already had an account and also created a new one. In my profile there is no option to generate access token. There is authorized applications section that I can create an oAuth client ID. 2. I dont see it in your main repo. Thanks.. Link to comment
HarryRosen Posted May 30, 2015 Share Posted May 30, 2015 Can you delete the files in the import folder once it is running? Link to comment
sparklyballs Posted May 30, 2015 Author Share Posted May 30, 2015 Can you delete the files in the import folder once it is running? yes Link to comment
HarryRosen Posted May 30, 2015 Share Posted May 30, 2015 great thanks, need to save space on my cache drive it's only 256g ssd Link to comment
sparklyballs Posted June 1, 2015 Author Share Posted June 1, 2015 added hot standby on to postgresql.conf allows transactions while postgres rebuilds from last shutdown (docker doesn't shut postgres gracefully), so container will start properly. Link to comment
miko3ed Posted June 6, 2015 Share Posted June 6, 2015 having some issues with this, do I need to have a separate postgres instance running on the server already for this docker to work? Link to comment
sparklyballs Posted June 6, 2015 Author Share Posted June 6, 2015 having some issues with this, do I need to have a separate postgres instance running on the server already for this docker to work? nope, has one built in. Link to comment
miko3ed Posted June 6, 2015 Share Posted June 6, 2015 getting the following after the docker installs: *** Running /etc/my_init.d/001-fix-the-time.sh... Current default time zone: 'America/New_York' Local time is now: Sat Jun 6 16:41:12 EDT 2015. Universal Time is now: Sat Jun 6 20:41:12 UTC 2015. *** Running /etc/my_init.d/002-configure-DBDefs.sh... DBDefs is in your config folder, may need editing *** Running /etc/my_init.d/003-postgres-initialise.sh... The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "en_US.UTF-8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /data/main ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting dynamic shared memory implementation ... posix creating configuration files ... ok LOG: could not link file "pg_xlog/xlogtemp.42" to "pg_xlog/000000010000000000000001" (initialization of log file): Function not implemented FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory child process exited with exit code 1 initdb: removing contents of data directory "/data/main" creating template1 database in /data/main/base/1 ... 2015-06-06 16:41:18,033 CRIT Supervisor running as root (no user in config file) 2015-06-06 16:41:18,034 INFO supervisord started with pid 44 2015-06-06 16:41:19,036 INFO spawned: 'redis-server' with pid 48 2015-06-06 16:41:19,037 INFO spawned: 'postgres' with pid 49 2015-06-06 16:41:19,041 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:20,042 INFO success: redis-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-06-06 16:41:20,043 INFO spawned: 'postgres' with pid 52 2015-06-06 16:41:20,047 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:22,050 INFO spawned: 'postgres' with pid 53 2015-06-06 16:41:22,054 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:25,058 INFO spawned: 'postgres' with pid 54 2015-06-06 16:41:25,062 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:26,063 INFO gave up: postgres entered FATAL state, too many start retries too quickly BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED Link to comment
sparklyballs Posted June 6, 2015 Author Share Posted June 6, 2015 don't know, no issues here. just did a ripdown and restart and it's proceeding normally. Link to comment
BRiT Posted June 6, 2015 Share Posted June 6, 2015 getting the following after the docker installs: *** Running /etc/my_init.d/001-fix-the-time.sh... Current default time zone: 'America/New_York' Local time is now: Sat Jun 6 16:41:12 EDT 2015. Universal Time is now: Sat Jun 6 20:41:12 UTC 2015. *** Running /etc/my_init.d/002-configure-DBDefs.sh... DBDefs is in your config folder, may need editing *** Running /etc/my_init.d/003-postgres-initialise.sh... The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "en_US.UTF-8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /data/main ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting dynamic shared memory implementation ... posix creating configuration files ... ok LOG: could not link file "pg_xlog/xlogtemp.42" to "pg_xlog/000000010000000000000001" (initialization of log file): Function not implemented FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory child process exited with exit code 1 initdb: removing contents of data directory "/data/main" creating template1 database in /data/main/base/1 ... 2015-06-06 16:41:18,033 CRIT Supervisor running as root (no user in config file) 2015-06-06 16:41:18,034 INFO supervisord started with pid 44 2015-06-06 16:41:19,036 INFO spawned: 'redis-server' with pid 48 2015-06-06 16:41:19,037 INFO spawned: 'postgres' with pid 49 2015-06-06 16:41:19,041 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:20,042 INFO success: redis-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-06-06 16:41:20,043 INFO spawned: 'postgres' with pid 52 2015-06-06 16:41:20,047 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:22,050 INFO spawned: 'postgres' with pid 53 2015-06-06 16:41:22,054 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:25,058 INFO spawned: 'postgres' with pid 54 2015-06-06 16:41:25,062 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:26,063 INFO gave up: postgres entered FATAL state, too many start retries too quickly BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED I haven't used this container but... What is your volume mappings? ie: what do you have /data and /config mapped to? Im going to guess you have it set to /mnt/user or something using user shares, which do not support symlinks. You will need to use /mnt/cache or /mnt/disk# directly. Link to comment
miko3ed Posted June 7, 2015 Share Posted June 7, 2015 I haven't used this container but... What is your volume mappings? ie: what do you have /data and /config mapped to? Im going to guess you have it set to /mnt/user or something using user shares, which do not support symlinks. You will need to use /mnt/cache or /mnt/disk# directly. That was it, Thanks! Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 I've been trying to get this to run and have been having numerous problems. I'm trying a fresh install and it seems that the postgresql server won't start. I have config and data pointing to separate directories on my cache drive, so I've solved that problem at least. Just can't seem to get it running "out of the box". Anyone else having similar problems? Here's the docker log: ?*** Running /etc/my_init.d/001-fix-the-time.sh... Current default time zone: 'America/New_York' Local time is now: Wed Jun 17 21:16:12 EDT 2015. Universal Time is now: Thu Jun 18 01:16:12 UTC 2015. *** Running /etc/my_init.d/002-configure-DBDefs.sh... DBDefs is in your config folder, may need editing *** Running /etc/my_init.d/003-postgres-initialise.sh... The files belonging to this database system will be owned by user "postgres". This user must also own the server process. initdb: directory "/data/main" exists but is not empty If you want to create a new database system, either remove or empty the directory "/data/main" or run initdb with an argument other than "/data/main". The database cluster will be initialized with locale "en_US.UTF-8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. 2015-06-17 21:16:19,134 CRIT Supervisor running as root (no user in config file) 2015-06-17 21:16:19,148 INFO supervisord started with pid 38 2015-06-17 21:16:20,151 INFO spawned: 'redis-server' with pid 42 2015-06-17 21:16:20,154 INFO spawned: 'postgres' with pid 43 2015-06-17 21:16:20,166 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:21,167 INFO success: redis-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-06-17 21:16:21,169 INFO spawned: 'postgres' with pid 46 2015-06-17 21:16:21,185 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:23,188 INFO spawned: 'postgres' with pid 47 2015-06-17 21:16:23,202 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:26,207 INFO spawned: 'postgres' with pid 48 2015-06-17 21:16:26,223 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:27,224 INFO gave up: postgres entered FATAL state, too many start retries too quickly BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED Link to comment
sparklyballs Posted June 18, 2015 Author Share Posted June 18, 2015 I've been trying to get this to run and have been having numerous problems. I'm trying a fresh install and it seems that the postgresql server won't start. I have config and data pointing to separate directories on my cache drive, so I've solved that problem at least. Just can't seem to get it running "out of the box". Anyone else having similar problems? Here's the docker log: ?*** Running /etc/my_init.d/001-fix-the-time.sh... Current default time zone: 'America/New_York' Local time is now: Wed Jun 17 21:16:12 EDT 2015. Universal Time is now: Thu Jun 18 01:16:12 UTC 2015. *** Running /etc/my_init.d/002-configure-DBDefs.sh... DBDefs is in your config folder, may need editing *** Running /etc/my_init.d/003-postgres-initialise.sh... The files belonging to this database system will be owned by user "postgres". This user must also own the server process. initdb: directory "/data/main" exists but is not empty If you want to create a new database system, either remove or empty the directory "/data/main" or run initdb with an argument other than "/data/main". The database cluster will be initialized with locale "en_US.UTF-8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. 2015-06-17 21:16:19,134 CRIT Supervisor running as root (no user in config file) 2015-06-17 21:16:19,148 INFO supervisord started with pid 38 2015-06-17 21:16:20,151 INFO spawned: 'redis-server' with pid 42 2015-06-17 21:16:20,154 INFO spawned: 'postgres' with pid 43 2015-06-17 21:16:20,166 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:21,167 INFO success: redis-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-06-17 21:16:21,169 INFO spawned: 'postgres' with pid 46 2015-06-17 21:16:21,185 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:23,188 INFO spawned: 'postgres' with pid 47 2015-06-17 21:16:23,202 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:26,207 INFO spawned: 'postgres' with pid 48 2015-06-17 21:16:26,223 INFO exited: postgres (exit status 2; not expected) 2015-06-17 21:16:27,224 INFO gave up: postgres entered FATAL state, too many start retries too quickly BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED if you stop the container before the import completes, you must delete the contents of the data folder before restarting it again. Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 Ok, I'll give that a shot. Here's the rest of the log though, after it finished downloading the import /data/import / mbdump-cdstubs.tar.bz2: OK mbdump-cover-art-archive.tar.bz2: OK mbdump-derived.tar.bz2: OK mbdump-documentation.tar.bz2: FAILED open or read mbdump-edit.tar.bz2: FAILED open or read md5sum: mbdump-documentation.tar.bz2: No such file or directory md5sum: mbdump-edit.tar.bz2: No such file or directory mbdump-editor.tar.bz2: OK mbdump-stats.tar.bz2: OK mbdump-wikidocs.tar.bz2: OK mbdump.tar.bz2: OK md5sum: WARNING: 2 listed files could not be read Failed query: 'SELECT version();' () 08006 DBI connect('dbname=template1','postgres',...) failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? at /opt/musicbrainz/admin/../lib/MusicBrainz/Server/Connector.pm line 42. at /opt/musicbrainz/admin/../lib/Sql.pm line 463. Sql::catch {...} ('08006 DBI connect(\'dbname=template1\',\'postgres\',...) fail...') called at /usr/local/share/perl/5.18.2/Try/Tiny.pm line 104 Try::Tiny::try('CODE(0x212fd38)', 'Try::Tiny::Catch=REF(0x2142090)', 'Try::Tiny::Finally=REF(0x906188)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 469 Sql::_select_list('Sql=HASH(0x1e48998)', 'SELECT version();', 'ARRAY(0x205a960)', 'arrayref', 'CODE(0x2069688)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 488 Sql::select_list_of_lists('Sql=HASH(0x1e48998)', 'SELECT version();') called at /opt/musicbrainz/admin/../lib/Sql.pm line 419 Sql::select_single_column_array('Sql=HASH(0x1e48998)', 'SELECT version();') called at /opt/musicbrainz/admin/../lib/Sql.pm line 434 Sql::select_single_value('Sql=HASH(0x1e48998)', 'SELECT version();') called at ./admin/InitDb.pl line 74 main::RequireMinimumPostgreSQLVersion() called at ./admin/InitDb.pl line 529 08006 DBI connect('dbname=template1','postgres',...) failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? at /opt/musicbrainz/admin/../lib/MusicBrainz/Server/Connector.pm line 42. at /opt/musicbrainz/admin/../lib/Sql.pm line 465. Sql::catch {...} ('08006 DBI connect(\'dbname=template1\',\'postgres\',...) fail...') called at /usr/local/share/perl/5.18.2/Try/Tiny.pm line 104 Try::Tiny::try('CODE(0x212fd38)', 'Try::Tiny::Catch=REF(0x2142090)', 'Try::Tiny::Finally=REF(0x906188)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 469 Sql::_select_list('Sql=HASH(0x1e48998)', 'SELECT version();', 'ARRAY(0x205a960)', 'arrayref', 'CODE(0x2069688)') called at /opt/musicbrainz/admin/../lib/Sql.pm line 488 Sql::select_list_of_lists('Sql=HASH(0x1e48998)', 'SELECT version();') called at /opt/musicbrainz/admin/../lib/Sql.pm line 419 Sql::select_single_column_array('Sql=HASH(0x1e48998)', 'SELECT version();') called at /opt/musicbrainz/admin/../lib/Sql.pm line 434 Sql::select_single_value('Sql=HASH(0x1e48998)', 'SELECT version();') called at ./admin/InitDb.pl line 74 main::RequireMinimumPostgreSQLVersion() called at ./admin/InitDb.pl line 529 IMPORT IS COMPLETE, MOVING TO NEXT PHASE *** Running /etc/my_init.d/004-import-databases--and-or-run-everything.sh... *** Running /etc/my_init.d/00_regen_ssh_host_keys.sh... *** Running /etc/rc.local... *** Booting runit daemon... *** Runit started as PID 112 Jun 17 22:25:51 0ae6ac8db702 syslog-ng[117]: syslog-ng starting up; version='3.5.3' And if I go to port 5000, I get this error: caught exception in MusicBrainz::Server::Controller::Root->begin "08006 DBI connect('dbname=musicbrainz_db','musicbrainz',...) failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? at lib/MusicBrainz/Server/Connector.pm line 42. [plus about 500 additional lines of error...] I'll try a complete clean rebuild. Thanks! Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 Ok, deleted /mnt/cache/apps/musicbrainz and /mnt/cache/data/musicbrainz and started a new build. Import files are downloading but postgresql is definitely not running. Same error as before. Tried to spawn postgres but immediately exited with error status 2. Tried a few times and then gave up. I looked around for another log to try to find the error, but couldn't locate anything that looked relevant. Can you point me to something? Thanks again! Scott Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 Well, here's maybe one thing... In the /root directory in the docker, there's a supervisord.conf. The command line for postgres is this: command=/usr/lib/postgresql/9.4/bin/postgres -D /data/main -c config_file=/data/main/postgresql.conf I did notice that the config_file appears to be at /data/postgresql.conf, not /data/main/postgresql.conf, unless it puts it there later and I just haven't gotten there yet. ?? Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 Standby... tried again and made sure everything was really cleaned out this time. May be making it further... Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 Ok, let it run all night and got much, much further. The ui fails now with this error: Caught exception in MusicBrainz::Server::Controller::Root->begin "08006 DBI connect('dbname=musicbrainz_db','musicbrainz',...) failed: FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory at lib/MusicBrainz/Server/Connector.pm line 42. If I look in /data/main, it's empty except for the global and pg_xlog directories. The database files that were there seem to have disappeared. Hmm, I think mover may be killing me... let me look into that. Link to comment
sparklyballs Posted June 18, 2015 Author Share Posted June 18, 2015 Ok, let it run all night and got much, much further. The ui fails now with this error: Caught exception in MusicBrainz::Server::Controller::Root->begin "08006 DBI connect('dbname=musicbrainz_db','musicbrainz',...) failed: FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory at lib/MusicBrainz/Server/Connector.pm line 42. If I look in /data/main, it's empty except for the global and pg_xlog directories. The database files that were there seem to have disappeared. Hmm, I think mover may be killing me... let me look into that. you want the data to be in a share that is not subject to mover Link to comment
barbergeek Posted June 18, 2015 Share Posted June 18, 2015 You'd think I'd remember that by now... That was it. Sorry for the false alarm. I think I'm good now. Thanks again. Very much appreciate all your support! Link to comment
johnodon Posted June 20, 2015 Share Posted June 20, 2015 Hey sparkly...I am eyeballing this one. My only concern: I have a monthly bandwidth allotment imposed by my ISP. Do you have any idea how much data is synced on a daily, weekly or monthly basis once you get past the initial 5GB DL? John Link to comment
sparklyballs Posted June 20, 2015 Author Share Posted June 20, 2015 Hey sparkly...I am eyeballing this one. My only concern: I have a monthly bandwidth allotment imposed by my ISP. Do you have any idea how much data is synced on a daily, weekly or monthly basis once you get past the initial 5GB DL? John no idea Link to comment
johnodon Posted June 21, 2015 Share Posted June 21, 2015 getting the following after the docker installs: *** Running /etc/my_init.d/001-fix-the-time.sh... Current default time zone: 'America/New_York' Local time is now: Sat Jun 6 16:41:12 EDT 2015. Universal Time is now: Sat Jun 6 20:41:12 UTC 2015. *** Running /etc/my_init.d/002-configure-DBDefs.sh... DBDefs is in your config folder, may need editing *** Running /etc/my_init.d/003-postgres-initialise.sh... The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "en_US.UTF-8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /data/main ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting dynamic shared memory implementation ... posix creating configuration files ... ok LOG: could not link file "pg_xlog/xlogtemp.42" to "pg_xlog/000000010000000000000001" (initialization of log file): Function not implemented FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory child process exited with exit code 1 initdb: removing contents of data directory "/data/main" creating template1 database in /data/main/base/1 ... 2015-06-06 16:41:18,033 CRIT Supervisor running as root (no user in config file) 2015-06-06 16:41:18,034 INFO supervisord started with pid 44 2015-06-06 16:41:19,036 INFO spawned: 'redis-server' with pid 48 2015-06-06 16:41:19,037 INFO spawned: 'postgres' with pid 49 2015-06-06 16:41:19,041 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:20,042 INFO success: redis-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2015-06-06 16:41:20,043 INFO spawned: 'postgres' with pid 52 2015-06-06 16:41:20,047 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:22,050 INFO spawned: 'postgres' with pid 53 2015-06-06 16:41:22,054 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:25,058 INFO spawned: 'postgres' with pid 54 2015-06-06 16:41:25,062 INFO exited: postgres (exit status 2; not expected) 2015-06-06 16:41:26,063 INFO gave up: postgres entered FATAL state, too many start retries too quickly BEGINNING INITIAL DATABASE IMPORT ROUTINE, THIS COULD TAKE SEVERAL HOURS AND THE DOCKER MAY LOOK UNRESPONSIVE DO NOT STOP DOCKER UNTIL IT IS COMPLETED I haven't used this container but... What is your volume mappings? ie: what do you have /data and /config mapped to? Im going to guess you have it set to /mnt/user or something using user shares, which do not support symlinks. You will need to use /mnt/cache or /mnt/disk# directly. Brit, can you expand on this? Is this just a MusicBrainz issue or a larger Docker issue? I use cache-only shares for my container configs/data (i.e. /mnt/user/Docker/*app*). Is this bad practice? Honestly, I don't think I have seen any issues with my other containers. sparkly...you may want to put this in the OP. I was also using a user share and ran into the same issue. John Link to comment
BRiT Posted June 21, 2015 Share Posted June 21, 2015 It is just how unraid works, the /mnt/user system does not support symlinks. So any program which uses symlinks will fail the same way. Link to comment
Recommended Posts