Galaxy Startup Failing

classic Classic list List threaded Threaded
7 messages Options
| Threaded
Open this post in threaded view
|

Galaxy Startup Failing

Keith Suderman
Greetings,

I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:

> sudo service galaxy start

no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.

Galaxy does start with:

> sudo su galaxy
> ./run.sh --daemon

so I have left Galaxy running in a screen session for now, but that is less than ideal.

I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?

Cheers,
Keith

1. http://simison.com/brian/GalaxyAdmin_notes.html
2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29

------------------------------
Research Associate
Department of Computer Science
Vassar College
Poughkeepsie, NY


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Keith Suderman
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Will Holtz
Hi Keith,

I've dealt with similar situations by adding the sourcing of .venv/bin/activate to the etc/init.d/galaxy script. Then none of the pre-existing pid code needs to be changed.

-Will


On Tue, Nov 3, 2015 at 8:55 AM, Suderman Keith <[hidden email]> wrote:
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Keith Suderman
Hi Will,

I tried that and it didn’t work for me.  Are you using start-stop-daemon to launch python?  Or do you run “python scripts/paster.py serve” from /etc/init.d/galaxy?

If I use start-stop-daemon then python does not seem to be picking up the .venv.  I suspect it has something to do with .venv/bin/activate being sourced as root in one process and then python running as the galaxy user in another process.

If I run “python scripts/paster.py serve” in the /etc/init.d/galaxy script then Galaxy is running as the root user and this is (will be) a production server so that is not an option.

If I run python as the galaxy user from the /etc/init.d/galaxy script then I am back to the problem of python not being able to write to /var/run/galaxy.pid

Keith

On Nov 3, 2015, at 7:39 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I've dealt with similar situations by adding the sourcing of .venv/bin/activate to the etc/init.d/galaxy script. Then none of the pre-existing pid code needs to be changed.

-Will


On Tue, Nov 3, 2015 at 8:55 AM, Suderman Keith <[hidden email]> wrote:
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Will Holtz
Hi Keith,

I use start-stop-daemon to launch python and I pass --chuid and --group to start-stop-daemon so that galaxy does not run as root. It is my understanding that start-stop-daemon creates the pid file as root before changing user, as my /var/run/galaxy.pid is owned by root:root. I have attached my /etc/init.d/galaxy script. This is from a Ubuntu 12.04 system.

-Will



On Wed, Nov 4, 2015 at 8:13 AM, Suderman Keith <[hidden email]> wrote:
Hi Will,

I tried that and it didn’t work for me.  Are you using start-stop-daemon to launch python?  Or do you run “python scripts/paster.py serve” from /etc/init.d/galaxy?

If I use start-stop-daemon then python does not seem to be picking up the .venv.  I suspect it has something to do with .venv/bin/activate being sourced as root in one process and then python running as the galaxy user in another process.

If I run “python scripts/paster.py serve” in the /etc/init.d/galaxy script then Galaxy is running as the root user and this is (will be) a production server so that is not an option.

If I run python as the galaxy user from the /etc/init.d/galaxy script then I am back to the problem of python not being able to write to /var/run/galaxy.pid

Keith

On Nov 3, 2015, at 7:39 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I've dealt with similar situations by adding the sourcing of .venv/bin/activate to the etc/init.d/galaxy script. Then none of the pre-existing pid code needs to be changed.

-Will


On Tue, Nov 3, 2015 at 8:55 AM, Suderman Keith <[hidden email]> wrote:
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.




--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

galaxy (1K) Download Attachment
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Keith Suderman
Hi Will,

Thanks for the response, but I am still not having any luck.  The script you attached is almost identical to what I have with a few minor exceptions:

1. You include -ES parameters when invoking python.
2. You export PYTHON_EGG_CACHE
3. Your config is in universe_wsgi.ini (in, I assume, /home/galaxy/galaxy-dist) while mine is in config/galaxy.ini
4. You source the virtualenv from /home/galaxy/galaxy_env/bin/activate while mine is in /home/galaxy/galaxy_dist/.venv/bin/activate

I have made the changes for #1 and #2, #3 shouldn't be an issue, which leaves #4 as the only possible remaining culprit.  I have attached my .venv/bin/activate in case you can spot any differences, but since this is generated automatically it shouldn't be the problem.  I am starting to think I might just have to live with my hack-around for now.

Thanks for the help.
Cheers,
Keith

On Nov 5, 2015, at 7:44 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I use start-stop-daemon to launch python and I pass --chuid and --group to start-stop-daemon so that galaxy does not run as root. It is my understanding that start-stop-daemon creates the pid file as root before changing user, as my /var/run/galaxy.pid is owned by root:root. I have attached my /etc/init.d/galaxy script. This is from a Ubuntu 12.04 system.

-Will



On Wed, Nov 4, 2015 at 8:13 AM, Suderman Keith <[hidden email]> wrote:
Hi Will,

I tried that and it didn’t work for me.  Are you using start-stop-daemon to launch python?  Or do you run “python scripts/paster.py serve” from /etc/init.d/galaxy?

If I use start-stop-daemon then python does not seem to be picking up the .venv.  I suspect it has something to do with .venv/bin/activate being sourced as root in one process and then python running as the galaxy user in another process.

If I run “python scripts/paster.py serve” in the /etc/init.d/galaxy script then Galaxy is running as the root user and this is (will be) a production server so that is not an option.

If I run python as the galaxy user from the /etc/init.d/galaxy script then I am back to the problem of python not being able to write to /var/run/galaxy.pid

Keith

On Nov 3, 2015, at 7:39 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I've dealt with similar situations by adding the sourcing of .venv/bin/activate to the etc/init.d/galaxy script. Then none of the pre-existing pid code needs to be changed.

-Will


On Tue, Nov 3, 2015 at 8:55 AM, Suderman Keith <[hidden email]> wrote:
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.




--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.
<galaxy>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

activate (2K) Download Attachment
| Threaded
Open this post in threaded view
|

Re: Galaxy Startup Failing

Will Holtz
Hi Keith,

Sorry for the delayed response, as I was on vacation. Your activate script is the same as mine except for the obvious directory name differences. Unfortunately I am out of ideas.

-Will

On Fri, Nov 6, 2015 at 6:23 PM, Suderman Keith <[hidden email]> wrote:
Hi Will,

Thanks for the response, but I am still not having any luck.  The script you attached is almost identical to what I have with a few minor exceptions:

1. You include -ES parameters when invoking python.
2. You export PYTHON_EGG_CACHE
3. Your config is in universe_wsgi.ini (in, I assume, /home/galaxy/galaxy-dist) while mine is in config/galaxy.ini
4. You source the virtualenv from /home/galaxy/galaxy_env/bin/activate while mine is in /home/galaxy/galaxy_dist/.venv/bin/activate

I have made the changes for #1 and #2, #3 shouldn't be an issue, which leaves #4 as the only possible remaining culprit.  I have attached my .venv/bin/activate in case you can spot any differences, but since this is generated automatically it shouldn't be the problem.  I am starting to think I might just have to live with my hack-around for now.

Thanks for the help.
Cheers,
Keith

On Nov 5, 2015, at 7:44 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I use start-stop-daemon to launch python and I pass --chuid and --group to start-stop-daemon so that galaxy does not run as root. It is my understanding that start-stop-daemon creates the pid file as root before changing user, as my /var/run/galaxy.pid is owned by root:root. I have attached my /etc/init.d/galaxy script. This is from a Ubuntu 12.04 system.

-Will



On Wed, Nov 4, 2015 at 8:13 AM, Suderman Keith <[hidden email]> wrote:
Hi Will,

I tried that and it didn’t work for me.  Are you using start-stop-daemon to launch python?  Or do you run “python scripts/paster.py serve” from /etc/init.d/galaxy?

If I use start-stop-daemon then python does not seem to be picking up the .venv.  I suspect it has something to do with .venv/bin/activate being sourced as root in one process and then python running as the galaxy user in another process.

If I run “python scripts/paster.py serve” in the /etc/init.d/galaxy script then Galaxy is running as the root user and this is (will be) a production server so that is not an option.

If I run python as the galaxy user from the /etc/init.d/galaxy script then I am back to the problem of python not being able to write to /var/run/galaxy.pid

Keith

On Nov 3, 2015, at 7:39 PM, Will Holtz <[hidden email]> wrote:

Hi Keith,

I've dealt with similar situations by adding the sourcing of .venv/bin/activate to the etc/init.d/galaxy script. Then none of the pre-existing pid code needs to be changed.

-Will


On Tue, Nov 3, 2015 at 8:55 AM, Suderman Keith <[hidden email]> wrote:
The solution was rather simple, once I figured it out.

The /etc/init.d/galaxy script was invoking scripts/paster.py script directly so .venv/bin/activate was not being sourced.  The solution was to modify /etc/init.d/galaxy to call a startup script that sourced .venv/bin/activate before starting Galaxy.  I can write my solution up in more detail, but now I am having some problems setting the pid file correctly.

If I use the --make-pidfile and --pidfile options for start-stop-daemon in /etc/init.d/galaxy then /var/run/galaxy.pid contains the pid of the startup script and not the pid of the Galaxy instance.

If I try to set the pid in the startup script Galaxy won't start as the script (now running as the Galaxy user) can't create/write the /var/run/galaxy.pid file.

My hack-around has been to touch the pid file (in case it doesn't exist) and then chown it to the galaxy user in /etc/init.d/galaxy, but this doesn't strike me as the correct way to do this.  Do any sysadmin gurus have any better suggestions?

Cheers,
Keith

> On Nov 1, 2015, at 9:31 PM, Keith Suderman <[hidden email]> wrote:
>
> Greetings,
>
> I am trying to upgrade our Galaxy instance to the latest stable code from master and now our instance won't start using my /etc/init.d/galaxy script.  That is:
>
>> sudo service galaxy start
>
> no longer works. It says it does (i.e. prints "[OK]"), but Galaxy isn't running and nothing is written to the log. I am using the init.d script from simison [1], which is basically the same as the one posted to the Galaxy wiki [2], both of which are quite old.
>
> Galaxy does start with:
>
>> sudo su galaxy
>> ./run.sh --daemon
>
> so I have left Galaxy running in a screen session for now, but that is less than ideal.
>
> I suspect something to do with the .venv as `./run.sh --skip-venv` also fails and I don't recall Galaxy being that finicky about a venv being active. Does anyone know how I can get the /etc/init.d/galaxy script (i.e. start-stop-daemon) to activate the venv for paster.py?
>
> Cheers,
> Keith
>
> 1. http://simison.com/brian/GalaxyAdmin_notes.html
> 2. https://wiki.galaxyproject.org/Events/GCC2012/TrainingDay/WS5?highlight=%28init.d%29
>
> ------------------------------
> Research Associate
> Department of Computer Science
> Vassar College
> Poughkeepsie, NY
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.




--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.
<galaxy>





--
The information contained in this e-mail message or any attachment(s) may be confidential and/or privileged and is intended for use only by the individual(s) to whom this message is addressed.  If you are not the intended recipient, any dissemination, distribution, copying, or use is strictly prohibited.  If you receive this e-mail message in error, please e-mail the sender at [hidden email] and destroy this message and remove the transmission from all computer directories (including e-mail servers).

Please consider the environment before printing this email.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/