restarting Galaxy without affecting jobs

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

restarting Galaxy without affecting jobs

David Hoover
What are the configuration steps required for allowing a local Galaxy installation to be restarted without affecting currently running jobs?  I have Galaxy using DRMAA to submit jobs onto a backend cluster.  I thought that enable_job_recovery = True should allow this, but in a few tests I have found that although the batch jobs completed, Galaxy lost track of the jobs and classified them as failed.  Would track_jobs_in_database = True be required?  This is currently set to the default 'None'.

Our local Galaxy installation has become quite busy, and restarts are not possible without forcing users to restart their jobs.

David Hoover
Helix Systems Staff
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: restarting Galaxy without affecting jobs

Nate Coraor (nate@bx.psu.edu)
Hi David,

Setting track_jobs_in_database = True should not be required, recovery is supposed to work either way.

Does Galaxy lose all jobs, or just the ones that completed while Galaxy was restarting? Can you provide the output from the Galaxy log that shows an attempt to recover a job and all related messages?

Thanks,
--nate


On Mon, Mar 24, 2014 at 11:13 AM, David Hoover <[hidden email]> wrote:
What are the configuration steps required for allowing a local Galaxy installation to be restarted without affecting currently running jobs?  I have Galaxy using DRMAA to submit jobs onto a backend cluster.  I thought that enable_job_recovery = True should allow this, but in a few tests I have found that although the batch jobs completed, Galaxy lost track of the jobs and classified them as failed.  Would track_jobs_in_database = True be required?  This is currently set to the default 'None'.

Our local Galaxy installation has become quite busy, and restarts are not possible without forcing users to restart their jobs.

David Hoover
Helix Systems Staff
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/