Are there examples of tools that run processes in the background?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Are there examples of tools that run processes in the background?

Melissa Cline
Hi folks,

We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications).  Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time.  Or is there a better approach I didn't consider?

Assuming the answer is "no", then are there any tools in the toolshed that launch background processes?  If so, I'd like to look at them.

And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down.  Is there a way to configure a process to be run when it's time to quit Galaxy?

Thanks!

Melissa



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Are there examples of tools that run processes in the background?

Ross-2
Hi Melissa,
The question in the title is easy to answer - most large galaxy instances run (most) jobs on a connected cluster which is effectively 'background' I guess so "yes - all of them on cluster backed Galaxy instances" might be a reasonable response. 

The questions in the text are more complex, but (eg) AFAIK there's no technical reason why some batch process owned by the Galaxy user shouldn't issue a
sh run.sh --stop-daemon
at any time to stop Galaxy. Exactly when that might be a good idea is not so clear to me so I'm probably missing something here - the Galaxy process doesn't consume a lot of resources and is typically left running permanently so users can take advantage of the persistence and reproducibility. Tools are AFAIK not generally written to launch their own background processes because they are managed by the Galaxy job runner which interfaces with (eg) SGE or Torque to run each separate tool invocation and clean up when it has completed. 

I'm sure you could run a transient Galaxy to batch up some workflows or other processes and I'm sure a VM might be part of that and the API allows a lot of automation. There's a new Galaxy docker image project which might be even more efficient and helpful - see http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019289.html



On Sat, Jun 7, 2014 at 10:18 AM, Melissa Cline <[hidden email]> wrote:
Hi folks,

We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications).  Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time.  Or is there a better approach I didn't consider?

Assuming the answer is "no", then are there any tools in the toolshed that launch background processes?  If so, I'd like to look at them.

And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down.  Is there a way to configure a process to be run when it's time to quit Galaxy?

Thanks!

Melissa



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/