Check running jobs

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Check running jobs

Katherine Beaulieu
Hi there,
We are trying to build our own Galaxy cluster and I was wondering if there was a way to check if there are any running Galaxy jobs on a specific server? This would be to determine whether or not to remove the server from the cluster.
Thanks!
Katherine

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Check running jobs

Peter Cock
Hi Katherine,

I don't think Galaxy itself will tell you this, in part because the
details would be dependent on the Cluster being used.

I would do that via whatever cluster management system you are running
(e.g. qstat or qmon if using SGE).

By default all the jobs will be submitted by the Galaxy user account,
but if you are submitting jobs as the individual users then you can
still spot Galaxy jobs via their naming convention (starting "g",
number, underscore, ..., tool name, user's email).

Peter

On Thu, Jan 26, 2017 at 6:58 PM, Katherine Beaulieu
<[hidden email]> wrote:

> Hi there,
> We are trying to build our own Galaxy cluster and I was wondering if there
> was a way to check if there are any running Galaxy jobs on a specific
> server? This would be to determine whether or not to remove the server from
> the cluster.
> Thanks!
> Katherine
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Check running jobs

Katherine Beaulieu
Ok so there is no way to access the galaxy object outside of galaxy? I'm looking for something like the 'trans' variable often seen throughout the Galaxy code base so that I can use sqlalchemy and make a database query.
Thanks for replying.
Katherine

On Fri, Jan 27, 2017 at 5:40 AM, Peter Cock <[hidden email]> wrote:
Hi Katherine,

I don't think Galaxy itself will tell you this, in part because the
details would be dependent on the Cluster being used.

I would do that via whatever cluster management system you are running
(e.g. qstat or qmon if using SGE).

By default all the jobs will be submitted by the Galaxy user account,
but if you are submitting jobs as the individual users then you can
still spot Galaxy jobs via their naming convention (starting "g",
number, underscore, ..., tool name, user's email).

Peter

On Thu, Jan 26, 2017 at 6:58 PM, Katherine Beaulieu
<[hidden email]> wrote:
> Hi there,
> We are trying to build our own Galaxy cluster and I was wondering if there
> was a way to check if there are any running Galaxy jobs on a specific
> server? This would be to determine whether or not to remove the server from
> the cluster.
> Thanks!
> Katherine
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Check running jobs

Peter Cock
Ah - I assumed you meant from the user-facing UI, or the reports page.

I would suggest exploring the advance job metrics options, see e.g.

https://github.com/galaxyproject/galaxy/blob/dev/config/job_metrics_conf.xml.sample

I've never used it, but perhaps the backend using Collectl
(http://collectl.sourceforge.net/) would cover this?

Peter

On Fri, Jan 27, 2017 at 2:50 PM, Katherine Beaulieu
<[hidden email]> wrote:
> Ok so there is no way to access the galaxy object outside of galaxy? I'm
> looking for something like the 'trans' variable often seen throughout the
> Galaxy code base so that I can use sqlalchemy and make a database query.
> Thanks for replying.
> Katherine
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Check running jobs

Katherine Beaulieu
I think this is a bit more complicated than what I am looking for. All I am looking for is a way to get the result of this query: 
jobs = trans.sa_session.query( trans.app.model.Job ) \
                               .filter( and_( trans.app.model.Job.table.c.update_time < cutoff_time,
                                              or_( trans.app.model.Job.state == trans.app.model.Job.states.NEW,
                                                   trans.app.model.Job.state == trans.app.model.Job.states.QUEUED,
                                                   trans.app.model.Job.state == trans.app.model.Job.states.RUNNING,
                                                   trans.app.model.Job.state == trans.app.model.Job.states.UPLOAD ) ) ) \
                               .order_by( trans.app.model.Job.table.c.update_time.desc() ).all()
from the file /lib/galaxy/web/base/controllers/admin.py, without having to use the UI. Has anyone done something like this before?  

On Fri, Jan 27, 2017 at 10:03 AM, Peter Cock <[hidden email]> wrote:
Ah - I assumed you meant from the user-facing UI, or the reports page.

I would suggest exploring the advance job metrics options, see e.g.

https://github.com/galaxyproject/galaxy/blob/dev/config/job_metrics_conf.xml.sample

I've never used it, but perhaps the backend using Collectl
(http://collectl.sourceforge.net/) would cover this?

Peter

On Fri, Jan 27, 2017 at 2:50 PM, Katherine Beaulieu
<[hidden email]> wrote:
> Ok so there is no way to access the galaxy object outside of galaxy? I'm
> looking for something like the 'trans' variable often seen throughout the
> Galaxy code base so that I can use sqlalchemy and make a database query.
> Thanks for replying.
> Katherine


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/