setting SGE arguments in galaxy (cloudman)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

setting SGE arguments in galaxy (cloudman)

Jim McCusker
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: setting SGE arguments in galaxy (cloudman)

Enis Afgan-2
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/

I've just tried it and worked fine.

Sorry for a delay in getting back to you,
Enis


On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker <[hidden email]> wrote:
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: setting SGE arguments in galaxy (cloudman)

Jim McCusker
thanks, I ended up doing subinvocations (submitting a job from within a job) because one tool in particular needs lots of memory. Is there a way to set these things on a per-tool basis (which would let me remove the subinvocation)?

Jim


On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan <[hidden email]> wrote:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/

I've just tried it and worked fine.

Sorry for a delay in getting back to you,
Enis


On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker <[hidden email]> wrote:
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: setting SGE arguments in galaxy (cloudman)

Enis Afgan-2
If I'm understanding your question correctly (ie, have the ability to specify SGE requirements on a per-tool basis), then yes. You'd do something like this in universe_wsgi.ini:
[galaxy:tool_runners]
bwa_wrapper = drmaa://-pe smp 2/
bowtie2 = drmaa://-pe smp 2/


On Thu, Aug 14, 2014 at 2:26 PM, Jim McCusker <[hidden email]> wrote:
thanks, I ended up doing subinvocations (submitting a job from within a job) because one tool in particular needs lots of memory. Is there a way to set these things on a per-tool basis (which would let me remove the subinvocation)?

Jim


On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan <[hidden email]> wrote:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/

I've just tried it and worked fine.

Sorry for a delay in getting back to you,
Enis


On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker <[hidden email]> wrote:
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: setting SGE arguments in galaxy (cloudman)

Jim McCusker
Perfect, thanks! That's in addition to the default arguments?




On Thu, Aug 14, 2014 at 2:52 PM, Enis Afgan <[hidden email]> wrote:
If I'm understanding your question correctly (ie, have the ability to specify SGE requirements on a per-tool basis), then yes. You'd do something like this in universe_wsgi.ini:
[galaxy:tool_runners]
bwa_wrapper = drmaa://-pe smp 2/
bowtie2 = drmaa://-pe smp 2/


On Thu, Aug 14, 2014 at 2:26 PM, Jim McCusker <[hidden email]> wrote:
thanks, I ended up doing subinvocations (submitting a job from within a job) because one tool in particular needs lots of memory. Is there a way to set these things on a per-tool basis (which would let me remove the subinvocation)?

Jim


On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan <[hidden email]> wrote:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/

I've just tried it and worked fine.

Sorry for a delay in getting back to you,
Enis


On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker <[hidden email]> wrote:
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/





___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: setting SGE arguments in galaxy (cloudman)

Enis Afgan-2
The default args will apply to all the tools. Tool-specific args will apply only for the tool you specify and those overwrite the default (ie, arguments will not add up).


On Thu, Aug 14, 2014 at 2:53 PM, Jim McCusker <[hidden email]> wrote:
Perfect, thanks! That's in addition to the default arguments?




On Thu, Aug 14, 2014 at 2:52 PM, Enis Afgan <[hidden email]> wrote:
If I'm understanding your question correctly (ie, have the ability to specify SGE requirements on a per-tool basis), then yes. You'd do something like this in universe_wsgi.ini:
[galaxy:tool_runners]
bwa_wrapper = drmaa://-pe smp 2/
bowtie2 = drmaa://-pe smp 2/


On Thu, Aug 14, 2014 at 2:26 PM, Jim McCusker <[hidden email]> wrote:
thanks, I ended up doing subinvocations (submitting a job from within a job) because one tool in particular needs lots of memory. Is there a way to set these things on a per-tool basis (which would let me remove the subinvocation)?

Jim


On Thu, Aug 14, 2014 at 2:09 PM, Enis Afgan <[hidden email]> wrote:
Hi Jim
Try setting it like this:
default_cluster_job_runner = drmaa://-w n -l mem_free=16G/

I've just tried it and worked fine.

Sorry for a delay in getting back to you,
Enis


On Tue, Aug 5, 2014 at 6:44 PM, Jim McCusker <[hidden email]> wrote:
I have a galaxy/cloudman cluster where I would like to pass a "-l mem_free=16G" parameter when galaxy submits the job to SGE. I've tried adding it like this:

default_cluster_job_runner = drmaa://-l mem_free=16G/

(it was:
default_cluster_job_runner = drmaa:///
)

but now the jobs don't even submit. Am I wiping out other important parameters? How should I do this?

Thanks,
Jim

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/






___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/