Galaxy job will stay in waiting under DRMAA runner until manually trigger a command line submission

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Galaxy job will stay in waiting under DRMAA runner until manually trigger a command line submission

Linda Cham

Hi.

Running standalone Galaxy stable_2013.06.03 version on RH6.2 with postgresl db with GPFS filesystem.
Setup via the directions in http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
to run all jobs through the LSF DRMAA runner.

job_conf.xml:
--------------------
<job_conf>
    <plugins>
        <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" workers="4"/>
    </plugins>
    <handlers>
        <handler id="main"/>
    </handlers>
    <destinations default="lsf">
        <destination id="lsf" runner="drmaa"/>
    </destinations>
</job_conf>

Scheduler and Galaxy were started.
Submit a FASTQ groomer job on a FASTQ file under Galaxy.
The groomer job in History will stay in GREY "Job is waiting to run" state until one physically go onto the node and submits a command line job through the scheduler.
Once that command line job (e.g. sleep 5)  is submitted then the groomer job will show up in the scheduler's job queue and galaxy's HISTORY will now show it as running.  
I can submit another groomer job after this and it will show up as running.

At times, one will see this type of scenario again where one has to manually start a job under the scheduler on the command line to trigger the galaxy's jobs to start running under the scheduler.

Is there some setup I am missing in the job_conf.xml above or a flag not set in universe_wsgi.ini?

Searching the Galaxy web, I had read that one can set the track_jobs_in_database=True to trigger it to look in memory or database.  But that didn't work either.

Wonder if anyone seen this type of issue with any runners and have any suggestions on how to debug and fix this.
Thanks.

Linda
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/