Daemon mode fail job submission

classic Classic list List threaded Threaded
3 messages Options
| Threaded
Open this post in threaded view
|

Daemon mode fail job submission

Luc Cornet
Hi galaxy community,

I am quite new in galaxy but I would like to install a local instance on our cluster (SGE).
I have a problem with DRMAA :

When I activate the --daemon mode, the job don't pass in the queue system but with no Damon, it pass (the log below)
The second problem is that $GALAXY_SLOTS is not define, Could it be related to the problem below ?
Someone can help me, please ? (available at : https://help.galaxyproject.org/t/job-submission-failed-when-daemon/351)

Thanks,
Luc

## with no daemon
    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,424 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/47/galaxy_47.sh
    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,425 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) native specification is: -q cassava.q -pe snode 10
    galaxy.jobs.runners.drmaa INFO 2020-04-28 17:07:21,440 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) queued as 1007701
    galaxy.jobs DEBUG 2020-04-28 17:07:21,441 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) Persisting job destination (destination id: big_jobs)
    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:22,042 [p:32191,w:1,m:0] [DRMAARunner.monitor_thread] (47/1007701) state change: job is queued and active

## with daemon
    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/45/galaxy_45.sh
    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) native specification is: -q cassava.q -pe snode 10

## job_conf

    <?xml version="1.0"?>
    <!-- A sample job config that explicitly configures job running the way it is
         configured by default (if there is no explicit config). -->
    <job_conf>
        <plugins>
            <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
            <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
                <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
        </plugins>
        <destinations default="local">
            <destination id="local" runner="local"/>
            <destination id="big_jobs" runner="drmaa">
                <param id="nativeSpecification">-q cassava.q -pe snode 10</param>
            </destination>
        </destinations>
        <tools>
           <tool id="maker" destination="big_jobs" />
           <tool id="metaspades" destination="big_jobs" />
        </tools>
    </job_conf>



------------
Luc Cornet, PhD
Bio-informatician
Mycology and Aerobiology
Sciensano
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Daemon mode fail job submission

Gildas Le Corguillé
Hi,

There is at least one issue

           <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
               <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>


           <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner">
               <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
          </plugin>      

I don’t know if it’s THE issue :/




Gildas

----------------------------------------------------------------------------------------------------------------------------
Gildas Le Corguillé - Bioinformatician/Bioanalyste

Plateform ABiMS (Analyses and Bioinformatics for Marine Science): http://abims.sb-roscoff.fr <http://abims.sb-roscoff.fr/>

Co-leader of the IFB Core Cluster: https://www.france-bioinformatique.fr/en/cluster <https://www.france-bioinformatique.fr/fr/cluster>

Member of the Workflow4Metabolomics project: http://workflow4metabolomics.org <http://workflow4metabolomics.org/>

Station Biologique de Roscoff
Sorbonne Université / CNRS
Place Georges Teissier 29680 Roscoff FRANCE
tel: +33 2 98 29 23 81
----------------------------------------------------------------------------------------------------------------------------


> Le 28 avr. 2020 à 17:32, Luc Cornet <[hidden email]> a écrit :
>
> Hi galaxy community,
>
> I am quite new in galaxy but I would like to install a local instance on our cluster (SGE).
> I have a problem with DRMAA :
>
> When I activate the --daemon mode, the job don't pass in the queue system but with no Damon, it pass (the log below)
> The second problem is that $GALAXY_SLOTS is not define, Could it be related to the problem below ?
> Someone can help me, please ? (available at : https://help.galaxyproject.org/t/job-submission-failed-when-daemon/351)
>
> Thanks,
> Luc
>
> ## with no daemon
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,424 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/47/galaxy_47.sh
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,425 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) native specification is: -q cassava.q -pe snode 10
>    galaxy.jobs.runners.drmaa INFO 2020-04-28 17:07:21,440 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) queued as 1007701
>    galaxy.jobs DEBUG 2020-04-28 17:07:21,441 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) Persisting job destination (destination id: big_jobs)
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:22,042 [p:32191,w:1,m:0] [DRMAARunner.monitor_thread] (47/1007701) state change: job is queued and active
>
> ## with daemon
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/45/galaxy_45.sh
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) native specification is: -q cassava.q -pe snode 10
>
> ## job_conf
>
>    <?xml version="1.0"?>
>    <!-- A sample job config that explicitly configures job running the way it is
>         configured by default (if there is no explicit config). -->
>    <job_conf>
>        <plugins>
>            <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
>            <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
>                <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
>        </plugins>
>        <destinations default="local">
>            <destination id="local" runner="local"/>
>            <destination id="big_jobs" runner="drmaa">
>                <param id="nativeSpecification">-q cassava.q -pe snode 10</param>
>            </destination>
>        </destinations>
>        <tools>
>           <tool id="maker" destination="big_jobs" />
>           <tool id="metaspades" destination="big_jobs" />
>        </tools>
>    </job_conf>
>
>
>
> ------------
> Luc Cornet, PhD
> Bio-informatician
> Mycology and Aerobiology
> Sciensano
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Daemon mode fail job submission

Luc Cornet
Re,

Thanks for the correction.
Unfortunately, it was not the issue.

Luc,

##job_conf

<?xml version="1.0"?>
<job_conf>
   <plugins>
      <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
      <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner">
         <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
      </plugin>
   </plugins>
   <destinations default="local">
      <destination id="local" runner="local"/>
      <destination id="big_jobs" runner="drmaa">
         <param id="nativeSpecification">-q cassava.q -pe snode 10</param>
      </destination>
   </destinations>
   <tools>
         <tool id="maker" destination="big_jobs" />
         <tool id="metaspades" destination="big_jobs" />
   </tools>
</job_conf>

------------
Luc Cornet, PhD
Bio-informatician
Mycology and Aerobiology
Sciensano

----- Mail original -----
De: "Gildas Le Corguillé" <[hidden email]>
À: "Luc Cornet" <[hidden email]>
Cc: [hidden email]
Envoyé: Mardi 28 Avril 2020 18:13:58
Objet: Re: [galaxy-dev] Daemon mode fail job submission

Hi,

There is at least one issue

           <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
               <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>


           <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner">
               <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
          </plugin>      

I don’t know if it’s THE issue :/




Gildas

----------------------------------------------------------------------------------------------------------------------------
Gildas Le Corguillé - Bioinformatician/Bioanalyste

Plateform ABiMS (Analyses and Bioinformatics for Marine Science): http://abims.sb-roscoff.fr <http://abims.sb-roscoff.fr/>

Co-leader of the IFB Core Cluster: https://www.france-bioinformatique.fr/en/cluster <https://www.france-bioinformatique.fr/fr/cluster>

Member of the Workflow4Metabolomics project: http://workflow4metabolomics.org <http://workflow4metabolomics.org/>

Station Biologique de Roscoff
Sorbonne Université / CNRS
Place Georges Teissier 29680 Roscoff FRANCE
tel: +33 2 98 29 23 81
----------------------------------------------------------------------------------------------------------------------------


> Le 28 avr. 2020 à 17:32, Luc Cornet <[hidden email]> a écrit :
>
> Hi galaxy community,
>
> I am quite new in galaxy but I would like to install a local instance on our cluster (SGE).
> I have a problem with DRMAA :
>
> When I activate the --daemon mode, the job don't pass in the queue system but with no Damon, it pass (the log below)
> The second problem is that $GALAXY_SLOTS is not define, Could it be related to the problem below ?
> Someone can help me, please ? (available at : https://help.galaxyproject.org/t/job-submission-failed-when-daemon/351)
>
> Thanks,
> Luc
>
> ## with no daemon
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,424 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/47/galaxy_47.sh
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:21,425 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) native specification is: -q cassava.q -pe snode 10
>    galaxy.jobs.runners.drmaa INFO 2020-04-28 17:07:21,440 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) queued as 1007701
>    galaxy.jobs DEBUG 2020-04-28 17:07:21,441 [p:32191,w:1,m:0] [DRMAARunner.work_thread-1] (47) Persisting job destination (destination id: big_jobs)
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 17:07:22,042 [p:32191,w:1,m:0] [DRMAARunner.monitor_thread] (47/1007701) state change: job is queued and active
>
> ## with daemon
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) submitting file /media/vol2/home/galaxy/galaxy/database/jobs_directory/000/45/galaxy_45.sh
>    galaxy.jobs.runners.drmaa DEBUG 2020-04-28 15:39:16,138 [p:26779,w:1,m:0] [DRMAARunner.work_thread-1] (45) native specification is: -q cassava.q -pe snode 10
>
> ## job_conf
>
>    <?xml version="1.0"?>
>    <!-- A sample job config that explicitly configures job running the way it is
>         configured by default (if there is no explicit config). -->
>    <job_conf>
>        <plugins>
>            <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
>            <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
>                <param id="drmaa_library_path">/media/vol1/gridengine/lib/linux-x64/libdrmaa.so</param>
>        </plugins>
>        <destinations default="local">
>            <destination id="local" runner="local"/>
>            <destination id="big_jobs" runner="drmaa">
>                <param id="nativeSpecification">-q cassava.q -pe snode 10</param>
>            </destination>
>        </destinations>
>        <tools>
>           <tool id="maker" destination="big_jobs" />
>           <tool id="metaspades" destination="big_jobs" />
>        </tools>
>    </job_conf>
>
>
>
> ------------
> Luc Cornet, PhD
> Bio-informatician
> Mycology and Aerobiology
> Sciensano
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  %(web_page_url)s
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  %(web_page_url)s

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/