How To properly use GALAXY_SLOTS

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

How To properly use GALAXY_SLOTS

Geert Vandeweyer-4
Hi,

Is there documentation for the proper setup and utilisation of the
\${GALAXY_SLOTS:-4} style options?

I noticed that tools with this setting, including BWA and GATK run
single threaded, with the following settings respectively:

BWA  \${GALAXY_SLOTS:-4}"
===
ps output: python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
--threads=1 --fileSource=indexed
--ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
--do_not_build_index
--input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
--input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
--output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
--genAlignType=paired --params=pre_set --suppressHeader=false

xml config: --threads="\${GALAXY_SLOTS:-4}"


GATK Base Recalibrator \${GALAXY_SLOTS:-8}
==============
python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
--max_jvm_heap 6g --stdout
/galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
/galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
gatk_input -d
/galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat bam_index
gatk_input -p           java -jar
"$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
$GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
${GALAXY_SLOTS:-8}      --no_standard_covs          -R
"/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta"     --out
"/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
"ContextCovariate"             -cov "CycleCovariate"     -d
--knownSites:dbsnp,%(file_type)s
/galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
/galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
"NONE" -p          --baq "OFF" --baqGapOpenPenalty
"40.0"                  --defaultBaseQualities "-1"        
--validation_strictness "STRICT" --interval_merging "ALL"

java -Xmx6g -jar
/galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
-R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
--interval_set_rule UNION --downsampling_type NONE --baq OFF
--baqGapOpenPenalty 40.0 --defaultBaseQualities -1
--validation_strictness STRICT --interval_merging ALL -I
/tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
/tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
/tmp/tmp-gatk-fIuncH/input_intervals_0.bed


Best,

Geert




--


Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: [hidden email]
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: How To properly use GALAXY_SLOTS

Peter Cock
On Wed, Mar 12, 2014 at 3:32 PM, Geert Vandeweyer
<[hidden email]> wrote:

> Hi,
>
> Is there documentation for the proper setup and utilisation of the
> \${GALAXY_SLOTS:-4} style options?
>
> I noticed that tools with this setting, including BWA and GATK run single
> threaded, with the following settings respectively:
>
> BWA  \${GALAXY_SLOTS:-4}"
> ===
> ps output: python
> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
> --threads=1 --fileSource=indexed
> --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
> --do_not_build_index
> --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
> --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
> --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
> --genAlignType=paired --params=pre_set --suppressHeader=false
>
> xml config: --threads="\${GALAXY_SLOTS:-4}"

This is bash syntax meaning use the value of $GALAXY_SLOTS
or if undefined, default to 4 (the minus sign seems to be there
just to confuse everyone - joke - it is not Galaxy's fault).

The value will depend on how your job runners are configured,
for example we use SGE and set the threads there on a tool
by tool basis, e.g.

ncbi_blastp_wrapper  = drmaa://-V -pe smp 4/

That results in SGE allocating 4 threads, and Galaxy will
therefore set $GALAXY_SLOTS to four.

In your case, evidently $GALAXY_SLOTS was set to one.

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: How To properly use GALAXY_SLOTS

Björn Grüning-3
In reply to this post by Geert Vandeweyer-4
Hi Geert,

you can give every tool a different amount of SLOTS via the job_conf.xml
file.

https://wiki.galaxyproject.org/Admin/Config/Jobs

For example:

         <tool
id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
destination="12cores_24G" />


where 12cores_24G is defined with a parallel SGE environment.

Cheers,
Bjoern

Am 12.03.2014 16:32, schrieb Geert Vandeweyer:

> Hi,
>
> Is there documentation for the proper setup and utilisation of the
> \${GALAXY_SLOTS:-4} style options?
>
> I noticed that tools with this setting, including BWA and GATK run
> single threaded, with the following settings respectively:
>
> BWA  \${GALAXY_SLOTS:-4}"
> ===
> ps output: python
> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
> --threads=1 --fileSource=indexed
> --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
> --do_not_build_index
> --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
> --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
> --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
> --genAlignType=paired --params=pre_set --suppressHeader=false
>
> xml config: --threads="\${GALAXY_SLOTS:-4}"
>
>
> GATK Base Recalibrator \${GALAXY_SLOTS:-8}
> ==============
> python
> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
> --max_jvm_heap 6g --stdout
> /galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
> /galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
> gatk_input -d
> /galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat bam_index
> gatk_input -p           java -jar
> "$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
> $GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
> ${GALAXY_SLOTS:-8}      --no_standard_covs          -R
> "/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta"     --out
> "/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
> "ContextCovariate"             -cov "CycleCovariate"     -d
> --knownSites:dbsnp,%(file_type)s
> /galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
> input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
> /galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
> input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
> "NONE" -p          --baq "OFF" --baqGapOpenPenalty
> "40.0"                  --defaultBaseQualities "-1"
> --validation_strictness "STRICT" --interval_merging "ALL"
>
> java -Xmx6g -jar
> /galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
> BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
> -R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
> /galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
> ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
> --interval_set_rule UNION --downsampling_type NONE --baq OFF
> --baqGapOpenPenalty 40.0 --defaultBaseQualities -1
> --validation_strictness STRICT --interval_merging ALL -I
> /tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
> /tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
> /tmp/tmp-gatk-fIuncH/input_intervals_0.bed
>
>
> Best,
>
> Geert
>
>
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: How To properly use GALAXY_SLOTS

Geert Vandeweyer-4
How would those statements translate to pbs/torque (pbs_python)

I request resources using the nodes=1:ppn:4 syntax.  The -pe argument is
not available it seems...

Best,

Geert

On 03/12/2014 04:43 PM, Björn Grüning wrote:

> Hi Geert,
>
> you can give every tool a different amount of SLOTS via the
> job_conf.xml file.
>
> https://wiki.galaxyproject.org/Admin/Config/Jobs
>
> For example:
>
>         <tool
> id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
> destination="12cores_24G" />
>
>
> where 12cores_24G is defined with a parallel SGE environment.
>
> Cheers,
> Bjoern
>
> Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
>> Hi,
>>
>> Is there documentation for the proper setup and utilisation of the
>> \${GALAXY_SLOTS:-4} style options?
>>
>> I noticed that tools with this setting, including BWA and GATK run
>> single threaded, with the following settings respectively:
>>
>> BWA  \${GALAXY_SLOTS:-4}"
>> ===
>> ps output: python
>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
>>
>> --threads=1 --fileSource=indexed
>> --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
>> --do_not_build_index
>> --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
>> --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
>> --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
>> --genAlignType=paired --params=pre_set --suppressHeader=false
>>
>> xml config: --threads="\${GALAXY_SLOTS:-4}"
>>
>>
>> GATK Base Recalibrator \${GALAXY_SLOTS:-8}
>> ==============
>> python
>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
>>
>> --max_jvm_heap 6g --stdout
>> /galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
>> /galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
>> gatk_input -d
>> /galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
>> bam_index
>> gatk_input -p           java -jar
>> "$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
>> $GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
>> ${GALAXY_SLOTS:-8}      --no_standard_covs          -R
>> "/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
>> "/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
>> "ContextCovariate"             -cov "CycleCovariate"     -d
>> --knownSites:dbsnp,%(file_type)s
>> /galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
>> input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
>> /galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
>> input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
>> "NONE" -p          --baq "OFF" --baqGapOpenPenalty
>> "40.0"                  --defaultBaseQualities "-1"
>> --validation_strictness "STRICT" --interval_merging "ALL"
>>
>> java -Xmx6g -jar
>> /galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
>> BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
>> -R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
>> /galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
>> ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
>> --interval_set_rule UNION --downsampling_type NONE --baq OFF
>> --baqGapOpenPenalty 40.0 --defaultBaseQualities -1
>> --validation_strictness STRICT --interval_merging ALL -I
>> /tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
>> /tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
>> /tmp/tmp-gatk-fIuncH/input_intervals_0.bed
>>
>>
>> Best,
>>
>> Geert
>>
>>
>>
>>


--

Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: [hidden email]
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: How To properly use GALAXY_SLOTS

John Chilton-4
I am not sure I understand what you mean, but if you are setting the
job to run with a ppn of greater than one but GALAXY_SLOTS is being
evaluated as 1 then you have likely uncovered a bug - maybe in the
runner, your job configuration, or the GALAXY_SLOTS logic for
pbs/torque.

I have created a Galaxy tool for debugging the problem -
https://gist.github.com/jmchilton/9548516. If you could run in your
environment and assign it multiple cores it should spit out some
interesting information relevant to debugging the problem. It will
create Galaxy datasets corresponding to the runtime-computed Galaxy
slots, the contents of PBS_NODEFILE which Galaxy attempts to use to
compute Galaxy slots, and the full contents of your worker node
environment. This last one may contain sensitive information - but if
you could post the first two or send them to me directly it would
hopefully help debug the problem.

-John

On Wed, Mar 12, 2014 at 12:37 PM, Geert Vandeweyer
<[hidden email]> wrote:

> How would those statements translate to pbs/torque (pbs_python)
>
> I request resources using the nodes=1:ppn:4 syntax.  The -pe argument is not
> available it seems...
>
> Best,
>
> Geert
>
>
> On 03/12/2014 04:43 PM, Björn Grüning wrote:
>>
>> Hi Geert,
>>
>> you can give every tool a different amount of SLOTS via the job_conf.xml
>> file.
>>
>> https://wiki.galaxyproject.org/Admin/Config/Jobs
>>
>> For example:
>>
>>         <tool
>> id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
>> destination="12cores_24G" />
>>
>>
>> where 12cores_24G is defined with a parallel SGE environment.
>>
>> Cheers,
>> Bjoern
>>
>> Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
>>>
>>> Hi,
>>>
>>> Is there documentation for the proper setup and utilisation of the
>>> \${GALAXY_SLOTS:-4} style options?
>>>
>>> I noticed that tools with this setting, including BWA and GATK run
>>> single threaded, with the following settings respectively:
>>>
>>> BWA  \${GALAXY_SLOTS:-4}"
>>> ===
>>> ps output: python
>>>
>>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
>>> --threads=1 --fileSource=indexed
>>> --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
>>> --do_not_build_index
>>> --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
>>> --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
>>> --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
>>> --genAlignType=paired --params=pre_set --suppressHeader=false
>>>
>>> xml config: --threads="\${GALAXY_SLOTS:-4}"
>>>
>>>
>>> GATK Base Recalibrator \${GALAXY_SLOTS:-8}
>>> ==============
>>> python
>>>
>>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
>>> --max_jvm_heap 6g --stdout
>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
>>> gatk_input -d
>>> /galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
>>> bam_index
>>> gatk_input -p           java -jar
>>> "$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
>>> $GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
>>> ${GALAXY_SLOTS:-8}      --no_standard_covs          -R
>>> "/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
>>> "/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
>>> "ContextCovariate"             -cov "CycleCovariate"     -d
>>> --knownSites:dbsnp,%(file_type)s
>>> /galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
>>> input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
>>> /galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
>>> input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
>>> "NONE" -p          --baq "OFF" --baqGapOpenPenalty
>>> "40.0"                  --defaultBaseQualities "-1"
>>> --validation_strictness "STRICT" --interval_merging "ALL"
>>>
>>> java -Xmx6g -jar
>>> /galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
>>> BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
>>> -R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
>>> ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
>>> --interval_set_rule UNION --downsampling_type NONE --baq OFF
>>> --baqGapOpenPenalty 40.0 --defaultBaseQualities -1
>>> --validation_strictness STRICT --interval_merging ALL -I
>>> /tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
>>> /tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
>>> /tmp/tmp-gatk-fIuncH/input_intervals_0.bed
>>>
>>>
>>> Best,
>>>
>>> Geert
>>>
>>>
>>>
>>>
>
>
> --
>
> Geert Vandeweyer, Ph.D.
> Department of Medical Genetics
> University of Antwerp
> Prins Boudewijnlaan 43
> 2650 Edegem
> Belgium
> Tel: +32 (0)3 275 97 56
> E-mail: [hidden email]
> http://ua.ac.be/cognitivegenetics
> http://www.linkedin.com/in/geertvandeweyer
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: How To properly use GALAXY_SLOTS

Geert Vandeweyer-4
This is strange.

After restarting galaxy to include the debug tool, all galaxy_slots
values are correctly assigned (tested for BWA and GATK as rerun of
original posted jobs)....

If you still need the output, let me know.

Best,

Geert

On 03/14/2014 03:25 PM, John Chilton wrote:

> I am not sure I understand what you mean, but if you are setting the
> job to run with a ppn of greater than one but GALAXY_SLOTS is being
> evaluated as 1 then you have likely uncovered a bug - maybe in the
> runner, your job configuration, or the GALAXY_SLOTS logic for
> pbs/torque.
>
> I have created a Galaxy tool for debugging the problem -
> https://gist.github.com/jmchilton/9548516. If you could run in your
> environment and assign it multiple cores it should spit out some
> interesting information relevant to debugging the problem. It will
> create Galaxy datasets corresponding to the runtime-computed Galaxy
> slots, the contents of PBS_NODEFILE which Galaxy attempts to use to
> compute Galaxy slots, and the full contents of your worker node
> environment. This last one may contain sensitive information - but if
> you could post the first two or send them to me directly it would
> hopefully help debug the problem.
>
> -John
>
> On Wed, Mar 12, 2014 at 12:37 PM, Geert Vandeweyer
> <[hidden email]> wrote:
>> How would those statements translate to pbs/torque (pbs_python)
>>
>> I request resources using the nodes=1:ppn:4 syntax.  The -pe argument is not
>> available it seems...
>>
>> Best,
>>
>> Geert
>>
>>
>> On 03/12/2014 04:43 PM, Björn Grüning wrote:
>>> Hi Geert,
>>>
>>> you can give every tool a different amount of SLOTS via the job_conf.xml
>>> file.
>>>
>>> https://wiki.galaxyproject.org/Admin/Config/Jobs
>>>
>>> For example:
>>>
>>>          <tool
>>> id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator"
>>> destination="12cores_24G" />
>>>
>>>
>>> where 12cores_24G is defined with a parallel SGE environment.
>>>
>>> Cheers,
>>> Bjoern
>>>
>>> Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
>>>> Hi,
>>>>
>>>> Is there documentation for the proper setup and utilisation of the
>>>> \${GALAXY_SLOTS:-4} style options?
>>>>
>>>> I noticed that tools with this setting, including BWA and GATK run
>>>> single threaded, with the following settings respectively:
>>>>
>>>> BWA  \${GALAXY_SLOTS:-4}"
>>>> ===
>>>> ps output: python
>>>>
>>>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py
>>>> --threads=1 --fileSource=indexed
>>>> --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta
>>>> --do_not_build_index
>>>> --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat
>>>> --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat
>>>> --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat
>>>> --genAlignType=paired --params=pre_set --suppressHeader=false
>>>>
>>>> xml config: --threads="\${GALAXY_SLOTS:-4}"
>>>>
>>>>
>>>> GATK Base Recalibrator \${GALAXY_SLOTS:-8}
>>>> ==============
>>>> python
>>>>
>>>> /galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py
>>>> --max_jvm_heap 6g --stdout
>>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I
>>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam
>>>> gatk_input -d
>>>> /galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat
>>>> bam_index
>>>> gatk_input -p           java -jar
>>>> "$GATK2_PATH/GenomeAnalysisTK.jar"        -T "BaseRecalibrator"
>>>> $GATK2_SITE_OPTIONS      --num_cpu_threads_per_data_thread
>>>> ${GALAXY_SLOTS:-8}      --no_standard_covs          -R
>>>> "/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out
>>>> "/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov
>>>> "ContextCovariate"             -cov "CycleCovariate"     -d
>>>> --knownSites:dbsnp,%(file_type)s
>>>> /galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf
>>>> input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals
>>>> /galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed
>>>> input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type
>>>> "NONE" -p          --baq "OFF" --baqGapOpenPenalty
>>>> "40.0"                  --defaultBaseQualities "-1"
>>>> --validation_strictness "STRICT" --interval_merging "ALL"
>>>>
>>>> java -Xmx6g -jar
>>>> /galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T
>>>> BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs
>>>> -R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out
>>>> /galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov
>>>> ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT
>>>> --interval_set_rule UNION --downsampling_type NONE --baq OFF
>>>> --baqGapOpenPenalty 40.0 --defaultBaseQualities -1
>>>> --validation_strictness STRICT --interval_merging ALL -I
>>>> /tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf
>>>> /tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals
>>>> /tmp/tmp-gatk-fIuncH/input_intervals_0.bed
>>>>
>>>>
>>>> Best,
>>>>
>>>> Geert
>>>>
>>>>
>>>>
>>>>
>>
>> --
>>
>> Geert Vandeweyer, Ph.D.
>> Department of Medical Genetics
>> University of Antwerp
>> Prins Boudewijnlaan 43
>> 2650 Edegem
>> Belgium
>> Tel: +32 (0)3 275 97 56
>> E-mail: [hidden email]
>> http://ua.ac.be/cognitivegenetics
>> http://www.linkedin.com/in/geertvandeweyer
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/


--

Geert Vandeweyer, Ph.D.
Department of Medical Genetics
University of Antwerp
Prins Boudewijnlaan 43
2650 Edegem
Belgium
Tel: +32 (0)3 275 97 56
E-mail: [hidden email]
http://ua.ac.be/cognitivegenetics
http://www.linkedin.com/in/geertvandeweyer

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/