Planemo 0.19.0 testing tools that produce dataset collections

classic Classic list List threaded Threaded
3 messages Options
| Threaded
Open this post in threaded view
|

Planemo 0.19.0 testing tools that produce dataset collections

Von Kuster, Greg
Hello Galaxy devs,

I’m developing a tool that uses a multi-select list to accept any number of input datasets.  The tool works fine when executing it within Galaxy, but when running it through planemo testing, it never passes, so I’m wondering if there are known issues with planemo testing tools that produce dataset collections.

I’ve spent some time adding debugging log statements into the Galaxy framework for planemo test runs, but haven’t uncovered anything obvious.  So if there are no known issues with planemo testing dataset_collections, I’m wondering if my tool config syntax is incorrect.

Does anyone see anything obvious?  Any insight is much appreciated!

Here is the command line.  Notice that I do not define any outputs since I am using a dataset collection.  Is this causing a problem for planemo?

    <command>
        python $__tool_directory__/genetrack.py
        --input_format $input_format_cond.input_format
        #if str($input_format_cond.input_format) == "genetracktool":
            #for $i in $input_format_cond.input_genetracktool:
                 --input "${i}" "${i.hid}" "${i.name}"
            #end for
        #elif str($input_format_cond.input_format) == "gff":
            #for $i in $input_format_cond.input_gff:
                 --input "${i}" "${i.hid}" "${i.name}"
            #end for
        #end if
        --sigma $sigma
        --exclusion $exclusion
        --up_width $up_width
        --down_width $down_width
        --filter $filter
        --chromosome $chromosome
        --chunk_size $chunk_size
        --output_format $output_format
    </command>

The input  dataset parameter looks like this:

<param  name="input_gff" type="data" format="gff" multiple="True" label="Predict peaks on" />

The tool produces a list of outputs, so I’ve defined a collection like this:

    <outputs>
        <collection name="genetrack_output" type="list" label="Genetrack results on ${on_string}">
            <discover_datasets pattern="(?P&lt;designation&gt;.*)" directory="output" ext="gff" visible="false" />
        </collection>
    </outputs>

I’m using planemo, version 0.19.0.

Here is one of the tools’ functional test definitions.

        <test>
            <param name="input" value="genetrack_input_unsorted4.gff" ftype="gff" />
            <param name="input_format" value="gff" />
            <param name="sigma" value="5" />
            <param name="exclusion" value="20" />
            <param name="up_width" value="10" />
            <param name="down_width" value="10" />
            <param name="filter" value="3" />
            <param name="chromosome" value="all" />
            <param name="chunk_size" value="10" />
            <param name="output_format" value="gff" />
            <output_collection name="genetrack_output" type="list">
                <element name="s5e20u10d10F3" file="genetrack_output4.gff" ftype="gff" />
            </output_collection>
        </test>

Here is the log from running the test with planemo.

Genetrack ( genetrack ) > Test-3 ... requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,292 Starting new HTTP connection (1): localhost
galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,295 Session authenticated using Galaxy master api key
requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,303 "GET /api/users?key=test_key HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,305 Starting new HTTP connection (1): localhost
galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,308 Session authenticated using Galaxy master api key
requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,326 "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,328 Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,418 "POST /api/histories HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,421 Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,475 "POST /api/tools HTTP/1.1" 400 None
ERROR

======================================================================
ERROR: Genetrack ( genetrack ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 289, in test_tool
    self.do_it( td )
  File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 58, in do_it
    raise e
RunToolException: Error creating a job for these tool inputs - Unknown error occurred while processing request.
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
requests.packages.urllib3.connectionpool: DEBUG: "GET /api/users?key=test_key HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/histories HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 400 None
--------------------- >> end captured logging << ——————————


Thanks very much,

Greg Von Kuster


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

R: Planemo 0.19.0 testing tools that produce dataset collections

Nicola Soranzo-2
Hi Greg,
in the inputs the param has name "input_gff", while in the test you are using name "input".

Cheers,
Nicola

---- Von Kuster, Greg ha scritto ----

>Hello Galaxy devs,
>
>I’m developing a tool that uses a multi-select list to accept any number of input datasets.  The tool works fine when executing it within Galaxy, but when running it through planemo testing, it never passes, so I’m wondering if there are known issues with planemo testing tools that produce dataset collections.
>
>I’ve spent some time adding debugging log statements into the Galaxy framework for planemo test runs, but haven’t uncovered anything obvious.  So if there are no known issues with planemo testing dataset_collections, I’m wondering if my tool config syntax is incorrect.
>
>Does anyone see anything obvious?  Any insight is much appreciated!
>
>Here is the command line.  Notice that I do not define any outputs since I am using a dataset collection.  Is this causing a problem for planemo?
>
>    <command>
>        python $__tool_directory__/genetrack.py
>        --input_format $input_format_cond.input_format
>        #if str($input_format_cond.input_format) == "genetracktool":
>            #for $i in $input_format_cond.input_genetracktool:
>                 --input "${i}" "${i.hid}" "${i.name}"
>            #end for
>        #elif str($input_format_cond.input_format) == "gff":
>            #for $i in $input_format_cond.input_gff:
>                 --input "${i}" "${i.hid}" "${i.name}"
>            #end for
>        #end if
>        --sigma $sigma
>        --exclusion $exclusion
>        --up_width $up_width
>        --down_width $down_width
>        --filter $filter
>        --chromosome $chromosome
>        --chunk_size $chunk_size
>        --output_format $output_format
>    </command>
>
>The input  dataset parameter looks like this:
>
><param  name="input_gff" type="data" format="gff" multiple="True" label="Predict peaks on" />
>
>The tool produces a list of outputs, so I’ve defined a collection like this:
>
>    <outputs>
>        <collection name="genetrack_output" type="list" label="Genetrack results on ${on_string}">
>            <discover_datasets pattern="(?P&lt;designation&gt;.*)" directory="output" ext="gff" visible="false" />
>        </collection>
>    </outputs>
>
>I’m using planemo, version 0.19.0.
>
>Here is one of the tools’ functional test definitions.
>
>        <test>
>            <param name="input" value="genetrack_input_unsorted4.gff" ftype="gff" />
>            <param name="input_format" value="gff" />
>            <param name="sigma" value="5" />
>            <param name="exclusion" value="20" />
>            <param name="up_width" value="10" />
>            <param name="down_width" value="10" />
>            <param name="filter" value="3" />
>            <param name="chromosome" value="all" />
>            <param name="chunk_size" value="10" />
>            <param name="output_format" value="gff" />
>            <output_collection name="genetrack_output" type="list">
>                <element name="s5e20u10d10F3" file="genetrack_output4.gff" ftype="gff" />
>            </output_collection>
>        </test>
>
>Here is the log from running the test with planemo.
>
>Genetrack ( genetrack ) > Test-3 ... requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,292 Starting new HTTP connection (1): localhost
>galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,295 Session authenticated using Galaxy master api key
>requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,303 "GET /api/users?key=test_key HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,305 Starting new HTTP connection (1): localhost
>galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,308 Session authenticated using Galaxy master api key
>requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,326 "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,328 Starting new HTTP connection (1): localhost
>requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,418 "POST /api/histories HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,421 Starting new HTTP connection (1): localhost
>requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,475 "POST /api/tools HTTP/1.1" 400 None
>ERROR
>
>======================================================================
>ERROR: Genetrack ( genetrack ) > Test-1
>----------------------------------------------------------------------
>Traceback (most recent call last):
>  File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 289, in test_tool
>    self.do_it( td )
>  File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 58, in do_it
>    raise e
>RunToolException: Error creating a job for these tool inputs - Unknown error occurred while processing request.
>-------------------- >> begin captured logging << --------------------
>requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>requests.packages.urllib3.connectionpool: DEBUG: "GET /api/users?key=test_key HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>requests.packages.urllib3.connectionpool: DEBUG: "POST /api/histories HTTP/1.1" 200 None
>requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 400 None
>--------------------- >> end captured logging << ——————————
>
>
>Thanks very much,
>
>Greg Von Kuster
>
>
>___________________________________________________________
>Please keep all replies on the list by using "reply all"
>in your mail client.  To manage your subscriptions to this
>and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
>To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Planemo 0.19.0 testing tools that produce dataset collections

Von Kuster, Greg
Nicola, thanks a bunch!  Your eyes are phenomenal!

Greg

> On Nov 4, 2015, at 8:09 PM, Nicola Soranzo <[hidden email]> wrote:
>
> Hi Greg,
> in the inputs the param has name "input_gff", while in the test you are using name "input".
>
> Cheers,
> Nicola
>
> ---- Von Kuster, Greg ha scritto ----
>
>> Hello Galaxy devs,
>>
>> I’m developing a tool that uses a multi-select list to accept any number of input datasets.  The tool works fine when executing it within Galaxy, but when running it through planemo testing, it never passes, so I’m wondering if there are known issues with planemo testing tools that produce dataset collections.
>>
>> I’ve spent some time adding debugging log statements into the Galaxy framework for planemo test runs, but haven’t uncovered anything obvious.  So if there are no known issues with planemo testing dataset_collections, I’m wondering if my tool config syntax is incorrect.
>>
>> Does anyone see anything obvious?  Any insight is much appreciated!
>>
>> Here is the command line.  Notice that I do not define any outputs since I am using a dataset collection.  Is this causing a problem for planemo?
>>
>>   <command>
>>       python $__tool_directory__/genetrack.py
>>       --input_format $input_format_cond.input_format
>>       #if str($input_format_cond.input_format) == "genetracktool":
>>           #for $i in $input_format_cond.input_genetracktool:
>>                --input "${i}" "${i.hid}" "${i.name}"
>>           #end for
>>       #elif str($input_format_cond.input_format) == "gff":
>>           #for $i in $input_format_cond.input_gff:
>>                --input "${i}" "${i.hid}" "${i.name}"
>>           #end for
>>       #end if
>>       --sigma $sigma
>>       --exclusion $exclusion
>>       --up_width $up_width
>>       --down_width $down_width
>>       --filter $filter
>>       --chromosome $chromosome
>>       --chunk_size $chunk_size
>>       --output_format $output_format
>>   </command>
>>
>> The input  dataset parameter looks like this:
>>
>> <param  name="input_gff" type="data" format="gff" multiple="True" label="Predict peaks on" />
>>
>> The tool produces a list of outputs, so I’ve defined a collection like this:
>>
>>   <outputs>
>>       <collection name="genetrack_output" type="list" label="Genetrack results on ${on_string}">
>>           <discover_datasets pattern="(?P&lt;designation&gt;.*)" directory="output" ext="gff" visible="false" />
>>       </collection>
>>   </outputs>
>>
>> I’m using planemo, version 0.19.0.
>>
>> Here is one of the tools’ functional test definitions.
>>
>>       <test>
>>           <param name="input" value="genetrack_input_unsorted4.gff" ftype="gff" />
>>           <param name="input_format" value="gff" />
>>           <param name="sigma" value="5" />
>>           <param name="exclusion" value="20" />
>>           <param name="up_width" value="10" />
>>           <param name="down_width" value="10" />
>>           <param name="filter" value="3" />
>>           <param name="chromosome" value="all" />
>>           <param name="chunk_size" value="10" />
>>           <param name="output_format" value="gff" />
>>           <output_collection name="genetrack_output" type="list">
>>               <element name="s5e20u10d10F3" file="genetrack_output4.gff" ftype="gff" />
>>           </output_collection>
>>       </test>
>>
>> Here is the log from running the test with planemo.
>>
>> Genetrack ( genetrack ) > Test-3 ... requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,292 Starting new HTTP connection (1): localhost
>> galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,295 Session authenticated using Galaxy master api key
>> requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,303 "GET /api/users?key=test_key HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,305 Starting new HTTP connection (1): localhost
>> galaxy.web.framework.webapp INFO 2015-11-04 15:22:03,308 Session authenticated using Galaxy master api key
>> requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,326 "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,328 Starting new HTTP connection (1): localhost
>> requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,418 "POST /api/histories HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool INFO 2015-11-04 15:22:03,421 Starting new HTTP connection (1): localhost
>> requests.packages.urllib3.connectionpool DEBUG 2015-11-04 15:22:03,475 "POST /api/tools HTTP/1.1" 400 None
>> ERROR
>>
>> ======================================================================
>> ERROR: Genetrack ( genetrack ) > Test-1
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 289, in test_tool
>>   self.do_it( td )
>> File "/Users/gvk/work/git_workspace/galaxy/test/functional/test_toolbox.py", line 58, in do_it
>>   raise e
>> RunToolException: Error creating a job for these tool inputs - Unknown error occurred while processing request.
>> -------------------- >> begin captured logging << --------------------
>> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>> galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>> requests.packages.urllib3.connectionpool: DEBUG: "GET /api/users?key=test_key HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>> galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>> galaxy.web.framework.webapp: INFO: Session authenticated using Galaxy master api key
>> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/users/2891970512fa2d5a/api_key HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/histories HTTP/1.1" 200 None
>> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
>> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 400 None
>> --------------------- >> end captured logging << ——————————
>>
>>
>> Thanks very much,
>>
>> Greg Von Kuster
>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/