Tool Testing: field inputs that occur inside repeat tag

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Tool Testing: field inputs that occur inside repeat tag

Dooley, Damion
Helop aI've been testing (using planemo) some tools scheduled for toolshed publication.  My first test works, but Galaxy reports that the second one fails with:

"Parameter %s requires a value, but has no legal values defined" % self.name
AssertionError: Parameter filter_column requires a value, but has no legal values defined
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 500 None

I presume its because of extra stuff needed to name variables have to be referred-to when they are inside a repeat section?  Or is it just something pecular to do with data_table_sources?

        <test><!-- Equivalent to output of old bccdc BLASTn XML to Tabular -->
                <param name="blastxml_file" value="blast_reporting_1.blastxml"/>
                <output name="tabular_file" file="blast_reporting_1a.tabular"/>
                <param name="out_format" value="std"/>
                <param name="column_labels" value="" />
                <param name="drop_redundant_hits" value="true"/>
                                       
        </test>
        <test>
                <param name="blastxml_file" value="blast_reporting_1.blastxml"/>
                <output name="tabular_file" file="blast_reporting_1b.tabular"/>
                <param name="out_format" value="std"/>
                <param name="column_labels" value="" />
                <param name="drop_redundant_hits" value="true"/>
               
                <param name="filter_column" value="pident"/>
                <param name="filter_comparison" value="gte"/>
                <param name="filter_value" value="97"/>
               
        </test>

The form repeat area:

        <repeat name="filter_num" title="Numeric Filter" min="0" max="4">

                <param name="filter_column" type="select" label="Col">
                        <options from_data_table="blast_reporting_fields">
                                <filter type="static_value" value="numeric" column="type" />
                                <filter type="sort_by" column="name"/>
                        </options>
                </param>
                       
                <repeat name="constraint" title="Constraint" min="1" max="3">
                        <param name="filter_comparison" type="select" label="Comparison">
                                <option value="gte">&gt;= </option>
                                <option value="gt">&gt;</option>
                                <option value="lt">&lt; </option>
                                <option value="lte">&lt;= </option>
                                <option value="==">equal to </option>
                                <option value="!=">not equal to </option>
                        </param>
                       
                        <param name="filter_value" type="text" value="" label="Value">
                                <validator type="regex" message="Please input a number">([0-9]+|[0-9]*\.[0-9]+)</validator>
                        </param>
                       
                </repeat>
        </repeat>

The working command section that sends the filter field info is:

        #for $i, $my_repeat in enumerate( $filter_num )
                $my_repeat.filter_column:
                #for $i, $my_repeat2 in enumerate( $my_repeat.constraint )
                        $my_repeat2.filter_comparison $my_repeat2.filter_value,
                #end for
                ;
        #end for




Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

FW: Tool Testing: field inputs that occur inside repeat tag

Dooley, Damion
Slightly garbled there ... meant to say

Help appreciated once again,

Damion

Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag

Peter Cock
In reply to this post by Dooley, Damion
On Thu, Feb 26, 2015 at 11:53 PM, Dooley, Damion <[hidden email]> wrote:
> Helop aI've been testing (using planemo) some tools scheduled for toolshed publication.  My first test works, but Galaxy reports that the second one fails with:
>
> "Parameter %s requires a value, but has no legal values defined" % self.name
> AssertionError: Parameter filter_column requires a value, but has no legal values defined
> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools HTTP/1.1" 500 None

You didn't say if it was test one, or test two which gave this error.

Test two appears to use a single repeat block, with the values
given. It uses the "old style" ambiguous syntax which only
really worked with a single repeat entry. So that ought to be OK.

Test two is not trying to use a repeat block (i.e. zero repeats).
I think this is the one that fails as the test framework may be
wrongly assuming one repeat? That is my guess anyway.

What I suggest you try is the "new style" explicit repeat
parameter naming using pipes. e.g.

https://github.com/peterjc/pico_galaxy/blob/master/tools/clc_assembly_cell/clc_assembler.xml

It looks like the wiki hasn't got an example of this yet,
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax

John Chilton can probably advise, and/or point us at some
of the Galaxy functional-tests where he would have made
some pathological examples. Tests using zero repeats
ought to be possible...

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag - semi resolved.

Dooley, Damion
First, hey Peter, thanks for input!  I'd seen the explicit piping before, couldn't remember its format though; I will switch to it.

Well, after a morning of testing, I see the test system is actually finding the repeat section variables.  It looks like its error "Parameter filter_column requires a value, but has no legal values defined" points to the fact that it can't process any option list having <options from_data_table="[whatever table we got]"> .  

I'd assumed Planemo in its test setup would install the tool's data tables, but now I see it is still on John's enhancement list. So I'll table those tests until that's a go.

I guess manually setting up the data tables in galaxy won't solve that for planemo?

*************************

I'm still stuck trying to get tool test directly in a fresh copy of Galaxy.  I edit config/tool_conf.xml.sample to have

  <section id="testing" name="TESTING">
    <tool file="TESTING/blast_reporting/blast_reporting.xml" />
  </section>

And I run "tests.sh -id blast_reporting" (with Galaxy as owner of tool folder):

======================================================================
ERROR: Failure: ValueError (No such test TestForTool_blast_reporting)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/galaxy/production2/galaxy-dist/eggs/nose-0.11.1-py2.6.egg/nose/failure.py", line 39, in runTest
    raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_blast_reporting

Though planemo has no problem linting and testing it.  Arg!  

*************************

Thanks,

Damion


d.

p.s. I did mention first test works (in p1) :) .

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag - semi resolved.

Peter Cock
On Fri, Feb 27, 2015 at 8:58 PM, Dooley, Damion <[hidden email]> wrote:

> First, hey Peter, thanks for input!  I'd seen the explicit piping before,
> couldn't remember its format though; I will switch to it.
>
> Well, after a morning of testing, I see the test system is actually
> finding the repeat section variables.  It looks like its error
> "Parameter filter_column requires a value, but has no legal
> values defined" points to the fact that it can't process any
> option list having <options from_data_table="[whatever table we got]"> .
>
> I'd assumed Planemo in its test setup would install the tool's data
> tables, but now I see it is still on John's enhancement list. So I'll
> table those tests until that's a go.
>
> I guess manually setting up the data tables in galaxy won't solve
> that for planemo?
>
> *************************
>
> I'm still stuck trying to get tool test directly in a fresh copy of
> Galaxy.  I edit config/tool_conf.xml.sample to have
>
>   <section id="testing" name="TESTING">
>     <tool file="TESTING/blast_reporting/blast_reporting.xml" />
>   </section>
>
> And I run "tests.sh -id blast_reporting" (with Galaxy as owner
> of tool folder):

While the tests used to use tool_conf.xml.sample last year
(or even longer ago?) this was switched to using the main
tool_conf.xml instead. This is much easier for testing what
you actually have installed.

If I've guessed wrong, watch the run_tests.sh output to see
which tools are loaded - and make sure your tool loads OK
(and for example doesn't fail with an XML error or something).

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag - resolved

Dooley, Damion
Yay, with your tip, galaxy direct tests now work with tool_conf.xml AND data table in place!

Galaxy DOC person: please update https://wiki.galaxyproject.org/Admin/RunningTests to state that tool_conf.xml.sample is NOT used for testing!  It is stated in the run_tests.sh help info half way down the page.

Tool wasn't loading on new site because data table it referenced wasn't set up.

I'll note for galaxy devs that the WARNING is actually fatal i.e. that missing data table caused some kind of weird error:

galaxy.tools.toolbox.base INFO 2015-02-27 13:48:48,190 Parsing the tool configuration ./config/tool_conf.xml
galaxy.tools.parameters.dynamic_options WARNING 2015-02-27 13:48:48,199 Data table named 'blast_reporting_fields' is required by tool but not configured
galaxy.tools.toolbox.base ERROR 2015-02-27 13:48:48,200 Error reading tool from path: TESTING/blast_reporting/blast_reporting.xml
Traceback (most recent call last):
  File "...galaxy-dist/lib/galaxy/tools/toolbox/base.py", line 670, in _load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
...
  File ".../galaxy-dist/lib/galaxy/tools/parameters/dynamic_options.py", line 608, in column_spec_to_index
    return int( column_spec )
ValueError: invalid literal for int() with base 10: 'type'

Thanks again, Peter!

Damion

Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control
655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada
________________________________________
From: Peter Cock [[hidden email]]
Sent: Friday, February 27, 2015 1:39 PM
To: Dooley, Damion
Cc: [hidden email]
Subject: Re: Tool Testing: field inputs that occur inside repeat tag - semi resolved.
...
> And I run "tests.sh -id blast_reporting" (with Galaxy as owner
> of tool folder):

While the tests used to use tool_conf.xml.sample last year
(or even longer ago?) this was switched to using the main
tool_conf.xml instead. This is much easier for testing what
you actually have installed.

If I've guessed wrong, watch the run_tests.sh output to see
which tools are loaded - and make sure your tool loads OK
(and for example doesn't fail with an XML error or something).

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag - resolved

Peter Cock
On Fri, Feb 27, 2015 at 10:36 PM, Dooley, Damion <[hidden email]> wrote:
> Yay, with your tip, galaxy direct tests now work with tool_conf.xml AND data table in place!
>
> Galaxy DOC person: please update https://wiki.galaxyproject.org/Admin/RunningTests
> to state that tool_conf.xml.sample is NOT used for testing!  It is stated in the
> run_tests.sh help info half way down the page.

Done - well spotted. I updated the run_test.sh help output that line was from.

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Tool Testing: field inputs that occur inside repeat tag - resolved

John Chilton-4
Glad you got it working!

Tests that use tool data tables should work in planemo - however you
will need to configure the data table for the tool though. Here is an
example of the how to setup a test data for a tool data table:

https://github.com/jmchilton/picard/commit/4df8974384081ee1bb0f97e1bb8d7f935ba09d73

planemo will detect the tool data table file automatically if it is
sitting new to the tool or an explicit file can be passed in with
(--tool_data_table)
http://planemo.readthedocs.org/en/latest/commands.html#test-command.

-John

On Sat, Feb 28, 2015 at 7:15 AM, Peter Cock <[hidden email]> wrote:

> On Fri, Feb 27, 2015 at 10:36 PM, Dooley, Damion <[hidden email]> wrote:
>> Yay, with your tip, galaxy direct tests now work with tool_conf.xml AND data table in place!
>>
>> Galaxy DOC person: please update https://wiki.galaxyproject.org/Admin/RunningTests
>> to state that tool_conf.xml.sample is NOT used for testing!  It is stated in the
>> run_tests.sh help info half way down the page.
>
> Done - well spotted. I updated the run_test.sh help output that line was from.
>
> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/