Unclear API and Twill test failure for undefined datatype

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Unclear API and Twill test failure for undefined datatype

Peter Cock
Hi all,

I am using TravisCI for testing my Galaxy Tools developed
on GitHub, see:
http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

Currently this uses a hard coded datatypes_conf.xml file consisting
of the local datatypes in use (e.g. BLAST databases) plus a cut down
list those core dataypes also being used (e.g. FASTA).

The following test run failed because one of the newly added tests
used "sam" format, which was not defined:

https://travis-ci.org/peterjc/pico_galaxy/builds/23041174

I've added the SAM definition now, the reason for this email
is to suggest clearer error messages from the test framework
for undefined dataypes.

Currently Twill (default) said:

======================================================================
ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_samtools_depad)
BAM mapping statistics ( samtools_depad ) > Test-2
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
line 106, in test_tool
    self.do_it( td )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
line 29, in do_it
    stage_data_in_history( galaxy_interactor, testdef.test_data(),
test_history, shed_tool_id )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
line 30, in stage_data_in_history
    upload_waits.append( galaxy_interactor.stage_data_async(
test_data, history, shed_tool_id ) )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
line 336, in stage_data_async
    wait=(not async) )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py",
line 209, in upload_file
    self.refresh_form( "file_type", ftype )  # Refresh, to support
composite files
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py",
line 1369, in refresh_form
    tc.fv( f.name, control.name, value )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/commands.py",
line 450, in formvalue
    set_form_control_value(control, value)
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/utils.py",
line 190, in set_form_control_value
    raise ClientForm.ItemNotFoundError('cannot find value/label "%s"
in list control' % (val,))
ItemNotFoundError: cannot find value/label "sam" in list control



Similarly unclear, the API test framework said:

galaxy.web.framework: ERROR: Uncaught exception in exposed API method:
Traceback (most recent call last):
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/web/framework/__init__.py",
line 199, in decorator
    rval = func( self, trans, *args, **kwargs)
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/webapps/galaxy/api/tools.py",
line 130, in create
    template, vars = tool.handle_input( trans, incoming,
history=target_history, process_state=process_state, source="json" )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
line 1921, in handle_input
    errors, params = self.__check_param_values( trans, incoming,
state, old_errors, process_state, history=history, source=source )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
line 2029, in __check_param_values
    errors = self.populate_state( trans, inputs, state.inputs,
incoming, history, source=source )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
line 2136, in populate_state
    current_case = input.get_current_case( value, trans )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/parameters/grouping.py",
line 462, in get_current_case
    raise Exception( "No case matched value:", self.name, str_value )
Exception: ('No case matched value:', 'files_metadata', u'sam')
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools
HTTP/1.1" 500 None


In both cases, I would prefer something like a ValueError
explicitly saying the problem is an undefined dataype
(in this case, "sam" was not defined).

This will be most useful when dealing with 3rd party tools
using 3rd part datatype definitions.

Thanks,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Unclear API and Twill test failure for undefined datatype

John Chilton-4
Thanks for the report Peter.

That API error message is really very unclear - I think this
https://bitbucket.org/galaxy/galaxy-central/commits/7c6fce1fd9e7f7adb4811cd92a56cdff05ea748d
should make it slightly less unclear what is going on and I have
created a Trello card here https://trello.com/c/iCdW0pt9 laying out
how to make the error message even less bad.

-John

On Tue, Apr 15, 2014 at 10:00 AM, Peter Cock <[hidden email]> wrote:

> Hi all,
>
> I am using TravisCI for testing my Galaxy Tools developed
> on GitHub, see:
> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>
> Currently this uses a hard coded datatypes_conf.xml file consisting
> of the local datatypes in use (e.g. BLAST databases) plus a cut down
> list those core dataypes also being used (e.g. FASTA).
>
> The following test run failed because one of the newly added tests
> used "sam" format, which was not defined:
>
> https://travis-ci.org/peterjc/pico_galaxy/builds/23041174
>
> I've added the SAM definition now, the reason for this email
> is to suggest clearer error messages from the test framework
> for undefined dataypes.
>
> Currently Twill (default) said:
>
> ======================================================================
> ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_samtools_depad)
> BAM mapping statistics ( samtools_depad ) > Test-2
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
> line 106, in test_tool
>     self.do_it( td )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
> line 29, in do_it
>     stage_data_in_history( galaxy_interactor, testdef.test_data(),
> test_history, shed_tool_id )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
> line 30, in stage_data_in_history
>     upload_waits.append( galaxy_interactor.stage_data_async(
> test_data, history, shed_tool_id ) )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
> line 336, in stage_data_async
>     wait=(not async) )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py",
> line 209, in upload_file
>     self.refresh_form( "file_type", ftype )  # Refresh, to support
> composite files
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/twilltestcase.py",
> line 1369, in refresh_form
>     tc.fv( f.name, control.name, value )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/commands.py",
> line 450, in formvalue
>     set_form_control_value(control, value)
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/twill-0.9-py2.7.egg/twill/utils.py",
> line 190, in set_form_control_value
>     raise ClientForm.ItemNotFoundError('cannot find value/label "%s"
> in list control' % (val,))
> ItemNotFoundError: cannot find value/label "sam" in list control
>
>
>
> Similarly unclear, the API test framework said:
>
> galaxy.web.framework: ERROR: Uncaught exception in exposed API method:
> Traceback (most recent call last):
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/web/framework/__init__.py",
> line 199, in decorator
>     rval = func( self, trans, *args, **kwargs)
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/webapps/galaxy/api/tools.py",
> line 130, in create
>     template, vars = tool.handle_input( trans, incoming,
> history=target_history, process_state=process_state, source="json" )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
> line 1921, in handle_input
>     errors, params = self.__check_param_values( trans, incoming,
> state, old_errors, process_state, history=history, source=source )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
> line 2029, in __check_param_values
>     errors = self.populate_state( trans, inputs, state.inputs,
> incoming, history, source=source )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/__init__.py",
> line 2136, in populate_state
>     current_case = input.get_current_case( value, trans )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/lib/galaxy/tools/parameters/grouping.py",
> line 462, in get_current_case
>     raise Exception( "No case matched value:", self.name, str_value )
> Exception: ('No case matched value:', 'files_metadata', u'sam')
> requests.packages.urllib3.connectionpool: DEBUG: "POST /api/tools
> HTTP/1.1" 500 None
>
>
> In both cases, I would prefer something like a ValueError
> explicitly saying the problem is an undefined dataype
> (in this case, "sam" was not defined).
>
> This will be most useful when dealing with 3rd party tools
> using 3rd part datatype definitions.
>
> Thanks,
>
> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/