Functional Tests and ftype

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Functional Tests and ftype

saketkc
I happened to copy Peter's .travis.yml for running tests for some of
my packages. However the process has not been smooth at all.
See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908

AssertionError: Attempting to set field 'input' to value
'['condel_input.tabular']' in form 'tool_form' threw exception: cannot
find value/label "condel_input.tabular" in list control


I am already passing it the correct ftype :
https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22


Is there something I am missing?


Saket
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

Peter Cock
On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
> I happened to copy Peter's .travis.yml for running tests for some of
> my packages.

Background information:
http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html

> However the process has not been smooth at all.
> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>
> AssertionError: Attempting to set field 'input' to value
> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
> find value/label "condel_input.tabular" in list control
>
> I am already passing it the correct ftype :
> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>
> Is there something I am missing?

This is similar to (but different) to the error message I recently
reported as being unclear when a datatype was not defined:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html

In your case, this is an input file which seems to be missing,
test-data/condel_input.tabular - but I see the line copying the
tool's test-data into the main Galaxy test-data folder is there
in the .travis.yml file.

I am puzzled too.

The key question I would ask you is: do your tests work locally?
If so, are they installed via a (local) Tool Shed, or manually along
the same lines of the attempted TravisCI tool installation?

Regards,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:

> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>> I happened to copy Peter's .travis.yml for running tests for some of
>> my packages.
>
> Background information:
> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>
>> However the process has not been smooth at all.
>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>
>> AssertionError: Attempting to set field 'input' to value
>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>> find value/label "condel_input.tabular" in list control
>>
>> I am already passing it the correct ftype :
>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>
>> Is there something I am missing?
>
> This is similar to (but different) to the error message I recently
> reported as being unclear when a datatype was not defined:
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>
> In your case, this is an input file which seems to be missing,
> test-data/condel_input.tabular - but I see the line copying the
> tool's test-data into the main Galaxy test-data folder is there
> in the .travis.yml file.
>
> I am puzzled too.
>
> The key question I would ask you is: do your tests work locally?
> If so, are they installed via a (local) Tool Shed, or manually along
> the same lines of the attempted TravisCI tool installation?
>

Unfortunately No.

TwillAssertionError: code is 500 != 200

I am running Galaxy under virtualenv. Unfortunately I get the same
error irrespective of whther my input/output files exits inside
test-data/ directory. I forgot to link them to test-data path earlier,
but I get the same error even otherwise.


> Regards,
>
> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

John Chilton-4
I don't see anywhere in your Travis YAML file where you are modifying
Galaxy's datatypes (like Peter does for instance on these lines
https://github.com/peterjc/pico_galaxy/blob/master/.travis.yml#L166).
If those datatypes are not defined you cannot upload a tool with a
specific type. Am I missing something?

-John

On Thu, Apr 17, 2014 at 7:33 AM, Saket Choudhary <[hidden email]> wrote:

> On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>>> I happened to copy Peter's .travis.yml for running tests for some of
>>> my packages.
>>
>> Background information:
>> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>
>>> However the process has not been smooth at all.
>>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>
>>> AssertionError: Attempting to set field 'input' to value
>>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>> find value/label "condel_input.tabular" in list control
>>>
>>> I am already passing it the correct ftype :
>>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>
>>> Is there something I am missing?
>>
>> This is similar to (but different) to the error message I recently
>> reported as being unclear when a datatype was not defined:
>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>
>> In your case, this is an input file which seems to be missing,
>> test-data/condel_input.tabular - but I see the line copying the
>> tool's test-data into the main Galaxy test-data folder is there
>> in the .travis.yml file.
>>
>> I am puzzled too.
>>
>> The key question I would ask you is: do your tests work locally?
>> If so, are they installed via a (local) Tool Shed, or manually along
>> the same lines of the attempted TravisCI tool installation?
>>
>
> Unfortunately No.
>
> TwillAssertionError: code is 500 != 200
>
> I am running Galaxy under virtualenv. Unfortunately I get the same
> error irrespective of whther my input/output files exits inside
> test-data/ directory. I forgot to link them to test-data path earlier,
> but I get the same error even otherwise.
>
>
>> Regards,
>>
>> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
In reply to this post by saketkc
Just to add, that is the case with all the tests:

$ sh run_functional_tests.sh
Ran 436 tests in 19.153s

FAILED (errors=434)

All tests fail locally and hence I decided to rely on travis.
https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
says it might be due to conflicting version of paste in my PYTHONPATH.
But it happens even if I run these under a virtualenv.

Saket

On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:

> On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>>> I happened to copy Peter's .travis.yml for running tests for some of
>>> my packages.
>>
>> Background information:
>> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>
>>> However the process has not been smooth at all.
>>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>
>>> AssertionError: Attempting to set field 'input' to value
>>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>> find value/label "condel_input.tabular" in list control
>>>
>>> I am already passing it the correct ftype :
>>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>
>>> Is there something I am missing?
>>
>> This is similar to (but different) to the error message I recently
>> reported as being unclear when a datatype was not defined:
>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>
>> In your case, this is an input file which seems to be missing,
>> test-data/condel_input.tabular - but I see the line copying the
>> tool's test-data into the main Galaxy test-data folder is there
>> in the .travis.yml file.
>>
>> I am puzzled too.
>>
>> The key question I would ask you is: do your tests work locally?
>> If so, are they installed via a (local) Tool Shed, or manually along
>> the same lines of the attempted TravisCI tool installation?
>>
>
> Unfortunately No.
>
> TwillAssertionError: code is 500 != 200
>
> I am running Galaxy under virtualenv. Unfortunately I get the same
> error irrespective of whther my input/output files exits inside
> test-data/ directory. I forgot to link them to test-data path earlier,
> but I get the same error even otherwise.
>
>
>> Regards,
>>
>> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
In reply to this post by John Chilton-4
Hi John,

My tool's input datatype is tabular. I do not have any custom datatypes defined.

Saket

On 17 April 2014 18:10, John Chilton <[hidden email]> wrote:

> I don't see anywhere in your Travis YAML file where you are modifying
> Galaxy's datatypes (like Peter does for instance on these lines
> https://github.com/peterjc/pico_galaxy/blob/master/.travis.yml#L166).
> If those datatypes are not defined you cannot upload a tool with a
> specific type. Am I missing something?
>
> -John
>
> On Thu, Apr 17, 2014 at 7:33 AM, Saket Choudhary <[hidden email]> wrote:
>> On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>>> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>>>> I happened to copy Peter's .travis.yml for running tests for some of
>>>> my packages.
>>>
>>> Background information:
>>> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>>
>>>> However the process has not been smooth at all.
>>>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>>
>>>> AssertionError: Attempting to set field 'input' to value
>>>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>>> find value/label "condel_input.tabular" in list control
>>>>
>>>> I am already passing it the correct ftype :
>>>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>>
>>>> Is there something I am missing?
>>>
>>> This is similar to (but different) to the error message I recently
>>> reported as being unclear when a datatype was not defined:
>>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>>
>>> In your case, this is an input file which seems to be missing,
>>> test-data/condel_input.tabular - but I see the line copying the
>>> tool's test-data into the main Galaxy test-data folder is there
>>> in the .travis.yml file.
>>>
>>> I am puzzled too.
>>>
>>> The key question I would ask you is: do your tests work locally?
>>> If so, are they installed via a (local) Tool Shed, or manually along
>>> the same lines of the attempted TravisCI tool installation?
>>>
>>
>> Unfortunately No.
>>
>> TwillAssertionError: code is 500 != 200
>>
>> I am running Galaxy under virtualenv. Unfortunately I get the same
>> error irrespective of whther my input/output files exits inside
>> test-data/ directory. I forgot to link them to test-data path earlier,
>> but I get the same error even otherwise.
>>
>>
>>> Regards,
>>>
>>> Peter
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

John Chilton-4
Oppps, duh sorry I thought condel_input.tabular was a datatype not a
file. That makes sense. I'll look some more.

-John

On Thu, Apr 17, 2014 at 7:45 AM, Saket Choudhary <[hidden email]> wrote:

> Hi John,
>
> My tool's input datatype is tabular. I do not have any custom datatypes defined.
>
> Saket
>
> On 17 April 2014 18:10, John Chilton <[hidden email]> wrote:
>> I don't see anywhere in your Travis YAML file where you are modifying
>> Galaxy's datatypes (like Peter does for instance on these lines
>> https://github.com/peterjc/pico_galaxy/blob/master/.travis.yml#L166).
>> If those datatypes are not defined you cannot upload a tool with a
>> specific type. Am I missing something?
>>
>> -John
>>
>> On Thu, Apr 17, 2014 at 7:33 AM, Saket Choudhary <[hidden email]> wrote:
>>> On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>>>> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>>>>> I happened to copy Peter's .travis.yml for running tests for some of
>>>>> my packages.
>>>>
>>>> Background information:
>>>> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>>>
>>>>> However the process has not been smooth at all.
>>>>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>>>
>>>>> AssertionError: Attempting to set field 'input' to value
>>>>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>>>> find value/label "condel_input.tabular" in list control
>>>>>
>>>>> I am already passing it the correct ftype :
>>>>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>>>
>>>>> Is there something I am missing?
>>>>
>>>> This is similar to (but different) to the error message I recently
>>>> reported as being unclear when a datatype was not defined:
>>>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>>>
>>>> In your case, this is an input file which seems to be missing,
>>>> test-data/condel_input.tabular - but I see the line copying the
>>>> tool's test-data into the main Galaxy test-data folder is there
>>>> in the .travis.yml file.
>>>>
>>>> I am puzzled too.
>>>>
>>>> The key question I would ask you is: do your tests work locally?
>>>> If so, are they installed via a (local) Tool Shed, or manually along
>>>> the same lines of the attempted TravisCI tool installation?
>>>>
>>>
>>> Unfortunately No.
>>>
>>> TwillAssertionError: code is 500 != 200
>>>
>>> I am running Galaxy under virtualenv. Unfortunately I get the same
>>> error irrespective of whther my input/output files exits inside
>>> test-data/ directory. I forgot to link them to test-data path earlier,
>>> but I get the same error even otherwise.
>>>
>>>
>>>> Regards,
>>>>
>>>> Peter
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

Ross-2
In reply to this post by saketkc
Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with your setup? 
What errors do you see?

Ross Lazarus MBBS MPH;
Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ


On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary <[hidden email]> wrote:
Just to add, that is the case with all the tests:

$ sh run_functional_tests.sh
Ran 436 tests in 19.153s

FAILED (errors=434)

All tests fail locally and hence I decided to rely on travis.
https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
says it might be due to conflicting version of paste in my PYTHONPATH.
But it happens even if I run these under a virtualenv.

Saket

On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:
> On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]> wrote:
>>> I happened to copy Peter's .travis.yml for running tests for some of
>>> my packages.
>>
>> Background information:
>> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>
>>> However the process has not been smooth at all.
>>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>
>>> AssertionError: Attempting to set field 'input' to value
>>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>> find value/label "condel_input.tabular" in list control
>>>
>>> I am already passing it the correct ftype :
>>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>
>>> Is there something I am missing?
>>
>> This is similar to (but different) to the error message I recently
>> reported as being unclear when a datatype was not defined:
>> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>
>> In your case, this is an input file which seems to be missing,
>> test-data/condel_input.tabular - but I see the line copying the
>> tool's test-data into the main Galaxy test-data folder is there
>> in the .travis.yml file.
>>
>> I am puzzled too.
>>
>> The key question I would ask you is: do your tests work locally?
>> If so, are they installed via a (local) Tool Shed, or manually along
>> the same lines of the attempted TravisCI tool installation?
>>
>
> Unfortunately No.
>
> TwillAssertionError: code is 500 != 200
>
> I am running Galaxy under virtualenv. Unfortunately I get the same
> error irrespective of whther my input/output files exits inside
> test-data/ directory. I forgot to link them to test-data path earlier,
> but I get the same error even otherwise.
>
>
>> Regards,
>>
>> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
Culprit: System Proxy.

I am behind a proxy and have my http_proxy, https_proxy variables set.
The trick is to unset them.
Because  I am relying on John's template[1] for testing, I had to just
unset  http_proxy ONLY.

Maybe I should add this to the wiki?

[1] https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite

On 17 April 2014 18:18, Ross <[hidden email]> wrote:

> Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with
> your setup?
> What errors do you see?
>
> Ross Lazarus MBBS MPH;
> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
> http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ
>
>
> On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary <[hidden email]> wrote:
>>
>> Just to add, that is the case with all the tests:
>>
>> $ sh run_functional_tests.sh
>> Ran 436 tests in 19.153s
>>
>> FAILED (errors=434)
>>
>> All tests fail locally and hence I decided to rely on travis.
>> https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
>> says it might be due to conflicting version of paste in my PYTHONPATH.
>> But it happens even if I run these under a virtualenv.
>>
>> Saket
>>
>> On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:
>> > On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>> >> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]>
>> >> wrote:
>> >>> I happened to copy Peter's .travis.yml for running tests for some of
>> >>> my packages.
>> >>
>> >> Background information:
>> >>
>> >> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>> >>
>> >>> However the process has not been smooth at all.
>> >>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>> >>>
>> >>> AssertionError: Attempting to set field 'input' to value
>> >>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>> >>> find value/label "condel_input.tabular" in list control
>> >>>
>> >>> I am already passing it the correct ftype :
>> >>>
>> >>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>> >>>
>> >>> Is there something I am missing?
>> >>
>> >> This is similar to (but different) to the error message I recently
>> >> reported as being unclear when a datatype was not defined:
>> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>> >>
>> >> In your case, this is an input file which seems to be missing,
>> >> test-data/condel_input.tabular - but I see the line copying the
>> >> tool's test-data into the main Galaxy test-data folder is there
>> >> in the .travis.yml file.
>> >>
>> >> I am puzzled too.
>> >>
>> >> The key question I would ask you is: do your tests work locally?
>> >> If so, are they installed via a (local) Tool Shed, or manually along
>> >> the same lines of the attempted TravisCI tool installation?
>> >>
>> >
>> > Unfortunately No.
>> >
>> > TwillAssertionError: code is 500 != 200
>> >
>> > I am running Galaxy under virtualenv. Unfortunately I get the same
>> > error irrespective of whther my input/output files exits inside
>> > test-data/ directory. I forgot to link them to test-data path earlier,
>> > but I get the same error even otherwise.
>> >
>> >
>> >> Regards,
>> >>
>> >> Peter
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

Peter Cock
In reply to this post by saketkc
HI John, Saket,

Looking at Skaet's .travis.yml he doesn't mess about with the
default datatypes_conf.xml(.sample) like I do, so his TravisCI
should have all the default Galaxy datatypes (including tabular).

In my more complex .travis.yml I modify this to two reasons,
first to add new datatypes, and second for a slight speedup
I remove most of the unused datatypes.

Peter

On Thu, Apr 17, 2014 at 1:45 PM, Saket Choudhary <[hidden email]> wrote:

> Hi John,
>
> My tool's input datatype is tabular. I do not have any custom datatypes defined.
>
> Saket
>
> On 17 April 2014 18:10, John Chilton <[hidden email]> wrote:
>> I don't see anywhere in your Travis YAML file where you are modifying
>> Galaxy's datatypes (like Peter does for instance on these lines
>> https://github.com/peterjc/pico_galaxy/blob/master/.travis.yml#L166).
>> If those datatypes are not defined you cannot upload a tool with a
>> specific type. Am I missing something?
>>
>> -John
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
In reply to this post by saketkc
Apparently my tool itself does a web API call, so the test is bound to
fail locally. My best bet is to rely on travis for now, however I
still see this failing:
https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721

The input file exists:
https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721

On testtoolshed: http://testtoolshed.g2.bx.psu.edu/view/saketkc/condel

Installation fails, because pycurl fails to install which I shall
report in other thread.

Saket

On 17 April 2014 18:45, Saket Choudhary <[hidden email]> wrote:

> Culprit: System Proxy.
>
> I am behind a proxy and have my http_proxy, https_proxy variables set.
> The trick is to unset them.
> Because  I am relying on John's template[1] for testing, I had to just
> unset  http_proxy ONLY.
>
> Maybe I should add this to the wiki?
>
> [1] https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite
>
> On 17 April 2014 18:18, Ross <[hidden email]> wrote:
>> Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with
>> your setup?
>> What errors do you see?
>>
>> Ross Lazarus MBBS MPH;
>> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
>> http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ
>>
>>
>> On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary <[hidden email]> wrote:
>>>
>>> Just to add, that is the case with all the tests:
>>>
>>> $ sh run_functional_tests.sh
>>> Ran 436 tests in 19.153s
>>>
>>> FAILED (errors=434)
>>>
>>> All tests fail locally and hence I decided to rely on travis.
>>> https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
>>> says it might be due to conflicting version of paste in my PYTHONPATH.
>>> But it happens even if I run these under a virtualenv.
>>>
>>> Saket
>>>
>>> On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:
>>> > On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>>> >> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]>
>>> >> wrote:
>>> >>> I happened to copy Peter's .travis.yml for running tests for some of
>>> >>> my packages.
>>> >>
>>> >> Background information:
>>> >>
>>> >> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>> >>
>>> >>> However the process has not been smooth at all.
>>> >>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>> >>>
>>> >>> AssertionError: Attempting to set field 'input' to value
>>> >>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>> >>> find value/label "condel_input.tabular" in list control
>>> >>>
>>> >>> I am already passing it the correct ftype :
>>> >>>
>>> >>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>> >>>
>>> >>> Is there something I am missing?
>>> >>
>>> >> This is similar to (but different) to the error message I recently
>>> >> reported as being unclear when a datatype was not defined:
>>> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>> >>
>>> >> In your case, this is an input file which seems to be missing,
>>> >> test-data/condel_input.tabular - but I see the line copying the
>>> >> tool's test-data into the main Galaxy test-data folder is there
>>> >> in the .travis.yml file.
>>> >>
>>> >> I am puzzled too.
>>> >>
>>> >> The key question I would ask you is: do your tests work locally?
>>> >> If so, are they installed via a (local) Tool Shed, or manually along
>>> >> the same lines of the attempted TravisCI tool installation?
>>> >>
>>> >
>>> > Unfortunately No.
>>> >
>>> > TwillAssertionError: code is 500 != 200
>>> >
>>> > I am running Galaxy under virtualenv. Unfortunately I get the same
>>> > error irrespective of whther my input/output files exits inside
>>> > test-data/ directory. I forgot to link them to test-data path earlier,
>>> > but I get the same error even otherwise.
>>> >
>>> >
>>> >> Regards,
>>> >>
>>> >> Peter
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   http://lists.bx.psu.edu/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>
>>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
John pointed out that the error its the upload of the test file itself
which is failing. I can  locate the input files
in test-data/

https://travis-ci.org/saketkc/galaxy_tools/jobs/23238748#L1723

Any clues as to where exactly am I going wrong?

On 17 April 2014 19:40, Saket Choudhary <[hidden email]> wrote:

> Apparently my tool itself does a web API call, so the test is bound to
> fail locally. My best bet is to rely on travis for now, however I
> still see this failing:
> https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721
>
> The input file exists:
> https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721
>
> On testtoolshed: http://testtoolshed.g2.bx.psu.edu/view/saketkc/condel
>
> Installation fails, because pycurl fails to install which I shall
> report in other thread.
>
> Saket
>
> On 17 April 2014 18:45, Saket Choudhary <[hidden email]> wrote:
>> Culprit: System Proxy.
>>
>> I am behind a proxy and have my http_proxy, https_proxy variables set.
>> The trick is to unset them.
>> Because  I am relying on John's template[1] for testing, I had to just
>> unset  http_proxy ONLY.
>>
>> Maybe I should add this to the wiki?
>>
>> [1] https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite
>>
>> On 17 April 2014 18:18, Ross <[hidden email]> wrote:
>>> Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with
>>> your setup?
>>> What errors do you see?
>>>
>>> Ross Lazarus MBBS MPH;
>>> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
>>> http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ
>>>
>>>
>>> On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary <[hidden email]> wrote:
>>>>
>>>> Just to add, that is the case with all the tests:
>>>>
>>>> $ sh run_functional_tests.sh
>>>> Ran 436 tests in 19.153s
>>>>
>>>> FAILED (errors=434)
>>>>
>>>> All tests fail locally and hence I decided to rely on travis.
>>>> https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
>>>> says it might be due to conflicting version of paste in my PYTHONPATH.
>>>> But it happens even if I run these under a virtualenv.
>>>>
>>>> Saket
>>>>
>>>> On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:
>>>> > On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>>>> >> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]>
>>>> >> wrote:
>>>> >>> I happened to copy Peter's .travis.yml for running tests for some of
>>>> >>> my packages.
>>>> >>
>>>> >> Background information:
>>>> >>
>>>> >> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>>> >>
>>>> >>> However the process has not been smooth at all.
>>>> >>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>> >>>
>>>> >>> AssertionError: Attempting to set field 'input' to value
>>>> >>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>>> >>> find value/label "condel_input.tabular" in list control
>>>> >>>
>>>> >>> I am already passing it the correct ftype :
>>>> >>>
>>>> >>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>> >>>
>>>> >>> Is there something I am missing?
>>>> >>
>>>> >> This is similar to (but different) to the error message I recently
>>>> >> reported as being unclear when a datatype was not defined:
>>>> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>>> >>
>>>> >> In your case, this is an input file which seems to be missing,
>>>> >> test-data/condel_input.tabular - but I see the line copying the
>>>> >> tool's test-data into the main Galaxy test-data folder is there
>>>> >> in the .travis.yml file.
>>>> >>
>>>> >> I am puzzled too.
>>>> >>
>>>> >> The key question I would ask you is: do your tests work locally?
>>>> >> If so, are they installed via a (local) Tool Shed, or manually along
>>>> >> the same lines of the attempted TravisCI tool installation?
>>>> >>
>>>> >
>>>> > Unfortunately No.
>>>> >
>>>> > TwillAssertionError: code is 500 != 200
>>>> >
>>>> > I am running Galaxy under virtualenv. Unfortunately I get the same
>>>> > error irrespective of whther my input/output files exits inside
>>>> > test-data/ directory. I forgot to link them to test-data path earlier,
>>>> > but I get the same error even otherwise.
>>>> >
>>>> >
>>>> >> Regards,
>>>> >>
>>>> >> Peter
>>>> ___________________________________________________________
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>   http://lists.bx.psu.edu/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
Can I throw in a bounty for this one? ;-)

On 18 April 2014 03:09, Saket Choudhary <[hidden email]> wrote:

> John pointed out that the error its the upload of the test file itself
> which is failing. I can  locate the input files
> in test-data/
>
> https://travis-ci.org/saketkc/galaxy_tools/jobs/23238748#L1723
>
> Any clues as to where exactly am I going wrong?
>
> On 17 April 2014 19:40, Saket Choudhary <[hidden email]> wrote:
>> Apparently my tool itself does a web API call, so the test is bound to
>> fail locally. My best bet is to rely on travis for now, however I
>> still see this failing:
>> https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721
>>
>> The input file exists:
>> https://travis-ci.org/saketkc/galaxy_tools/jobs/23206181#L1721
>>
>> On testtoolshed: http://testtoolshed.g2.bx.psu.edu/view/saketkc/condel
>>
>> Installation fails, because pycurl fails to install which I shall
>> report in other thread.
>>
>> Saket
>>
>> On 17 April 2014 18:45, Saket Choudhary <[hidden email]> wrote:
>>> Culprit: System Proxy.
>>>
>>> I am behind a proxy and have my http_proxy, https_proxy variables set.
>>> The trick is to unset them.
>>> Because  I am relying on John's template[1] for testing, I had to just
>>> unset  http_proxy ONLY.
>>>
>>> Maybe I should add this to the wiki?
>>>
>>> [1] https://github.com/jmchilton/galaxy-downloads/raw/master/db_gx_rev_0117.sqlite
>>>
>>> On 17 April 2014 18:18, Ross <[hidden email]> wrote:
>>>> Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with
>>>> your setup?
>>>> What errors do you see?
>>>>
>>>> Ross Lazarus MBBS MPH;
>>>> Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
>>>> http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ
>>>>
>>>>
>>>> On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary <[hidden email]> wrote:
>>>>>
>>>>> Just to add, that is the case with all the tests:
>>>>>
>>>>> $ sh run_functional_tests.sh
>>>>> Ran 436 tests in 19.153s
>>>>>
>>>>> FAILED (errors=434)
>>>>>
>>>>> All tests fail locally and hence I decided to rely on travis.
>>>>> https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests
>>>>> says it might be due to conflicting version of paste in my PYTHONPATH.
>>>>> But it happens even if I run these under a virtualenv.
>>>>>
>>>>> Saket
>>>>>
>>>>> On 17 April 2014 18:03, Saket Choudhary <[hidden email]> wrote:
>>>>> > On 17 April 2014 15:34, Peter Cock <[hidden email]> wrote:
>>>>> >> On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary <[hidden email]>
>>>>> >> wrote:
>>>>> >>> I happened to copy Peter's .travis.yml for running tests for some of
>>>>> >>> my packages.
>>>>> >>
>>>>> >> Background information:
>>>>> >>
>>>>> >> http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html
>>>>> >>
>>>>> >>> However the process has not been smooth at all.
>>>>> >>> See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908
>>>>> >>>
>>>>> >>> AssertionError: Attempting to set field 'input' to value
>>>>> >>> '['condel_input.tabular']' in form 'tool_form' threw exception: cannot
>>>>> >>> find value/label "condel_input.tabular" in list control
>>>>> >>>
>>>>> >>> I am already passing it the correct ftype :
>>>>> >>>
>>>>> >>> https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22
>>>>> >>>
>>>>> >>> Is there something I am missing?
>>>>> >>
>>>>> >> This is similar to (but different) to the error message I recently
>>>>> >> reported as being unclear when a datatype was not defined:
>>>>> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html
>>>>> >>
>>>>> >> In your case, this is an input file which seems to be missing,
>>>>> >> test-data/condel_input.tabular - but I see the line copying the
>>>>> >> tool's test-data into the main Galaxy test-data folder is there
>>>>> >> in the .travis.yml file.
>>>>> >>
>>>>> >> I am puzzled too.
>>>>> >>
>>>>> >> The key question I would ask you is: do your tests work locally?
>>>>> >> If so, are they installed via a (local) Tool Shed, or manually along
>>>>> >> the same lines of the attempted TravisCI tool installation?
>>>>> >>
>>>>> >
>>>>> > Unfortunately No.
>>>>> >
>>>>> > TwillAssertionError: code is 500 != 200
>>>>> >
>>>>> > I am running Galaxy under virtualenv. Unfortunately I get the same
>>>>> > error irrespective of whther my input/output files exits inside
>>>>> > test-data/ directory. I forgot to link them to test-data path earlier,
>>>>> > but I get the same error even otherwise.
>>>>> >
>>>>> >
>>>>> >> Regards,
>>>>> >>
>>>>> >> Peter
>>>>> ___________________________________________________________
>>>>> Please keep all replies on the list by using "reply all"
>>>>> in your mail client.  To manage your subscriptions to this
>>>>> and other Galaxy lists, please use the interface at:
>>>>>   http://lists.bx.psu.edu/
>>>>>
>>>>> To search Galaxy mailing lists use the unified search at:
>>>>>   http://galaxyproject.org/search/mailinglists/
>>>>
>>>>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

Peter Cock
In reply to this post by saketkc
On Thu, Apr 17, 2014 at 10:39 PM, Saket Choudhary <[hidden email]> wrote:
> John pointed out that the error its the upload of the test file itself
> which is failing. I can  locate the input files
> in test-data/
>
> https://travis-ci.org/saketkc/galaxy_tools/jobs/23238748#L1723
>
> Any clues as to where exactly am I going wrong?

Very strange. Adding "ls test-data" to the TravisCI list worked,
and showed the input file condel_input.tabular was present:
https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt

How about replacing that with "ls -l test-data/condel_input.tabular"
to verify the ownership and permissions (although if that was
the upload problem, Galaxy could give a clearer error).

Likely unrelated, but I don't think I have bothered to set the ftype
when uploading tabular files / using them as test inputs. Did you
ever try this without the ftype?

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:

> On Thu, Apr 17, 2014 at 10:39 PM, Saket Choudhary <[hidden email]> wrote:
>> John pointed out that the error its the upload of the test file itself
>> which is failing. I can  locate the input files
>> in test-data/
>>
>> https://travis-ci.org/saketkc/galaxy_tools/jobs/23238748#L1723
>>
>> Any clues as to where exactly am I going wrong?
>
> Very strange. Adding "ls test-data" to the TravisCI list worked,
> and showed the input file condel_input.tabular was present:
> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>
> How about replacing that with "ls -l test-data/condel_input.tabular"
> to verify the ownership and permissions (although if that was
> the upload problem, Galaxy could give a clearer error).
>

Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530

This happens irrespective of using ftype. and the plce where ERROR
occurs is this:

galaxy.jobs.runners.local DEBUG 2014-04-17 21:25:53,808 execution
finished: GALAXY_SLOTS="1"; export GALAXY_SLOTS; python
/home/travis/build/saketkc/galaxy_tools/galaxy-central-master/tools/data_source/upload.py
/home/travis/build/saketkc/galaxy_tools/galaxy-central-master
/tmp/tmpNenHnc/tmp83XrmN/new_files_path_4s4ndH/tmpvroRLA
/tmp/tmpNenHnc/tmp83XrmN/new_files_path_4s4ndH/tmpv9tzOU
1:/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/dataset_1_files:/tmp/tmpNenHnc/tmp83XrmN/database/files/000/dataset_1.dat
galaxy.datatypes.metadata INFO 2014-04-17 21:25:55,902
/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_in_HistoryDatasetAssociation_1_bVl30h,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_kwds_HistoryDatasetAssociation_1_jiQBR3,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_out_HistoryDatasetAssociation_1_aoG7yJ,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_results_HistoryDatasetAssociation_1_JM1wqM,,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_override_HistoryDatasetAssociation_1_ZUFWdn
galaxy.jobs.runners DEBUG 2014-04-17 21:25:55,903 executing external
set_meta script for job 1:
/home/travis/build/saketkc/galaxy_tools/galaxy-central-master/set_metadata.sh
/tmp/tmpNenHnc/tmp83XrmN/database/files
/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1 .
/tmp/tmpNenHnc/functional_tests_wsgi.ini
/tmp/tmpNenHnc/tmp83XrmN/new_files_path_4s4ndH/tmpvroRLA
/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/galaxy.json
/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_in_HistoryDatasetAssociation_1_bVl30h,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_kwds_HistoryDatasetAssociation_1_jiQBR3,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_out_HistoryDatasetAssociation_1_aoG7yJ,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_results_HistoryDatasetAssociation_1_JM1wqM,,/tmp/tmpNenHnc/tmp83XrmN/job_working_directory_AsN2Fs/000/1/metadata_override_HistoryDatasetAssociation_1_ZUFWdn
galaxy.jobs.runners DEBUG 2014-04-17 21:25:57,620 execution of
external set_meta for job 1 finished
galaxy.jobs DEBUG 2014-04-17 21:25:57,727 setting dataset state to ERROR



so my dataset state is set to ERROR, essentially upload failed. But
why, is what it doesn't tell me.

> Likely unrelated, but I don't think I have bothered to set the ftype
> when uploading tabular files / using them as test inputs. Did you
> ever try this without the ftype?
>
> Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

Peter Cock
On Fri, Apr 18, 2014 at 11:47 AM, Saket Choudhary <[hidden email]> wrote:

> On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:
>> Very strange. Adding "ls test-data" to the TravisCI list worked,
>> and showed the input file condel_input.tabular was present:
>> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>>
>> How about replacing that with "ls -l test-data/condel_input.tabular"
>> to verify the ownership and permissions (although if that was
>> the upload problem, Galaxy could give a clearer error).
>>
>
> Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530
>

I see, that looks fine:

$ ls -l test-data/condel_input.tabular
-rwxrwxr-x 1 travis travis 110 Apr 18 10:28 test-data/condel_input.tabular

> This happens irrespective of using ftype. and the plce where ERROR
> occurs is this:
>
> ...
>
> so my dataset state is set to ERROR, essentially upload failed. But
> why, is what it doesn't tell me.

If this were a systematic problem other people should have seen
it - but I have tests using tabular input files which still work fine.

I wonder where exactly in the process it is failing... perhaps in the
metadata processing which would be consistent with it being some
oddity in your sample file which is causing trouble?

Can you upload this file condel_input.tabular into Galaxy via the
UI without error?

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
On 18 April 2014 18:06, Peter Cock <[hidden email]> wrote:

> On Fri, Apr 18, 2014 at 11:47 AM, Saket Choudhary <[hidden email]> wrote:
>> On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:
>>> Very strange. Adding "ls test-data" to the TravisCI list worked,
>>> and showed the input file condel_input.tabular was present:
>>> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>>>
>>> How about replacing that with "ls -l test-data/condel_input.tabular"
>>> to verify the ownership and permissions (although if that was
>>> the upload problem, Galaxy could give a clearer error).
>>>
>>
>> Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530
>>
>
> I see, that looks fine:
>
> $ ls -l test-data/condel_input.tabular
> -rwxrwxr-x 1 travis travis 110 Apr 18 10:28 test-data/condel_input.tabular
>
>> This happens irrespective of using ftype. and the plce where ERROR
>> occurs is this:
>>
>> ...
>>
>> so my dataset state is set to ERROR, essentially upload failed. But
>> why, is what it doesn't tell me.
>
> If this were a systematic problem other people should have seen
> it - but I have tests using tabular input files which still work fine.
>
> I wonder where exactly in the process it is failing... perhaps in the
> metadata processing which would be consistent with it being some
> oddity in your sample file which is causing trouble?
>
> Can you upload this file condel_input.tabular into Galaxy via the
> UI without error?
>

Yes, the upload works fine with 'Auto-Detect'. It sets the type to be
'interval'. I am also able to upload if I manually set it to
'tabular', which is anyway expected.

Just to double check I tested it with 'filter1_test5.tab' that comes
with galaxy-central and that is failing too:
https://travis-ci.org/saketkc/galaxy_tools/jobs/23277155#L1283

I am going to clone your repository and try pushing in my tools to the
clone to see if that helps.


> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
* fork your repository.

On 18 April 2014 18:42, Saket Choudhary <[hidden email]> wrote:

> On 18 April 2014 18:06, Peter Cock <[hidden email]> wrote:
>> On Fri, Apr 18, 2014 at 11:47 AM, Saket Choudhary <[hidden email]> wrote:
>>> On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:
>>>> Very strange. Adding "ls test-data" to the TravisCI list worked,
>>>> and showed the input file condel_input.tabular was present:
>>>> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>>>>
>>>> How about replacing that with "ls -l test-data/condel_input.tabular"
>>>> to verify the ownership and permissions (although if that was
>>>> the upload problem, Galaxy could give a clearer error).
>>>>
>>>
>>> Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530
>>>
>>
>> I see, that looks fine:
>>
>> $ ls -l test-data/condel_input.tabular
>> -rwxrwxr-x 1 travis travis 110 Apr 18 10:28 test-data/condel_input.tabular
>>
>>> This happens irrespective of using ftype. and the plce where ERROR
>>> occurs is this:
>>>
>>> ...
>>>
>>> so my dataset state is set to ERROR, essentially upload failed. But
>>> why, is what it doesn't tell me.
>>
>> If this were a systematic problem other people should have seen
>> it - but I have tests using tabular input files which still work fine.
>>
>> I wonder where exactly in the process it is failing... perhaps in the
>> metadata processing which would be consistent with it being some
>> oddity in your sample file which is causing trouble?
>>
>> Can you upload this file condel_input.tabular into Galaxy via the
>> UI without error?
>>
>
> Yes, the upload works fine with 'Auto-Detect'. It sets the type to be
> 'interval'. I am also able to upload if I manually set it to
> 'tabular', which is anyway expected.
>
> Just to double check I tested it with 'filter1_test5.tab' that comes
> with galaxy-central and that is failing too:
> https://travis-ci.org/saketkc/galaxy_tools/jobs/23277155#L1283
>
> I am going to clone your repository and try pushing in my tools to the
> clone to see if that helps.
>
>
>> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

John Chilton-4
Pushed out this changeset
https://bitbucket.org/galaxy/galaxy-central/commits/183668ed911ca0ad937f3ffebf81557fb6567fd4
which should allow better reporting of dataset errors in histories.

Make sure export GALAXY_TEST_VERBOSE_ERRORS=True is executed before
your tests and then check the output of the API driven version - it
should have a breakdown of all datasets in error when it encounters a
history that is not ok. This should include the standard error and
standard output from the upload tool - which in turn should tell us
the exact problem (fingers crossed).



On Fri, Apr 18, 2014 at 8:13 AM, Saket Choudhary <[hidden email]> wrote:

> * fork your repository.
>
> On 18 April 2014 18:42, Saket Choudhary <[hidden email]> wrote:
>> On 18 April 2014 18:06, Peter Cock <[hidden email]> wrote:
>>> On Fri, Apr 18, 2014 at 11:47 AM, Saket Choudhary <[hidden email]> wrote:
>>>> On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:
>>>>> Very strange. Adding "ls test-data" to the TravisCI list worked,
>>>>> and showed the input file condel_input.tabular was present:
>>>>> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
>>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
>>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>>>>>
>>>>> How about replacing that with "ls -l test-data/condel_input.tabular"
>>>>> to verify the ownership and permissions (although if that was
>>>>> the upload problem, Galaxy could give a clearer error).
>>>>>
>>>>
>>>> Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530
>>>>
>>>
>>> I see, that looks fine:
>>>
>>> $ ls -l test-data/condel_input.tabular
>>> -rwxrwxr-x 1 travis travis 110 Apr 18 10:28 test-data/condel_input.tabular
>>>
>>>> This happens irrespective of using ftype. and the plce where ERROR
>>>> occurs is this:
>>>>
>>>> ...
>>>>
>>>> so my dataset state is set to ERROR, essentially upload failed. But
>>>> why, is what it doesn't tell me.
>>>
>>> If this were a systematic problem other people should have seen
>>> it - but I have tests using tabular input files which still work fine.
>>>
>>> I wonder where exactly in the process it is failing... perhaps in the
>>> metadata processing which would be consistent with it being some
>>> oddity in your sample file which is causing trouble?
>>>
>>> Can you upload this file condel_input.tabular into Galaxy via the
>>> UI without error?
>>>
>>
>> Yes, the upload works fine with 'Auto-Detect'. It sets the type to be
>> 'interval'. I am also able to upload if I manually set it to
>> 'tabular', which is anyway expected.
>>
>> Just to double check I tested it with 'filter1_test5.tab' that comes
>> with galaxy-central and that is failing too:
>> https://travis-ci.org/saketkc/galaxy_tools/jobs/23277155#L1283
>>
>> I am going to clone your repository and try pushing in my tools to the
>> clone to see if that helps.
>>
>>
>>> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Functional Tests and ftype

saketkc
Thanks John,

That really helped:

Exception: History in error state.

-------------------- >> begin captured stdout << ---------------------

History with id adb5f5c93f827949 in error - summary of datasets in error below.

--------------------------------------

| 1 - filter1_test5.tab (HID - NAME)

| Dataset Blurb:

| error

| Dataset Info:

| uploaded tabular file

| /usr/local/lib/python2.7/dist-packages/setuptools-2.2-py2.7.egg/pkg_resources.py:991:
UserWarning: /home/travis/.python-eggs is writable by group/others and
vulnerable to attack when used with get_resource_filename. Consider a
more

| Dataset Job Standard Output:

| *Standard output was empty.*

| Dataset Job Standard Error:

| /usr/local/lib/python2.7/dist-packages/setuptools-2.2-py2.7.egg/pkg_resources.py:991:
UserWarning: /home/travis/.python-eggs is writable by group/others and
vulnerable to attack when used with get_resource_filename. Consider a
more secure location (set with .set_extraction_path or the
PYTHON_EGG_CACHE environment variable).

|

--------------------------------------

--------------------- >> end captured stdout << ----------------------


Fixed here: https://travis-ci.org/saketkc/galaxy_tools/jobs/23282261#L1402
with:

before_install:
 - mkdir "$HOME/.python-eggs"
 - chmod 700 "$HOME/.python-eggs"



The test is failing, but that I expected it to anyway ;)

Thanks everyone! Especially John and Peter!

Saket


On 18 April 2014 19:15, John Chilton <[hidden email]> wrote:

> Pushed out this changeset
> https://bitbucket.org/galaxy/galaxy-central/commits/183668ed911ca0ad937f3ffebf81557fb6567fd4
> which should allow better reporting of dataset errors in histories.
>
> Make sure export GALAXY_TEST_VERBOSE_ERRORS=True is executed before
> your tests and then check the output of the API driven version - it
> should have a breakdown of all datasets in error when it encounters a
> history that is not ok. This should include the standard error and
> standard output from the upload tool - which in turn should tell us
> the exact problem (fingers crossed).
>
>
>
> On Fri, Apr 18, 2014 at 8:13 AM, Saket Choudhary <[hidden email]> wrote:
>> * fork your repository.
>>
>> On 18 April 2014 18:42, Saket Choudhary <[hidden email]> wrote:
>>> On 18 April 2014 18:06, Peter Cock <[hidden email]> wrote:
>>>> On Fri, Apr 18, 2014 at 11:47 AM, Saket Choudhary <[hidden email]> wrote:
>>>>> On 18 April 2014 14:59, Peter Cock <[hidden email]> wrote:
>>>>>> Very strange. Adding "ls test-data" to the TravisCI list worked,
>>>>>> and showed the input file condel_input.tabular was present:
>>>>>> https://github.com/saketkc/galaxy_tools/commit/c04ff946db0182de9aca4f6dc36ff194404cae05
>>>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239476/log.txt
>>>>>> https://s3.amazonaws.com/archive.travis-ci.org/jobs/23239477/log.txt
>>>>>>
>>>>>> How about replacing that with "ls -l test-data/condel_input.tabular"
>>>>>> to verify the ownership and permissions (although if that was
>>>>>> the upload problem, Galaxy could give a clearer error).
>>>>>>
>>>>>
>>>>> Thanks, tried it: https://travis-ci.org/saketkc/galaxy_tools/jobs/23270454#L530
>>>>>
>>>>
>>>> I see, that looks fine:
>>>>
>>>> $ ls -l test-data/condel_input.tabular
>>>> -rwxrwxr-x 1 travis travis 110 Apr 18 10:28 test-data/condel_input.tabular
>>>>
>>>>> This happens irrespective of using ftype. and the plce where ERROR
>>>>> occurs is this:
>>>>>
>>>>> ...
>>>>>
>>>>> so my dataset state is set to ERROR, essentially upload failed. But
>>>>> why, is what it doesn't tell me.
>>>>
>>>> If this were a systematic problem other people should have seen
>>>> it - but I have tests using tabular input files which still work fine.
>>>>
>>>> I wonder where exactly in the process it is failing... perhaps in the
>>>> metadata processing which would be consistent with it being some
>>>> oddity in your sample file which is causing trouble?
>>>>
>>>> Can you upload this file condel_input.tabular into Galaxy via the
>>>> UI without error?
>>>>
>>>
>>> Yes, the upload works fine with 'Auto-Detect'. It sets the type to be
>>> 'interval'. I am also able to upload if I manually set it to
>>> 'tabular', which is anyway expected.
>>>
>>> Just to double check I tested it with 'filter1_test5.tab' that comes
>>> with galaxy-central and that is failing too:
>>> https://travis-ci.org/saketkc/galaxy_tools/jobs/23277155#L1283
>>>
>>> I am going to clone your repository and try pushing in my tools to the
>>> clone to see if that helps.
>>>
>>>
>>>> Peter
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
12