Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

Peter Cock
Hi guys,

Yesterday I made a small tweak to the BLAST+ packages to fix a bash syntax error in the fall back action:

https://github.com/peterjc/galaxy_blast/commit/4b2fa1fbf485c4630cb6dc0beaf7286cb103bd71

I uploaded this update to the Test Tool Shed packages (all four, BLAST+ 2.2.26, 2.2.27, 2.2.28 and 2.2.29), most of which now unfortunately all show up under "Latest revision: installation errors" (along with the BLAST+ wrappers):




Name Latest Installable RevisionOwner
ncbi_blast_plus
peterjc
package_blast_plus_2_2_26
iuc
package_blast_plus_2_2_28
iuc
package_blast_plus_2_2_29
iuc


The BLAST+ 2.2.26 package doesn't seem to show an error:

Automated tool test results

TypeNameVersion
blast+ package 2.2.26+
Error
None


The BLAST+ 2.2.27 package seems to have no test results at all.

However, the BLAST+ 2.2.28 package has this strange unicode error:

Automated tool test results

Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'


And similarly for BLAST+ 2.2.29,

Automated tool test results

Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'


This is puzzling, since the changes I made to the packages seemed innocuous - making me wonder if something changed in the Test Tool Shed itself?

Regards,

Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

Greg Von Kuster
Hello Peter,

On Feb 21, 2014, at 4:53 AM, Peter Cock <[hidden email]> wrote:

Hi guys,

Yesterday I made a small tweak to the BLAST+ packages to fix a bash syntax error in the fall back action:

https://github.com/peterjc/galaxy_blast/commit/4b2fa1fbf485c4630cb6dc0beaf7286cb103bd71


The BLAST+ 2.2.27 package seems to have no test results at all.


The above issue was due to a bug that was introduced into the install and test framework late yesterday, causing the test to not run at all.  The bug has been corrected in preparation for tonight's test run.  The Test environment entry for the above repository was produced by the preparation script.



However, the BLAST+ 2.2.28 package has this strange unicode error:

Automated tool test results

Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'


And similarly for BLAST+ 2.2.29,

Automated tool test results

Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'
Error
'unicode' object has no attribute 'get'


This is puzzling, since the changes I made to the packages seemed innocuous - making me wonder if something changed in the Test Tool Shed itself?


The above 2 issues were the result of a bug in the Install and Test Framework that was exposed only with the changes you made to you repositories.  The bug resulted in invalid information being included in the test results database column, and the "unicode object has not attribute get" error you're seeing above is due to the exception handling for displaying the invalid data in the Tool Shed repository's Test runs container.  We've corrected the data in the database as well as the bug that produced it, so tonight's test run should not result in this behavior for these repositories.

Thanks very much!

Greg Von Kuster



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

Peter Cock
On Fri, Feb 21, 2014 at 6:54 PM, Greg Von Kuster <[hidden email]> wrote:

>
> Hello Peter,
>
> On Feb 21, 2014, at 4:53 AM, Peter Cock <[hidden email]> wrote:
>>
>> Hi guys,
>>
>> Yesterday I made a small tweak to the BLAST+ packages
>> to fix a bash syntax error in the fall back action:
>>
>> https://github.com/peterjc/galaxy_blast/commit/4b2fa1fbf485c4630cb6dc0beaf7286cb103bd71
>>
>> The BLAST+ 2.2.27 package seems to have no test results at all.
>>
>
> The above issue was due to a bug that was introduced
> into the install and test framework late yesterday, causing
> the test to not run at all.  The bug has been corrected in
> preparation for tonight's test run.  The Test environment
> entry for the above repository was produced by the
> preparation script.
>
>> However, the BLAST+ 2.2.28 package has this strange
>> unicode error:
>>
>> Automated tool test results
>> ...
>> Error
>> 'unicode' object has no attribute 'get'
>>
>> And similarly for BLAST+ 2.2.29,
>> ...
>> Error
>> 'unicode' object has no attribute 'get'
>>
>> This is puzzling, since the changes I made to the
>> packages seemed innocuous - making me wonder
>> if something changed in the Test Tool Shed itself?
>
> The above 2 issues were the result of a bug in the Install
> and Test Framework that was exposed only with the
> changes you made to you repositories.  The bug resulted
> in invalid information being included in the test results
> database column, and the "unicode object has not attribute
> get" error you're seeing above is due to the exception
> handling for displaying the invalid data in the Tool Shed
> repository's Test runs container.  We've corrected the data
> in the database as well as the bug that produced it, so
> tonight's test run should not result in this behaviour for
> these repositories.
>
> Thanks very much!
>
> Greg Von Kuster

Hi Greg,

Good progress - no sign of the unicode error :)

I now have the following three repositories shown under
"Latest revision: installation errors"

http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus
(Seems to list both a failed and a successful BLAST+ 2.2.29 install?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26
(No error?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_28
(No error?)

On the other hand,

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_27
(No test/install results?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_29
Looks good.

So it seems there is still something not quite right here...

Thanks,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

Greg Von Kuster
Hello Peter,


On Feb 24, 2014, at 3:32 AM, Peter Cock <[hidden email]> wrote:


Hi Greg,

Good progress - no sign of the unicode error :)

I now have the following three repositories shown under
"Latest revision: installation errors"

http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus
(Seems to list both a failed and a successful BLAST+ 2.2.29 install?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26
(No error?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_28
(No error?)

On the other hand,

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_27
(No test/install results?)

http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_29
Looks good.

So it seems there is still something not quite right here…

We've eliminated the use of fabric for installing tool dependencies on the test tool shed, and with this change I've seen the test tool shed fail at the same point the past 3 runs.  It fails attempting to compile boost where it times out ( we have abuildbot timeout set at 36000 seconds if nothing occurs).  This error occurs about half way through Stage 1 of the test run, so about 1/2 of the tool dependency definition repositories are not tested and no tests are run for repositories with tools.

 The following log shows the repeaated compilation error.  We'll be investigating approaches for handling issues like this.  Now that we've eliminated fabric we'll have many more options for handling installation issues like this.  Of course, if it turns out that our new code that eliminated fabric caused this, we'll get a fix in.

Thanks,

Greg Von Kuster

fabric_util.STDOUT DEBUG 2014-02-23 20:20:14,740 -- NUMBER_OF_JOBS: 2 (maximal number of concurrent compile jobs)
fabric_util.STDOUT DEBUG 2014-02-23 20:20:14,741 -- Extracting BOOST ..
fabric_util.STDOUT DEBUG 2014-02-23 20:20:20,006 -- Extracting BOOST .. done
fabric_util.STDOUT DEBUG 2014-02-23 20:20:20,006 -- Bootstrapping Boost libraries (./bootstrap.sh --prefix=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib --with-libraries=date_time,iostreams,math,regex) ...
fabric_util.STDOUT DEBUG 2014-02-23 20:20:37,231 -- Bootstrapping Boost libraries (./bootstrap.sh --prefix=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib --with-libraries=iostreams,math,date_time,regex) ... done
fabric_util.STDOUT DEBUG 2014-02-23 20:20:37,232 -- Installing Boost libraries (./bjam -j 2 -sZLIB_SOURCE=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib/src/zlib-1.2.5 -sBZIP2_SOURCE=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib/src/bzip2-1.0.5 link=static cxxflags=-fPIC install --build-type=complete --layout=tagged --threading=single,multi) ...
fabric_util.STDOUT DEBUG 2014-02-23 20:36:11,619 -- Installing Boost libraries (./bjam -j 2 -sZLIB_SOURCE=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib/src/zlib-1.2.5 -sBZIP2_SOURCE=/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/tool_dependency_definitions/tmp/tmpShf1jI/tmp-toolshed-mtdq8JEyP/OpenMS-1.11.1/contrib/src/bzip2-1.0.5 link=static cxxflags=-fPIC install --build-type=complete --layout=tagged --threading=single,multi) ... failed

command timed out: 36000 seconds without output, attempting to kill
process killed by signal 9




Thanks,

Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Test Tool Shed, nightly tests: Error 'unicode' object has no attribute 'get'

Peter Cock
On Mon, Feb 24, 2014 at 12:03 PM, Greg Von Kuster <[hidden email]> wrote:
> Hello Peter,
>
> We've eliminated the use of fabric for installing tool dependencies on the
> test tool shed, and with this change I've seen the test tool shed fail at
> the same point the past 3 runs.  It fails attempting to compile boost where
> it times out ( we have abuildbot timeout set at 36000 seconds if nothing
> occurs).  This error occurs about half way through Stage 1 of the test run,
> so about 1/2 of the tool dependency definition repositories are not tested
> and no tests are run for repositories with tools.

Ah. So the apparently random split in behaviour for the BLAST+
packages may just be down to if they were in the first or second
half of the repositories?

> The following log shows the repeaated compilation error.  We'll be
> investigating approaches for handling issues like this.  Now that we've
> eliminated fabric we'll have many more options for handling installation
> issues like this.  Of course, if it turns out that our new code that
> eliminated fabric caused this, we'll get a fix in.

If push comes to shove, you could declare BOOST an expected
library to be pre-installed?
https://wiki.galaxyproject.org/Admin/Config/ToolDependenciesList

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Error in Join two Datasets (version 2.0.2)

nephantes
Hi,
I got an error in “Join two Datasets" (version 2.0.2).

/home/galaxy-dist/lib/galaxy/__init__.py:5: UserWarning: Module psyco_full was already imported from /home/galaxy-dist/lib/psyco_full.py, but /usr/lib64/python2.6/site-packages is being added to sys.path
  __import__( "pkg_resources" ).declare_namespace( __name__ )

Do you know, what could be the reason? 
Thanks,

Alper Kucukural, PhD
Bioinformatics Core,
University of Massachusetts Medical School
368 Plantation St.Room AS4.2067
Worcester, MA 01605-2324
Phone: 774-312-4493
E-mail: [hidden email]


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Error in Join two Datasets (version 2.0.2)

Björn Grüning-3
Hi Alper,

as the error suggests you have two versions of psyco_full in your
PYTHONPATH, is it rseqc? You can get rid of any of these versions or run
Galaxy inside a virtualenv if you like:

wget https://raw.github.com/pypa/virtualenv/master/virtualenv.py
python ./virtualenv.py --no-site-packages galaxy_env
. ./galaxy_env/bin/activate

Cheers,
Bjoern

> Hi,
> I got an error in “Join two Datasets" (version 2.0.2).
>
> /home/galaxy-dist/lib/galaxy/__init__.py:5: UserWarning: Module psyco_full was already imported from /home/galaxy-dist/lib/psyco_full.py, but /usr/lib64/python2.6/site-packages is being added to sys.path
>    __import__( "pkg_resources" ).declare_namespace( __name__ )
>
> Do you know, what could be the reason?
> Thanks,
>
> Alper Kucukural, PhD
> Bioinformatics Core,
> University of Massachusetts Medical School
> 368 Plantation St.Room AS4.2067
> Worcester, MA 01605-2324
> Phone: 774-312-4493
> E-mail: [hidden email]
>
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>    http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>    http://galaxyproject.org/search/mailinglists/
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/