Test failure, JSONDecodeError: Unpaired high surrogate

classic Classic list List threaded Threaded
3 messages Options
| Threaded
Open this post in threaded view
|

Test failure, JSONDecodeError: Unpaired high surrogate

Peter Cock
Hi all,

I'm getting the following exception in a failing unit test:
https://travis-ci.org/peterjc/pico_galaxy/builds/39398677

Testing this tool (where two of the three near identical tests passed):
https://github.com/peterjc/pico_galaxy/blob/dd03346710e6a46cb6ec9dda1eed23d5fd301d03/tools/mummer/mummer.xml

```
Traceback (most recent call last):
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
line 116, in test_tool
    self.do_it( td )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
line 35, in do_it
    self._verify_outputs( testdef, test_history, jobs, shed_tool_id,
data_list, galaxy_interactor )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
line 75, in _verify_outputs
    galaxy_interactor.verify_output( history, jobs, output_data,
output_testdef=output_testdef, shed_tool_id=shed_tool_id,
maxseconds=maxseconds )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
line 89, in verify_output
    self._verify_metadata( history_id, hid, attributes )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
line 102, in _verify_metadata
    dataset = self._get( "histories/%s/contents/%s" % ( history_id,
hid ) ).json()
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/requests-2.2.1-py2.7.egg/requests/models.py",
line 740, in json
    return json.loads(self.content.decode(encoding), **kwargs)
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line
413, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
402, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
418, in raw_decode
    obj, end = self.scan_once(s, idx)
JSONDecodeError: Unpaired high surrogate: line 1 column 785 (char 785)
```

Probably relevant:
 - https://github.com/simplejson/simplejson/issues/62
 - http://bugs.python.org/issue11489

Any thoughts? What does Galaxy write to these job-associated JSON
metadata files?

Thanks,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Test failure, JSONDecodeError: Unpaired high surrogate

Peter Cock
I have "solved" this by commenting out the apparently harmless test:

https://github.com/peterjc/pico_galaxy/commit/f3d4261846566a86f9c85a158fb95877ca8bc7c5

Peter

On Wed, Oct 29, 2014 at 5:39 PM, Peter Cock <[hidden email]> wrote:

> Hi all,
>
> I'm getting the following exception in a failing unit test:
> https://travis-ci.org/peterjc/pico_galaxy/builds/39398677
>
> Testing this tool (where two of the three near identical tests passed):
> https://github.com/peterjc/pico_galaxy/blob/dd03346710e6a46cb6ec9dda1eed23d5fd301d03/tools/mummer/mummer.xml
>
> ```
> Traceback (most recent call last):
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
> line 116, in test_tool
>     self.do_it( td )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
> line 35, in do_it
>     self._verify_outputs( testdef, test_history, jobs, shed_tool_id,
> data_list, galaxy_interactor )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
> line 75, in _verify_outputs
>     galaxy_interactor.verify_output( history, jobs, output_data,
> output_testdef=output_testdef, shed_tool_id=shed_tool_id,
> maxseconds=maxseconds )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
> line 89, in verify_output
>     self._verify_metadata( history_id, hid, attributes )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
> line 102, in _verify_metadata
>     dataset = self._get( "histories/%s/contents/%s" % ( history_id,
> hid ) ).json()
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/requests-2.2.1-py2.7.egg/requests/models.py",
> line 740, in json
>     return json.loads(self.content.decode(encoding), **kwargs)
>   File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line
> 413, in loads
>     return _default_decoder.decode(s)
>   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
> 402, in decode
>     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
> 418, in raw_decode
>     obj, end = self.scan_once(s, idx)
> JSONDecodeError: Unpaired high surrogate: line 1 column 785 (char 785)
> ```
>
> Probably relevant:
>  - https://github.com/simplejson/simplejson/issues/62
>  - http://bugs.python.org/issue11489
>
> Any thoughts? What does Galaxy write to these job-associated JSON
> metadata files?
>
> Thanks,
>
> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Test failure, JSONDecodeError: Unpaired high surrogate

John Chilton-4
Ugh... sorry I missed this e-mail. I don't have any answers yet but I
have created a Trello card to track the issue.

https://trello.com/c/3yVLp8RA

Thanks for the detailed error report.

-John

On Fri, Oct 31, 2014 at 12:16 PM, Peter Cock <[hidden email]> wrote:

> I have "solved" this by commenting out the apparently harmless test:
>
> https://github.com/peterjc/pico_galaxy/commit/f3d4261846566a86f9c85a158fb95877ca8bc7c5
>
> Peter
>
> On Wed, Oct 29, 2014 at 5:39 PM, Peter Cock <[hidden email]> wrote:
>> Hi all,
>>
>> I'm getting the following exception in a failing unit test:
>> https://travis-ci.org/peterjc/pico_galaxy/builds/39398677
>>
>> Testing this tool (where two of the three near identical tests passed):
>> https://github.com/peterjc/pico_galaxy/blob/dd03346710e6a46cb6ec9dda1eed23d5fd301d03/tools/mummer/mummer.xml
>>
>> ```
>> Traceback (most recent call last):
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
>> line 116, in test_tool
>>     self.do_it( td )
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
>> line 35, in do_it
>>     self._verify_outputs( testdef, test_history, jobs, shed_tool_id,
>> data_list, galaxy_interactor )
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/functional/test_toolbox.py",
>> line 75, in _verify_outputs
>>     galaxy_interactor.verify_output( history, jobs, output_data,
>> output_testdef=output_testdef, shed_tool_id=shed_tool_id,
>> maxseconds=maxseconds )
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
>> line 89, in verify_output
>>     self._verify_metadata( history_id, hid, attributes )
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/test/base/interactor.py",
>> line 102, in _verify_metadata
>>     dataset = self._get( "histories/%s/contents/%s" % ( history_id,
>> hid ) ).json()
>>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-central-master/eggs/requests-2.2.1-py2.7.egg/requests/models.py",
>> line 740, in json
>>     return json.loads(self.content.decode(encoding), **kwargs)
>>   File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line
>> 413, in loads
>>     return _default_decoder.decode(s)
>>   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
>> 402, in decode
>>     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>>   File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line
>> 418, in raw_decode
>>     obj, end = self.scan_once(s, idx)
>> JSONDecodeError: Unpaired high surrogate: line 1 column 785 (char 785)
>> ```
>>
>> Probably relevant:
>>  - https://github.com/simplejson/simplejson/issues/62
>>  - http://bugs.python.org/issue11489
>>
>> Any thoughts? What does Galaxy write to these job-associated JSON
>> metadata files?
>>
>> Thanks,
>>
>> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/