Quantcast

corrupted binary file

classic Classic list List threaded Threaded
8 messages Options
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

corrupted binary file

Yingwei HU
Dear all,

I found one of my binary files was corrupted after its uploading through "Get data'. I used editplus to check its hex code and found some "0D" is changed to "0A" and also one byte is lost. May I ask how to solve this problem?

Thank you for your help.

Best Regards,
HU Yingwei

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Peter Cock
On Fri, Feb 24, 2012 at 4:03 PM, Yingwei HU <[hidden email]> wrote:

> Dear all,
>
> I found one of my binary files was corrupted after its uploading through
> "Get data'. I used editplus to check its hex code and found some "0D" is
> changed to "0A" and also one byte is lost. May I ask how to solve this
> problem?
>
> Thank you for your help.
>
> Best Regards,
> HU Yingwei

There has been a similar discussion on the archives - those hex codes
are ASCII carriage return and line feed characters, meaning the changes
were made assuming the file was plain text to standardize the new lines.

What file type is it, and how is this defined on your Galaxy? i.e. Is it a
custom binary file type?

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Yingwei HU
Dear Peter,

Thank you for your response.
Yes. it is a custom binary file type. I attached the sample binary files.
humanK562C_d2.splib is the original one.
temp.splib is the uploaded and corrupted one.
For definition in Galaxy, I just simply inherit the binary.Binary. 
The definition is below:
In Python script:
class Splib( binary.Binary ):
        """ Class describing a spectral library (splib)"""
        file_ext = "splib"  
I did not add the sniffer but specify the file format manually in the page of "get data".

Best Regards,
HU Yingwei
2012/2/25 Peter Cock <[hidden email]>
On Fri, Feb 24, 2012 at 4:03 PM, Yingwei HU <[hidden email]> wrote:
> Dear all,
>
> I found one of my binary files was corrupted after its uploading through
> "Get data'. I used editplus to check its hex code and found some "0D" is
> changed to "0A" and also one byte is lost. May I ask how to solve this
> problem?
>
> Thank you for your help.
>
> Best Regards,
> HU Yingwei

There has been a similar discussion on the archives - those hex codes
are ASCII carriage return and line feed characters, meaning the changes
were made assuming the file was plain text to standardize the new lines.

What file type is it, and how is this defined on your Galaxy? i.e. Is it a
custom binary file type?

Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

humanK562C_d2.splib (2M) Download Attachment
temp.splib (2M) Download Attachment
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Peter Cock


On Saturday, February 25, 2012, Yingwei HU wrote:
Dear Peter,

Thank you for your response.
Yes. it is a custom binary file type. I attached the sample binary files.
humanK562C_d2.splib is the original one.
temp.splib is the uploaded and corrupted one.
For definition in Galaxy, I just simply inherit the binary.Binary. 
The definition is below:
In Python script:
class Splib( binary.Binary ):
        """ Class describing a spectral library (splib)"""
        file_ext = "splib"  
I did not add the sniffer but specify the file format manually in the page of "get data".

Which version of Galaxy (hg revision) are you using?
I'd have to go back over the email archive but I think
were some changes made to help user defined
formats.

Peter

 

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Yingwei HU
Dear Peter,

I am sorry I do not know the version of my galaxy by using hg. I directly downloaded the zip file from the address:
http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
The zipped files are compressed on 2011-06-29. Is this version too old?
Thank you for your help.

Best Regards,
HU Yingwei

2012/2/25 Peter Cock <[hidden email]>


On Saturday, February 25, 2012, Yingwei HU wrote:
Dear Peter,

Thank you for your response.
Yes. it is a custom binary file type. I attached the sample binary files.
humanK562C_d2.splib is the original one.
temp.splib is the uploaded and corrupted one.
For definition in Galaxy, I just simply inherit the binary.Binary. 
The definition is below:
In Python script:
class Splib( binary.Binary ):
        """ Class describing a spectral library (splib)"""
        file_ext = "splib"  
I did not add the sniffer but specify the file format manually in the page of "get data".

Which version of Galaxy (hg revision) are you using?
I'd have to go back over the email archive but I think
were some changes made to help user defined
formats.

Peter

 


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Peter Cock
On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <[hidden email]> wrote:
> Dear Peter,
>
> I am sorry I do not know the version of my galaxy by using hg. I directly
> downloaded the zip file from the address:
> http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip

That would be fine for a one-off test installation, but it will make
doing an upgrade extremely difficult. That is why the Galaxy
recommend using the mercurial (hg) source code system.

> The zipped files are compressed on 2011-06-29. Is this version too old?
>

The changes I am thinking of were earlier this month (Feb 2012)
by Greg on galaxy-central and have not yet been released on
galaxy-dist. Hopefully he can provide more information next week.

e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d

Regards,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Greg Von Kuster
It is highly recommended that you get the latest source from our Galaxy distribution repository using mercurial.  As Peter stated, this will enable you to get regular updates as new Galaxy distribution releases occur.  Your downloaded zip file contains what we consider a very old release.  The Galaxy distribution is available using mercurial at this url.

https://bitbucket.org/galaxy/galaxy-dist

The changes I made to the Galaxy datatypes components were implemented to provide support for proprietary datatypes contained in installed tool shed repositories, so are not related to problems that may occur when uploading.

If I understand the issue correctly, it sounds like there is a problem when you upload your file with contents that are a custom datatype which you've subclassed from the Galaxy binary datatype.  If your file is being changed by the Galaxy sniff process, then there may be a bug in that process.  What do your paster logs show?

Also, your data attachments have been lost in the mail thread - did only Peter get them?

Greg Von Kuster

On Feb 25, 2012, at 5:20 AM, Peter Cock wrote:

> On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <[hidden email]> wrote:
>> Dear Peter,
>>
>> I am sorry I do not know the version of my galaxy by using hg. I directly
>> downloaded the zip file from the address:
>> http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
>
> That would be fine for a one-off test installation, but it will make
> doing an upgrade extremely difficult. That is why the Galaxy
> recommend using the mercurial (hg) source code system.
>
>> The zipped files are compressed on 2011-06-29. Is this version too old?
>>
>
> The changes I am thinking of were earlier this month (Feb 2012)
> by Greg on galaxy-central and have not yet been released on
> galaxy-dist. Hopefully he can provide more information next week.
>
> e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d
>
> Regards,
>
> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: corrupted binary file

Yingwei HU
Dear Greg and Peter,

Thank you for your reply.
I think I had better try the newest version first. I need to ask our administrator to install mercurial in our server.
I think only Peter have the binary files now. Let me try to attach them again for you.

Best Regards,
HU Yingwei

2012/2/25 Greg Von Kuster <[hidden email]>
It is highly recommended that you get the latest source from our Galaxy distribution repository using mercurial.  As Peter stated, this will enable you to get regular updates as new Galaxy distribution releases occur.  Your downloaded zip file contains what we consider a very old release.  The Galaxy distribution is available using mercurial at this url.

https://bitbucket.org/galaxy/galaxy-dist

The changes I made to the Galaxy datatypes components were implemented to provide support for proprietary datatypes contained in installed tool shed repositories, so are not related to problems that may occur when uploading.

If I understand the issue correctly, it sounds like there is a problem when you upload your file with contents that are a custom datatype which you've subclassed from the Galaxy binary datatype.  If your file is being changed by the Galaxy sniff process, then there may be a bug in that process.  What do your paster logs show?

Also, your data attachments have been lost in the mail thread - did only Peter get them?

Greg Von Kuster

On Feb 25, 2012, at 5:20 AM, Peter Cock wrote:

> On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU <[hidden email]> wrote:
>> Dear Peter,
>>
>> I am sorry I do not know the version of my galaxy by using hg. I directly
>> downloaded the zip file from the address:
>> http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip
>
> That would be fine for a one-off test installation, but it will make
> doing an upgrade extremely difficult. That is why the Galaxy
> recommend using the mercurial (hg) source code system.
>
>> The zipped files are compressed on <a href="tel:2011-06-29" value="+85220110629" target="_blank">2011-06-29. Is this version too old?
>>
>
> The changes I am thinking of were earlier this month (Feb 2012)
> by Greg on galaxy-central and have not yet been released on
> galaxy-dist. Hopefully he can provide more information next week.
>
> e.g. https://bitbucket.org/galaxy/galaxy-central/changeset/865d998a693d
>
> Regards,
>
> Peter
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
> http://lists.bx.psu.edu/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

humanK562C_d2.splib (2M) Download Attachment
temp.splib (2M) Download Attachment
Loading...