multiple datatypes input in 16.07

classic Classic list List threaded Threaded
12 messages Options
| Threaded
Open this post in threaded view
|

multiple datatypes input in 16.07

Cristel Thomas
Hi Galaxy Devs,

We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types.
Anyone else run into this? Any idea of what's going on?

Thanks much,
Cris

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Hey Cris,

Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?

Thanks,
Sam

On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <[hidden email]> wrote:
Hi Galaxy Devs,

We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types.
Anyone else run into this? Any idea of what's going on?

Thanks much,
Cris

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Cristel Thomas
Hi Sam,
I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........)
I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes.
What I found so far on our test instance is the following:
- the problem exists with all tools on the instance, not just the few that are set to take in several datatypes.
- the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up.
- the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes.
thanks for your help!
-Cris

On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?

Thanks,
Sam

On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <[hidden email]> wrote:
Hi Galaxy Devs,

We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types.
Anyone else run into this? Any idea of what's going on?

Thanks much,
Cris

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Thanks for the quick response. What is the set of datatypes you are specifying? It is possible that one of the specified datatypes can be implicitly converted to html and text e.g. a fasta file will be listed even if only tabular has been specifed as valid datatype. In other words, Galaxy will implicitly convert the fasta file to a tabular file and present it as a valid option in the select field.



On Mon, Dec 5, 2016 at 11:48 AM, Cristel Thomas <[hidden email]> wrote:
Hi Sam,
I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........)
I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes.
What I found so far on our test instance is the following:
- the problem exists with all tools on the instance, not just the few that are set to take in several datatypes.
- the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up.
- the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes.
thanks for your help!
-Cris

On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?

Thanks,
Sam

On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <[hidden email]> wrote:
Hi Galaxy Devs,

We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types.
Anyone else run into this? Any idea of what's going on?

Thanks much,
Cris

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Cristel Thomas
there are 8 datatypes, 7 are tabular / text files, one is binary. The binary format also shows up in the dropdown.
I can send you the datatype specification python code if you'd like, Im not sure what you need to see.

On Mon, Dec 5, 2016 at 12:01 PM, Aysam Guerler <[hidden email]> wrote:
Thanks for the quick response. What is the set of datatypes you are specifying? It is possible that one of the specified datatypes can be implicitly converted to html and text e.g. a fasta file will be listed even if only tabular has been specifed as valid datatype. In other words, Galaxy will implicitly convert the fasta file to a tabular file and present it as a valid option in the select field.



On Mon, Dec 5, 2016 at 11:48 AM, Cristel Thomas <[hidden email]> wrote:
Hi Sam,
I'm not sure what tools have multiple datatypes as input on usegalaxy.org (and my files are still waiting to be uploaded for some reasons.........)
I can give you more details though. Those are tools we developed for flow cytometry with new datatypes. I wonder if the issue is with the fact that we added datatypes and I didn't do it right, although again, we don't have any issues running 16.01 with those datatypes.
What I found so far on our test instance is the following:
- the problem exists with all tools on the instance, not just the few that are set to take in several datatypes.
- the files that appear in the selection drop-down are of several datatypes. I thought it was limited to the new datatypes we have defined, but .html, .ypnb and text files are picked up although interestingly .png and .pdf don't show up.
- the problem disappears when I limit the datatypes back to one for the 3 tools that we had set as accepting several datatypes.
thanks for your help!
-Cris

On Mon, Dec 5, 2016 at 10:29 AM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Thanks for reporting this. I am looking into it. In the meanwhile, I am wondering if you can reproduce the error on usegalaxy.org? Does it happen for all tools?

Thanks,
Sam

On Mon, Dec 5, 2016 at 8:47 AM, Cristel Thomas <[hidden email]> wrote:
Hi Galaxy Devs,

We have a few tools with multiple datatypes as input. They work fine in 16.01. In 16.07 though, the selection of files in the tools' drop-downs include every single file in the history, instead of just the files from the specified data types.
Anyone else run into this? Any idea of what's going on?

Thanks much,
Cris

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/





___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Ok, so this seems to work properly, and implicit conversion is the reason why you see html and text, but not png and pdf files. Currently there is no option to disable implicit conversion but its an enhancement we can suggest to be implemented by creating a github issue.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Cristel Thomas
If it's implicit conversion, why does it pick up the FCS bianry format along with the other text formats?
and why does it get triggered on every single tool instead of just the few that have the multiple datatypes?

On Mon, Dec 5, 2016 at 12:31 PM, Aysam Guerler <[hidden email]> wrote:
Ok, so this seems to work properly, and implicit conversion is the reason why you see html and text, but not png and pdf files. Currently there is no option to disable implicit conversion but its an enhancement we can suggest to be implemented by creating a github issue.


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Ok, let me look a little bit further then. Can you send me the corresponding tool xml file and let me know which datatypes appear in the dropdown list and which of those you would not expect to see? Ideally we can reproduce it with datatypes available to everyone, so I can test it locally. 

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Hey Cris,

Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports: 


Removing them fixes the issue. The error for this is shown in the server log.

Thanks,
Sam

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Cristel Thomas
Hey Sam,
thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R).
We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted). 

In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there.

Thanks again,
Cris

On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports: 


Removing them fixes the issue. The error for this is shown in the server log.

Thanks,
Sam


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Cristel Thomas
Hi again,
ok, ran some more tests - we switched to a new dev server recently and we were having the issue because of the R/rpy2 thing. 

With that taken care of though, we still have files in the dropdown that shouldn't be there - no more FCS, but .ypnb still show up. I guess this is because of the implicit conversion?

I would love some input on what you guys think we should do about adding the FCS datatype to Galaxy.
Thanks again,
Cris

On Tue, Dec 13, 2016 at 8:52 PM, Cristel Thomas <[hidden email]> wrote:
Hey Sam,
thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R).
We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted). 

In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there.

Thanks again,
Cris

On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports: 


Removing them fixes the issue. The error for this is shown in the server log.

Thanks,
Sam



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: multiple datatypes input in 16.07

Aysam Guerler
Hi Chris,

I am glad that we figured this out. So I am not sure what the best strategy for datatypes with code dependencies is, we should open a new thread or git issue and see if someone can help with that.

Thanks,
Sam

On Wed, Dec 14, 2016 at 10:59 AM, Cristel Thomas <[hidden email]> wrote:
Hi again,
ok, ran some more tests - we switched to a new dev server recently and we were having the issue because of the R/rpy2 thing. 

With that taken care of though, we still have files in the dropdown that shouldn't be there - no more FCS, but .ypnb still show up. I guess this is because of the implicit conversion?

I would love some input on what you guys think we should do about adding the FCS datatype to Galaxy.
Thanks again,
Cris

On Tue, Dec 13, 2016 at 8:52 PM, Cristel Thomas <[hidden email]> wrote:
Hey Sam,
thanks, I'll check tomorrow at work but I suspect there might be something else - I use rpy2 to call R, so that FCS files can be read. As of 2 months ago there was no efficient way of doing it strictly in python. rpy2 does not load if R isn't installed on the system (and for this specific script, it requires flowCore installed in R).
We have R installed on our instance and still have the issues I described earlier (though I will double check tomorrow and keep you posted). 

In the long run the rpy2 thing might be an issue. We want to add our tools and the datatypes to galaxy. I'm not sure how to get around to doing it without rpy2. I'd like to keep a sniffer function in there.

Thanks again,
Cris

On Tue, Dec 13, 2016 at 1:24 PM, Aysam Guerler <[hidden email]> wrote:
Hey Cris,

Dannon and I figured out, that the problem is that your datatypes are not loaded properly, due to these two invalid imports: 


Removing them fixes the issue. The error for this is shown in the server log.

Thanks,
Sam




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/