Re: galaxy-dev Digest, Vol 33, Issue 21

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: galaxy-dev Digest, Vol 33, Issue 21

Ross-2
I may be missing something, but right now, and without any changes to
Galaxy, there's no reason why a new tool should not wrap an existing
(or new) command line executable program that uses web services to do
something interesting!

I have an rgenetics tool that uses the Haploview java executable in
command line mode to prepare image files of  LD patterns in the Hapmap
populations with an inhouse sample over a region, and then stitches
them together into a single image so they can be visually compared  -
Galaxy doesn't know or care if the tool executable uses web services -
the job runner sets up the call and then deals with the resulting tool
outputs correctly - as long as the cluster node executing the tool has
internet access, it can interact with the web services transparently -
as Greg says, the tool will be fragile to changes in the web service
call syntax but that's always a problem in the long term.

I agree with Greg that bringing web services interfaces *directly*
into the existing Galaxy framework does not appear to have a high
priority at this time and I'm not even sure that it makes an awful lot
of sense in the long term - especially since we can let web service
interaction live out in tool space where it already works? Is there a
specific use-case anyone has in mind where the web services interface
really needs to be brought inside the framework itself?


On Tue, Sep 30, 2008 at 2:00 AM,  <[hidden email]> wrote:

>
> Today's Topics:
>
>   1. Re: Galaxy with web services (Greg Von Kuster)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 29 Sep 2008 09:52:39 -0400
> From: Greg Von Kuster <[hidden email]>
> Subject: Re: [galaxy-dev] Galaxy with web services
> To: Praveen Agrawal <[hidden email]>
> Cc: [hidden email]
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hello Praveen,
>
> Currently web-based interaction is only used for external data sources
> (biomart, UCSC table browser, most other things under "Get Data").  For
> the other tools, we generate a command line based on the user input,
> which Galaxy then runs.  The motivation for this is control and trust.
> One of our major goals with Galaxy is to ensure reproducibility, and
> relying on web services which can change their interfaces or behavior
> makes this very challenging.  We are certainly interested in supporting
> web services in the Galaxy framework, but implementation is not yet
> under way.  Several of the methods currently in ~/tools/__init__.py (
> among others ) will need to evolve and grow in order to support a web
> services interface for remote tools.  We'll certainly keep you informed
> as we begin to support this, but it may be a while...
>
> Greg Von Kuster
> Galaxy Development Team
>
>
> Praveen Agrawal wrote:
>> Hi,
>> I am trying to use Galaxy at my site and cann't figure out Galaxy with web
>> services. I understand that it's possible to integrate a data source using
>> WSDL/SOAP. Is the same possible for a remote tool as well?



--
python -c "foo = map(None,'[hidden email]'); foo.reverse();
print ''.join(foo)"

Reply | Threaded
Open this post in threaded view
|

Re: galaxy-dev Digest, Vol 33, Issue 21

Praveen Agrawal

Hi Ross, Hi Greg,
Thanks for the elaborate answers. Certainly, having programs installed
locally gives lot more control over their execution.

In my case I am going to have tools which would be running across Indian
"grid" and then the scientist would have the freedom to use the workflow
capability to join those tools. I couldn't understand "we can let web
service interaction live out in tool space where it already works". Please
elaborate.

Thanks,
Praveen

> I may be missing something, but right now, and without any changes to
> Galaxy, there's no reason why a new tool should not wrap an existing
> (or new) command line executable program that uses web services to do
> something interesting!
>
> I have an rgenetics tool that uses the Haploview java executable in
> command line mode to prepare image files of  LD patterns in the Hapmap
> populations with an inhouse sample over a region, and then stitches
> them together into a single image so they can be visually compared  -
> Galaxy doesn't know or care if the tool executable uses web services -
> the job runner sets up the call and then deals with the resulting tool
> outputs correctly - as long as the cluster node executing the tool has
> internet access, it can interact with the web services transparently -
> as Greg says, the tool will be fragile to changes in the web service
> call syntax but that's always a problem in the long term.
>
> I agree with Greg that bringing web services interfaces *directly*
> into the existing Galaxy framework does not appear to have a high
> priority at this time and I'm not even sure that it makes an awful lot
> of sense in the long term - especially since we can let web service
> interaction live out in tool space where it already works? Is there a
> specific use-case anyone has in mind where the web services interface
> really needs to be brought inside the framework itself?
>
>
> On Tue, Sep 30, 2008 at 2:00 AM,  <[hidden email]> wrote:
>
>>
>> Today's Topics:
>>
>>   1. Re: Galaxy with web services (Greg Von Kuster)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Mon, 29 Sep 2008 09:52:39 -0400
>> From: Greg Von Kuster <[hidden email]>
>> Subject: Re: [galaxy-dev] Galaxy with web services
>> To: Praveen Agrawal <[hidden email]>
>> Cc: [hidden email]
>> Message-ID: <[hidden email]>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> Hello Praveen,
>>
>> Currently web-based interaction is only used for external data sources
>> (biomart, UCSC table browser, most other things under "Get Data").  For
>> the other tools, we generate a command line based on the user input,
>> which Galaxy then runs.  The motivation for this is control and trust.
>> One of our major goals with Galaxy is to ensure reproducibility, and
>> relying on web services which can change their interfaces or behavior
>> makes this very challenging.  We are certainly interested in supporting
>> web services in the Galaxy framework, but implementation is not yet
>> under way.  Several of the methods currently in ~/tools/__init__.py (
>> among others ) will need to evolve and grow in order to support a web
>> services interface for remote tools.  We'll certainly keep you informed
>> as we begin to support this, but it may be a while...
>>
>> Greg Von Kuster
>> Galaxy Development Team
>>
>>
>> Praveen Agrawal wrote:
>>> Hi,
>>> I am trying to use Galaxy at my site and cann't figure out Galaxy with
>>> web
>>> services. I understand that it's possible to integrate a data source
>>> using
>>> WSDL/SOAP. Is the same possible for a remote tool as well?
>
>
>
> --
> python -c "foo = map(None,'[hidden email]'); foo.reverse();
> print ''.join(foo)"
>


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
http://www.scfbio-iitd.res.in
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Reply | Threaded
Open this post in threaded view
|

Re: galaxy-dev Digest, Vol 33, Issue 21

Ross-2
On Tue, Sep 30, 2008 at 6:32 PM, Praveen Agrawal
<[hidden email]> wrote:

>
> Hi Ross, Hi Greg,
> Thanks for the elaborate answers. Certainly, having programs installed
> locally gives lot more control over their execution.
>
> In my case I am going to have tools which would be running across Indian
> "grid" and then the scientist would have the freedom to use the workflow
> capability to join those tools. I couldn't understand "we can let web
> service interaction live out in tool space where it already works". Please
> elaborate.

I'm trying to say that changes to the core of Galaxy to allow (eg)
remote web services interaction that Greg mentioned may eventually
appear, but you can already make use of existing web services in new
tools that you write for Galaxy.

The Galaxy framework already allows you to wrap a web services based
application into a tool - like the Haploview example I mentioned. From
the point of view of the user (and Galaxy!), it's just another Galaxy
tool - it happens to use web services (eg xmlrpc or SOAP) and they are
being executed out in "tool space" - meaning that the Galaxy framework
doesn't know or care about the web services calls made by whatever
executable the tool has invoked.

In other words, there's nothing stopping you writing tools that use
web services exposed on the Indian grid or any other web services -
you don't need any changes to the Galaxy core - in the example I gave,
we have a Java executable (haploview) which uses xml-rpc to talk to
the hapmap web site, wrapped into a Galaxy tool (using Python code
that I wrote to do some tricks with the Haploview output files and
images).

If you're interested in seeing how I made it work, take a look at
http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.py
(and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.xml
and http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView_code.py)
The code is brutally ugly but works pretty well.

I hope this helps?

> Thanks,
> Praveen
>
>> I may be missing something, but right now, and without any changes to
>> Galaxy, there's no reason why a new tool should not wrap an existing
>> (or new) command line executable program that uses web services to do
>> something interesting!
>>
>> I have an rgenetics tool that uses the Haploview java executable in
>> command line mode to prepare image files of  LD patterns in the Hapmap
>> populations with an inhouse sample over a region, and then stitches
>> them together into a single image so they can be visually compared  -
>> Galaxy doesn't know or care if the tool executable uses web services -
>> the job runner sets up the call and then deals with the resulting tool
>> outputs correctly - as long as the cluster node executing the tool has
>> internet access, it can interact with the web services transparently -
>> as Greg says, the tool will be fragile to changes in the web service
>> call syntax but that's always a problem in the long term.
>>
>> I agree with Greg that bringing web services interfaces *directly*
>> into the existing Galaxy framework does not appear to have a high
>> priority at this time and I'm not even sure that it makes an awful lot
>> of sense in the long term - especially since we can let web service
>> interaction live out in tool space where it already works? Is there a
>> specific use-case anyone has in mind where the web services interface
>> really needs to be brought inside the framework itself?
>>
>>
>> On Tue, Sep 30, 2008 at 2:00 AM,  <[hidden email]> wrote:
>>
>>>
>>> Today's Topics:
>>>
>>>   1. Re: Galaxy with web services (Greg Von Kuster)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Mon, 29 Sep 2008 09:52:39 -0400
>>> From: Greg Von Kuster <[hidden email]>
>>> Subject: Re: [galaxy-dev] Galaxy with web services
>>> To: Praveen Agrawal <[hidden email]>
>>> Cc: [hidden email]
>>> Message-ID: <[hidden email]>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> Hello Praveen,
>>>
>>> Currently web-based interaction is only used for external data sources
>>> (biomart, UCSC table browser, most other things under "Get Data").  For
>>> the other tools, we generate a command line based on the user input,
>>> which Galaxy then runs.  The motivation for this is control and trust.
>>> One of our major goals with Galaxy is to ensure reproducibility, and
>>> relying on web services which can change their interfaces or behavior
>>> makes this very challenging.  We are certainly interested in supporting
>>> web services in the Galaxy framework, but implementation is not yet
>>> under way.  Several of the methods currently in ~/tools/__init__.py (
>>> among others ) will need to evolve and grow in order to support a web
>>> services interface for remote tools.  We'll certainly keep you informed
>>> as we begin to support this, but it may be a while...
>>>
>>> Greg Von Kuster
>>> Galaxy Development Team
>>>
>>>
>>> Praveen Agrawal wrote:
>>>> Hi,
>>>> I am trying to use Galaxy at my site and cann't figure out Galaxy with
>>>> web
>>>> services. I understand that it's possible to integrate a data source
>>>> using
>>>> WSDL/SOAP. Is the same possible for a remote tool as well?
>>
>>
>>
>> --
>> python -c "foo = map(None,'[hidden email]'); foo.reverse();
>> print ''.join(foo)"
>>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> http://www.scfbio-iitd.res.in
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>



--
python -c "foo = map(None,'[hidden email]'); foo.reverse();
print ''.join(foo)"

Reply | Threaded
Open this post in threaded view
|

Re: galaxy-dev Digest, Vol 33, Issue 21

Praveen Agrawal

Thanks Ross, that solves my purpose.

Regards,
Praveen

> On Tue, Sep 30, 2008 at 6:32 PM, Praveen Agrawal
> <[hidden email]> wrote:
>>
>> Hi Ross, Hi Greg,
>> Thanks for the elaborate answers. Certainly, having programs installed
>> locally gives lot more control over their execution.
>>
>> In my case I am going to have tools which would be running across Indian
>> "grid" and then the scientist would have the freedom to use the workflow
>> capability to join those tools. I couldn't understand "we can let web
>> service interaction live out in tool space where it already works".
>> Please
>> elaborate.
>
> I'm trying to say that changes to the core of Galaxy to allow (eg)
> remote web services interaction that Greg mentioned may eventually
> appear, but you can already make use of existing web services in new
> tools that you write for Galaxy.
>
> The Galaxy framework already allows you to wrap a web services based
> application into a tool - like the Haploview example I mentioned. From
> the point of view of the user (and Galaxy!), it's just another Galaxy
> tool - it happens to use web services (eg xmlrpc or SOAP) and they are
> being executed out in "tool space" - meaning that the Galaxy framework
> doesn't know or care about the web services calls made by whatever
> executable the tool has invoked.
>
> In other words, there's nothing stopping you writing tools that use
> web services exposed on the Indian grid or any other web services -
> you don't need any changes to the Galaxy core - in the example I gave,
> we have a Java executable (haploview) which uses xml-rpc to talk to
> the hapmap web site, wrapped into a Galaxy tool (using Python code
> that I wrote to do some tricks with the Haploview output files and
> images).
>
> If you're interested in seeing how I made it work, take a look at
> http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.py
> (and
> http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView.xml
> and
> http://esphealth.org/trac/rgalaxy/browser/trunk/tools/rgenetics/rgHaploView_code.py)
> The code is brutally ugly but works pretty well.
>
> I hope this helps?
>
>> Thanks,
>> Praveen
>>
>>> I may be missing something, but right now, and without any changes to
>>> Galaxy, there's no reason why a new tool should not wrap an existing
>>> (or new) command line executable program that uses web services to do
>>> something interesting!
>>>
>>> I have an rgenetics tool that uses the Haploview java executable in
>>> command line mode to prepare image files of  LD patterns in the Hapmap
>>> populations with an inhouse sample over a region, and then stitches
>>> them together into a single image so they can be visually compared  -
>>> Galaxy doesn't know or care if the tool executable uses web services -
>>> the job runner sets up the call and then deals with the resulting tool
>>> outputs correctly - as long as the cluster node executing the tool has
>>> internet access, it can interact with the web services transparently -
>>> as Greg says, the tool will be fragile to changes in the web service
>>> call syntax but that's always a problem in the long term.
>>>
>>> I agree with Greg that bringing web services interfaces *directly*
>>> into the existing Galaxy framework does not appear to have a high
>>> priority at this time and I'm not even sure that it makes an awful lot
>>> of sense in the long term - especially since we can let web service
>>> interaction live out in tool space where it already works? Is there a
>>> specific use-case anyone has in mind where the web services interface
>>> really needs to be brought inside the framework itself?
>>>
>>>
>>> On Tue, Sep 30, 2008 at 2:00 AM,  <[hidden email]>
>>> wrote:
>>>
>>>>
>>>> Today's Topics:
>>>>
>>>>   1. Re: Galaxy with web services (Greg Von Kuster)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Mon, 29 Sep 2008 09:52:39 -0400
>>>> From: Greg Von Kuster <[hidden email]>
>>>> Subject: Re: [galaxy-dev] Galaxy with web services
>>>> To: Praveen Agrawal <[hidden email]>
>>>> Cc: [hidden email]
>>>> Message-ID: <[hidden email]>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>
>>>> Hello Praveen,
>>>>
>>>> Currently web-based interaction is only used for external data sources
>>>> (biomart, UCSC table browser, most other things under "Get Data").
>>>> For
>>>> the other tools, we generate a command line based on the user input,
>>>> which Galaxy then runs.  The motivation for this is control and trust.
>>>> One of our major goals with Galaxy is to ensure reproducibility, and
>>>> relying on web services which can change their interfaces or behavior
>>>> makes this very challenging.  We are certainly interested in
>>>> supporting
>>>> web services in the Galaxy framework, but implementation is not yet
>>>> under way.  Several of the methods currently in ~/tools/__init__.py (
>>>> among others ) will need to evolve and grow in order to support a web
>>>> services interface for remote tools.  We'll certainly keep you
>>>> informed
>>>> as we begin to support this, but it may be a while...
>>>>
>>>> Greg Von Kuster
>>>> Galaxy Development Team
>>>>
>>>>
>>>> Praveen Agrawal wrote:
>>>>> Hi,
>>>>> I am trying to use Galaxy at my site and cann't figure out Galaxy
>>>>> with
>>>>> web
>>>>> services. I understand that it's possible to integrate a data source
>>>>> using
>>>>> WSDL/SOAP. Is the same possible for a remote tool as well?
>>>
>>>
>>>
>>> --
>>> python -c "foo = map(None,'[hidden email]'); foo.reverse();
>>> print ''.join(foo)"
>>>
>>
>>
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> http://www.scfbio-iitd.res.in
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>
>
>
> --
> python -c "foo = map(None,'[hidden email]'); foo.reverse();
> print ''.join(foo)"
>


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
http://www.scfbio-iitd.res.in
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~