Partial automation for generating those twisty R dependency tool shed installation sequences

classic Classic list List threaded Threaded
8 messages Options
| Threaded
Open this post in threaded view
|

Partial automation for generating those twisty R dependency tool shed installation sequences

Ross-2
This may be helpful for anyone else struggling to get complex nested R package dependency installation from the tool shed sorted out. That whole can of worms. While we have setup_r_packages, the developer still has to figure out the right magical incantation and make sure the tarballs are available.

https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've started - contribitions welcome. 

It has a more or less reusable R script to generate tool_dependencies.xml boilerplate, assuming you set the constant libdir to your local git repository path where those tarballs will be downloaded from.

I hope this helps someone!

Could make a tool to do this if enough developers want access to it without the pain of managing yet another R script?


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Björn Grüning-3
Hi Ross,

this is great!
Have you seen this notebook?

http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb

It tries to do the same thing. Maybe it's also worth to mention? Maybe
we can enhance it?

Thanks,
Bjoern

Am 08.01.2015 um 08:09 schrieb Ross:

> This may be helpful for anyone else struggling to get complex nested R
> package dependency installation from the tool shed sorted out. That whole
> can of worms. While we have setup_r_packages, the developer still has to
> figure out the right magical incantation and make sure the tarballs are
> available.
>
> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
> started - contribitions welcome.
>
> It has a more or less reusable R script to generate tool_dependencies.xml
> boilerplate, assuming you set the constant libdir to your local git
> repository path where those tarballs will be downloaded from.
>
> I hope this helps someone!
>
> Could make a tool to do this if enough developers want access to it without
> the pain of managing yet another R script?
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Ross-2
Hi, Björn,
Looks pretty similar!
Aren't the links your notebook generates transient? I think if you put them into a tool_dependencies.xml, they will fail permanently immediately after any of the package authors updates one of the relevant svn repositories?
 
AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a link that works today like http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp next gets updated and Rcpp_0.11.3.tar.gz is migrated to http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement (eg) http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the contrib directory?

That's why my more complex script downloads all the latest archives into my local github archive repo and generates a permanent link to suit that github repo.
We definitely need an automated solution as this is a really infuriating aspect of trying to make code relying on R/BioC packages reproducible.


On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <[hidden email]> wrote:
Hi Ross,

this is great!
Have you seen this notebook?

http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb

It tries to do the same thing. Maybe it's also worth to mention? Maybe
we can enhance it?

Thanks,
Bjoern

Am 08.01.2015 um 08:09 schrieb Ross:
> This may be helpful for anyone else struggling to get complex nested R
> package dependency installation from the tool shed sorted out. That whole
> can of worms. While we have setup_r_packages, the developer still has to
> figure out the right magical incantation and make sure the tarballs are
> available.
>
> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
> started - contribitions welcome.
>
> It has a more or less reusable R script to generate tool_dependencies.xml
> boilerplate, assuming you set the constant libdir to your local git
> repository path where those tarballs will be downloaded from.
>
> I hope this helps someone!
>
> Could make a tool to do this if enough developers want access to it without
> the pain of managing yet another R script?
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Björn Grüning-3
Hi Ross,

you are absolutely right.
My download_store repository is exactly for this purpose.

https://github.com/bgruening/download_store

If you are interested we could integrate your additional magic into the
notebook.

Thanks,
Bjoern

Am 11.01.2015 um 01:33 schrieb Ross:

> Hi, Björn,
> Looks pretty similar!
> Aren't the links your notebook generates transient? I think if you put them
> into a tool_dependencies.xml, they will fail permanently immediately after
> any of the package authors updates one of the relevant svn repositories?
>
> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a
> link that works today like
> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp
> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement (eg)
> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
> contrib directory?
>
> That's why my more complex script downloads all the latest archives into my
> local github archive repo and generates a permanent link to suit that
> github repo.
> We definitely need an automated solution as this is a really infuriating
> aspect of trying to make code relying on R/BioC packages reproducible.
>
>
> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <[hidden email]>
> wrote:
>
>> Hi Ross,
>>
>> this is great!
>> Have you seen this notebook?
>>
>>
>> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
>>
>> It tries to do the same thing. Maybe it's also worth to mention? Maybe
>> we can enhance it?
>>
>> Thanks,
>> Bjoern
>>
>> Am 08.01.2015 um 08:09 schrieb Ross:
>>> This may be helpful for anyone else struggling to get complex nested R
>>> package dependency installation from the tool shed sorted out. That whole
>>> can of worms. While we have setup_r_packages, the developer still has to
>>> figure out the right magical incantation and make sure the tarballs are
>>> available.
>>>
>>> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
>>> started - contribitions welcome.
>>>
>>> It has a more or less reusable R script to generate tool_dependencies.xml
>>> boilerplate, assuming you set the constant libdir to your local git
>>> repository path where those tarballs will be downloaded from.
>>>
>>> I hope this helps someone!
>>>
>>> Could make a tool to do this if enough developers want access to it
>> without
>>> the pain of managing yet another R script?
>>>
>>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Ross-2
Hi  Björn,
I'm a bit old fashioned and think I prefer a proper Galaxy tool rather than a notebook :) so I've set up a temporary demonstration/test site of a toolfactory generated tool that does what I think I need - can some kind soul please test it and let me know how it goes ? If it's useful, it needs to be adjusted to depend on whatever version of package_R you want to work with - currently just uses the system R for demonstration purposes.

I used the toolfactory2 (main toolshed) (which now allows any number of (optionally non editable) parameters!!!) to wrap the script shown at the bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are currently three parameters - the names of the R/BioC packages from sessionInfo(), the local directory where all the tarballs should be stowed and the XML output prefix to prepend to each row of the generated XML stanza for tool_dependencies.xml

The resulting toolshed tarball was uploaded to a local toolshed and then installed to produce a new tool in the "tool generators" section - r_bioc_depgen Generate dependencies for R/BioC packages

If you import the history at http://130.56.252.21/history/list_published you will see the toolfactory job (#1,#2,#3) - rerunning will show how the parameters are defined - fugly but it does work.
After generating/uploading/installing the new tool, outputs from a test run are in #4 and #5 for DESeq

Comments and suggestions welcomed!

On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <[hidden email]> wrote:
Hi Ross,

you are absolutely right.
My download_store repository is exactly for this purpose.

https://github.com/bgruening/download_store

If you are interested we could integrate your additional magic into the
notebook.

Thanks,
Bjoern

Am 11.01.2015 um 01:33 schrieb Ross:
> Hi, Björn,
> Looks pretty similar!
> Aren't the links your notebook generates transient? I think if you put them
> into a tool_dependencies.xml, they will fail permanently immediately after
> any of the package authors updates one of the relevant svn repositories?
>
> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a
> link that works today like
> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp
> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement (eg)
> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
> contrib directory?
>
> That's why my more complex script downloads all the latest archives into my
> local github archive repo and generates a permanent link to suit that
> github repo.
> We definitely need an automated solution as this is a really infuriating
> aspect of trying to make code relying on R/BioC packages reproducible.
>
>
> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <[hidden email]>
> wrote:
>
>> Hi Ross,
>>
>> this is great!
>> Have you seen this notebook?
>>
>>
>> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
>>
>> It tries to do the same thing. Maybe it's also worth to mention? Maybe
>> we can enhance it?
>>
>> Thanks,
>> Bjoern
>>
>> Am 08.01.2015 um 08:09 schrieb Ross:
>>> This may be helpful for anyone else struggling to get complex nested R
>>> package dependency installation from the tool shed sorted out. That whole
>>> can of worms. While we have setup_r_packages, the developer still has to
>>> figure out the right magical incantation and make sure the tarballs are
>>> available.
>>>
>>> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
>>> started - contribitions welcome.
>>>
>>> It has a more or less reusable R script to generate tool_dependencies.xml
>>> boilerplate, assuming you set the constant libdir to your local git
>>> repository path where those tarballs will be downloaded from.
>>>
>>> I hope this helps someone!
>>>
>>> Could make a tool to do this if enough developers want access to it
>> without
>>> the pain of managing yet another R script?
>>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Björn Grüning-3
Hi Ross,

Am 13.01.2015 um 00:13 schrieb Ross:
> Hi  Björn,
> I'm a bit old fashioned and think I prefer a proper Galaxy tool rather than
> a notebook :)

:)

> So I've set up a temporary demonstration/test site of a
> toolfactory generated tool that does what I think I need - can some kind
> soul please test it and let me know how it goes ? If it's useful, it needs
> to be adjusted to depend on whatever version of package_R you want to work
> with - currently just uses the system R for demonstration purposes.
>
> I used the toolfactory2 (main toolshed) (which now allows any number of
> (optionally non editable) parameters!!!) to wrap the script shown at the
> bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are
> currently three parameters - the names of the R/BioC packages from
> sessionInfo(), the local directory where all the tarballs should be stowed
> and the XML output prefix to prepend to each row of the generated XML
> stanza for tool_dependencies.xml
>
> The resulting toolshed tarball was uploaded to a local toolshed and then
> installed to produce a new tool in the "tool generators" section
> - r_bioc_depgen Generate dependencies for R/BioC packages
>
> If you import the history at http://130.56.252.21/history/list_published
> you will see the toolfactory job (#1,#2,#3) - rerunning will show how the
> parameters are defined - fugly but it does work.
> After generating/uploading/installing the new tool, outputs from a test run
> are in #4 and #5 for DESeq

This is cool!
I have to try this on a few packages!
This could also be of interest for our GSOC idea:
https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations

Thanks Ross for working on this!
Bjoern

> Comments and suggestions welcomed!
>
> On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <[hidden email]>
> wrote:
>
>> Hi Ross,
>>
>> you are absolutely right.
>> My download_store repository is exactly for this purpose.
>>
>> https://github.com/bgruening/download_store
>>
>> If you are interested we could integrate your additional magic into the
>> notebook.
>>
>> Thanks,
>> Bjoern
>>
>> Am 11.01.2015 um 01:33 schrieb Ross:
>>> Hi, Björn,
>>> Looks pretty similar!
>>> Aren't the links your notebook generates transient? I think if you put
>> them
>>> into a tool_dependencies.xml, they will fail permanently immediately
>> after
>>> any of the package authors updates one of the relevant svn repositories?
>>>
>>> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a
>>> link that works today like
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp
>>> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
>>> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement
>> (eg)
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
>>> contrib directory?
>>>
>>> That's why my more complex script downloads all the latest archives into
>> my
>>> local github archive repo and generates a permanent link to suit that
>>> github repo.
>>> We definitely need an automated solution as this is a really infuriating
>>> aspect of trying to make code relying on R/BioC packages reproducible.
>>>
>>>
>>> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <
>> [hidden email]>
>>> wrote:
>>>
>>>> Hi Ross,
>>>>
>>>> this is great!
>>>> Have you seen this notebook?
>>>>
>>>>
>>>>
>> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
>>>>
>>>> It tries to do the same thing. Maybe it's also worth to mention? Maybe
>>>> we can enhance it?
>>>>
>>>> Thanks,
>>>> Bjoern
>>>>
>>>> Am 08.01.2015 um 08:09 schrieb Ross:
>>>>> This may be helpful for anyone else struggling to get complex nested R
>>>>> package dependency installation from the tool shed sorted out. That
>> whole
>>>>> can of worms. While we have setup_r_packages, the developer still has
>> to
>>>>> figure out the right magical incantation and make sure the tarballs are
>>>>> available.
>>>>>
>>>>> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
>>>>> started - contribitions welcome.
>>>>>
>>>>> It has a more or less reusable R script to generate
>> tool_dependencies.xml
>>>>> boilerplate, assuming you set the constant libdir to your local git
>>>>> repository path where those tarballs will be downloaded from.
>>>>>
>>>>> I hope this helps someone!
>>>>>
>>>>> Could make a tool to do this if enough developers want access to it
>>>> without
>>>>> the pain of managing yet another R script?
>>>>>
>>>>
>>>
>>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Ross-2
Hi Bjorn,
Thanks for caring :)

R/Bioc dependencies are determined by the target R package.
This realisation made me change tool factory 2 in the test toolshed so it now allows R (or perl or python - not bash!) dependency package_r_ 
selection. It still allows the local system R but that's not recommended for anything reproducible.
The generated tool will always use that R and so I can generate an RBioC dependency generator using that script https://wiki.galaxyproject.org/SetUpREnvironment for any available toolshed package_r_
Feedback always appreciated...


On Sun, Jan 18, 2015 at 9:25 PM, Björn Grüning <[hidden email]> wrote:
Hi Ross,

Am 13.01.2015 um 00:13 schrieb Ross:
> Hi  Björn,
> I'm a bit old fashioned and think I prefer a proper Galaxy tool rather than
> a notebook :)

:)

> So I've set up a temporary demonstration/test site of a
> toolfactory generated tool that does what I think I need - can some kind
> soul please test it and let me know how it goes ? If it's useful, it needs
> to be adjusted to depend on whatever version of package_R you want to work
> with - currently just uses the system R for demonstration purposes.
>
> I used the toolfactory2 (main toolshed) (which now allows any number of
> (optionally non editable) parameters!!!) to wrap the script shown at the
> bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are
> currently three parameters - the names of the R/BioC packages from
> sessionInfo(), the local directory where all the tarballs should be stowed
> and the XML output prefix to prepend to each row of the generated XML
> stanza for tool_dependencies.xml
>
> The resulting toolshed tarball was uploaded to a local toolshed and then
> installed to produce a new tool in the "tool generators" section
> - r_bioc_depgen Generate dependencies for R/BioC packages
>
> If you import the history at http://130.56.252.21/history/list_published
> you will see the toolfactory job (#1,#2,#3) - rerunning will show how the
> parameters are defined - fugly but it does work.
> After generating/uploading/installing the new tool, outputs from a test run
> are in #4 and #5 for DESeq

This is cool!
I have to try this on a few packages!
This could also be of interest for our GSOC idea:
https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations

Thanks Ross for working on this!
Bjoern

> Comments and suggestions welcomed!
>
> On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <[hidden email]>
> wrote:
>
>> Hi Ross,
>>
>> you are absolutely right.
>> My download_store repository is exactly for this purpose.
>>
>> https://github.com/bgruening/download_store
>>
>> If you are interested we could integrate your additional magic into the
>> notebook.
>>
>> Thanks,
>> Bjoern
>>
>> Am 11.01.2015 um 01:33 schrieb Ross:
>>> Hi, Björn,
>>> Looks pretty similar!
>>> Aren't the links your notebook generates transient? I think if you put
>> them
>>> into a tool_dependencies.xml, they will fail permanently immediately
>> after
>>> any of the package authors updates one of the relevant svn repositories?
>>>
>>> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a
>>> link that works today like
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp
>>> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
>>> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement
>> (eg)
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
>>> contrib directory?
>>>
>>> That's why my more complex script downloads all the latest archives into
>> my
>>> local github archive repo and generates a permanent link to suit that
>>> github repo.
>>> We definitely need an automated solution as this is a really infuriating
>>> aspect of trying to make code relying on R/BioC packages reproducible.
>>>
>>>
>>> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <
>> [hidden email]>
>>> wrote:
>>>
>>>> Hi Ross,
>>>>
>>>> this is great!
>>>> Have you seen this notebook?
>>>>
>>>>
>>>>
>> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
>>>>
>>>> It tries to do the same thing. Maybe it's also worth to mention? Maybe
>>>> we can enhance it?
>>>>
>>>> Thanks,
>>>> Bjoern
>>>>
>>>> Am 08.01.2015 um 08:09 schrieb Ross:
>>>>> This may be helpful for anyone else struggling to get complex nested R
>>>>> package dependency installation from the tool shed sorted out. That
>> whole
>>>>> can of worms. While we have setup_r_packages, the developer still has
>> to
>>>>> figure out the right magical incantation and make sure the tarballs are
>>>>> available.
>>>>>
>>>>> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
>>>>> started - contribitions welcome.
>>>>>
>>>>> It has a more or less reusable R script to generate
>> tool_dependencies.xml
>>>>> boilerplate, assuming you set the constant libdir to your local git
>>>>> repository path where those tarballs will be downloaded from.
>>>>>
>>>>> I hope this helps someone!
>>>>>
>>>>> Could make a tool to do this if enough developers want access to it
>>>> without
>>>>> the pain of managing yet another R script?
>>>>>
>>>>
>>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Partial automation for generating those twisty R dependency tool shed installation sequences

Ross-2
In case anyone needs to prepare a complete set of twisty R/BioC dependencies in the right order for automated toolshed installation, https://testtoolshed.g2.bx.psu.edu/view/fubar/biocdepgen_r_3_1_2 contains a dependency package generator for the iuc package_r_3_1_2 - eg for DESeq2, you end up with 46 dependent tarballs in addition to DESeq2 itself !!!  

Please, DO NOT install this on a public instance - like the tool factory 2 it's intended only for tool development on throw away instances.

Not much documentation - set the tardir to where you want the tarballs saved, the XML and path prefix needed before the tarball name in tool_dependencies.xml to point to where the toolshed should download those packages from (I use a github repo eg) and the names of all the R/BioC packages showing in sessionInfo() as "other attached packages" after your R script has run successfully.
 
I hope this helps minimise the pain for other tool developers. 
Let me know how you go, or if anyone wants specific versions for other r base packages please.



On Tue, Jan 20, 2015 at 8:06 PM, Ross <[hidden email]> wrote:
Hi Bjorn,
Thanks for caring :)

R/Bioc dependencies are determined by the target R package.
This realisation made me change tool factory 2 in the test toolshed so it now allows R (or perl or python - not bash!) dependency package_r_ 
selection. It still allows the local system R but that's not recommended for anything reproducible.
The generated tool will always use that R and so I can generate an RBioC dependency generator using that script https://wiki.galaxyproject.org/SetUpREnvironment for any available toolshed package_r_
Feedback always appreciated...


On Sun, Jan 18, 2015 at 9:25 PM, Björn Grüning <[hidden email]> wrote:
Hi Ross,

Am 13.01.2015 um 00:13 schrieb Ross:
> Hi  Björn,
> I'm a bit old fashioned and think I prefer a proper Galaxy tool rather than
> a notebook :)

:)

> So I've set up a temporary demonstration/test site of a
> toolfactory generated tool that does what I think I need - can some kind
> soul please test it and let me know how it goes ? If it's useful, it needs
> to be adjusted to depend on whatever version of package_R you want to work
> with - currently just uses the system R for demonstration purposes.
>
> I used the toolfactory2 (main toolshed) (which now allows any number of
> (optionally non editable) parameters!!!) to wrap the script shown at the
> bottom of https://wiki.galaxyproject.org/SetUpREnvironment. There are
> currently three parameters - the names of the R/BioC packages from
> sessionInfo(), the local directory where all the tarballs should be stowed
> and the XML output prefix to prepend to each row of the generated XML
> stanza for tool_dependencies.xml
>
> The resulting toolshed tarball was uploaded to a local toolshed and then
> installed to produce a new tool in the "tool generators" section
> - r_bioc_depgen Generate dependencies for R/BioC packages
>
> If you import the history at http://130.56.252.21/history/list_published
> you will see the toolfactory job (#1,#2,#3) - rerunning will show how the
> parameters are defined - fugly but it does work.
> After generating/uploading/installing the new tool, outputs from a test run
> are in #4 and #5 for DESeq

This is cool!
I have to try this on a few packages!
This could also be of interest for our GSOC idea:
https://wiki.galaxyproject.org/Develop/GSOC/2015Ideas#Fostering_Bioconductor_Collaborations

Thanks Ross for working on this!
Bjoern

> Comments and suggestions welcomed!
>
> On Sun, Jan 11, 2015 at 10:41 PM, Björn Grüning <[hidden email]>
> wrote:
>
>> Hi Ross,
>>
>> you are absolutely right.
>> My download_store repository is exactly for this purpose.
>>
>> https://github.com/bgruening/download_store
>>
>> If you are interested we could integrate your additional magic into the
>> notebook.
>>
>> Thanks,
>> Bjoern
>>
>> Am 11.01.2015 um 01:33 schrieb Ross:
>>> Hi, Björn,
>>> Looks pretty similar!
>>> Aren't the links your notebook generates transient? I think if you put
>> them
>>> into a tool_dependencies.xml, they will fail permanently immediately
>> after
>>> any of the package authors updates one of the relevant svn repositories?
>>>
>>> AFAIK, it looks like the whole BioC/CRAN infrastructure is automated so a
>>> link that works today like
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.3.tar.gz will fail when Rcpp
>>> next gets updated and Rcpp_0.11.3.tar.gz is migrated to
>>> http://cran.fhcrc.org/src/contrib/00Archive/Rcpp/ with a replacement
>> (eg)
>>> http://cran.fhcrc.org/src/contrib/Rcpp_0.11.4.tar.gz appearing in the
>>> contrib directory?
>>>
>>> That's why my more complex script downloads all the latest archives into
>> my
>>> local github archive repo and generates a permanent link to suit that
>>> github repo.
>>> We definitely need an automated solution as this is a really infuriating
>>> aspect of trying to make code relying on R/BioC packages reproducible.
>>>
>>>
>>> On Thu, Jan 8, 2015 at 11:28 PM, Björn Grüning <
>> [hidden email]>
>>> wrote:
>>>
>>>> Hi Ross,
>>>>
>>>> this is great!
>>>> Have you seen this notebook?
>>>>
>>>>
>>>>
>> http://nbviewer.ipython.org/github/bgruening/notebooks/blob/master/R/extract_all_dependencies_from_an_r_package.ipynb
>>>>
>>>> It tries to do the same thing. Maybe it's also worth to mention? Maybe
>>>> we can enhance it?
>>>>
>>>> Thanks,
>>>> Bjoern
>>>>
>>>> Am 08.01.2015 um 08:09 schrieb Ross:
>>>>> This may be helpful for anyone else struggling to get complex nested R
>>>>> package dependency installation from the tool shed sorted out. That
>> whole
>>>>> can of worms. While we have setup_r_packages, the developer still has
>> to
>>>>> figure out the right magical incantation and make sure the tarballs are
>>>>> available.
>>>>>
>>>>> https://wiki.galaxyproject.org/SetUpREnvironment has some notes I've
>>>>> started - contribitions welcome.
>>>>>
>>>>> It has a more or less reusable R script to generate
>> tool_dependencies.xml
>>>>> boilerplate, assuming you set the constant libdir to your local git
>>>>> repository path where those tarballs will be downloaded from.
>>>>>
>>>>> I hope this helps someone!
>>>>>
>>>>> Could make a tool to do this if enough developers want access to it
>>>> without
>>>>> the pain of managing yet another R script?
>>>>>
>>>>
>>>
>>
>



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/