Control on versioning in toolshed tools

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Control on versioning in toolshed tools

Eric Kuyt
Hi All,

I am playing around with putting a tool in testtoolshed. Now when changes to dependency versions are detected, the toolshed detects a new version and a dropdown is created. 

but sometimes I do not want this behavior when the first version was erroneous for example. I tried hg strip on the repository and pushing it back to the testtoolshed but sadly it didn't result in a clean repository but a multi-headered mess.

Is there a way to cleanup the remote repository and start over. And what would be a cleaner way to develop tools on a toolshed still making use of repository dependencies?

Thanks, 

Eric

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Control on versioning in toolshed tools

Peter Cock
Hi Eric,

Despite the fact that internal hg repositories are used, the idea
is NOT to use them as development repositories - but ONLY push
releases to the ToolShed.

In the interests of reproducibility (other people might use your
ToolShed entry in a workflow, or as a dependency), you should
not be able to ever rewrite history or delete commits - something
you can do with a git or hg repository but should generally avoid.

i.e. Being allowed to "cleapup and start again" is blocked by the
Galaxy goal of reproducibility.

I personally prefer git to hg, and therefore use that for development
tracking of my own ToolShed releases - but if you like hg then I
would suggest using a BitBucket.org hosted hg repository for
developing your tool.

You can see examples here - many of these tools do have
explicit dependencies on other tools/packages in the ToolShed
(either my own, or from 3rd parties):

https://github.com/peterjc/galaxy_blast
https://github.com/peterjc/pico_galaxy

Regards,

Peter

On Tue, Jun 24, 2014 at 1:15 PM, Eric Kuyt <[hidden email]> wrote:

> Hi All,
>
> I am playing around with putting a tool in testtoolshed. Now when changes to
> dependency versions are detected, the toolshed detects a new version and a
> dropdown is created.
>
> but sometimes I do not want this behavior when the first version was
> erroneous for example. I tried hg strip on the repository and pushing it
> back to the testtoolshed but sadly it didn't result in a clean repository
> but a multi-headered mess.
>
> Is there a way to cleanup the remote repository and start over. And what
> would be a cleaner way to develop tools on a toolshed still making use of
> repository dependencies?
>
> Thanks,
>
> Eric
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Control on versioning in toolshed tools

Greg Von Kuster
In reply to this post by Eric Kuyt
Hello Eric,

The public Galaxy test and main Tool Sheds are for sharing validated, functionally correct tools with the Galaxy community, not for developing them.  As you've discovered, developing tools using the public Tool Sheds results in undesired behavior since you have restricted control over what can easily be eliminated from sharing.  The primary reason for this is the Tool Shed's goal of reproducible analyses in galaxy.  Any version of any Galaxy utility uploaded to the public Tool Shed will always be available for installation into Galaxy.  This is the behavior you've seen when changing dependency versions, and similar behavior results when any Galaxy utility (tools, custom datatypes, Galaxy Data Managers, etc) version is changed.

Instead of using the public Galaxy Tool Sheds for development, you should be using a local development Tool Shed as a complementary component to your local Galaxy development environment.  The general steps in the development process are:

  1. Set up a local development Tool Shed - see the "Hosting Your Own Tool Shed for Developing Galaxy Tools and Utilities" section of: http://gregvonkuster.org/galaxy-tool-shed-introduction/
  2. Export repositories containing validated tool dependencies from the main Galaxy Tool Shed into a capsule - see http://gregvonkuster.org/galaxy-tool-shed-leveraging-community-contributions-repository-capsules/
  3. Import the capsule into your local Tool Shed.
  4. Create an empty repository in your local Tool Shed to contain the new Galaxy tool.
  5. Develop your tool in your local Galaxy environment, using the local Tool Shed as a source code repository if necessary.
  6. When development is finished, make sure the finished tool is available in hyour local Tool Shed's repository and execute your local Tool Shed's Install and Test framework to validate the tool - see the "Using the Install and Test Framework With a Local Tool Shed" section of: http://gregvonkuster.org/galaxy-tool-shed-certifying-repositories-install-test-framework/
  7. Export the repository containing the tool from your local Tool Shed.
  8. Import the repository containing the tool into the test and main Galaxy Tool Sheds for additional validation and sharing.
Greg Von Kuster


On Jun 24, 2014, at 8:15 AM, Eric Kuyt <[hidden email]> wrote:

Hi All,

I am playing around with putting a tool in testtoolshed. Now when changes to dependency versions are detected, the toolshed detects a new version and a dropdown is created. 

but sometimes I do not want this behavior when the first version was erroneous for example. I tried hg strip on the repository and pushing it back to the testtoolshed but sadly it didn't result in a clean repository but a multi-headered mess.

Is there a way to cleanup the remote repository and start over. And what would be a cleaner way to develop tools on a toolshed still making use of repository dependencies?

Thanks, 

Eric
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Control on versioning in toolshed tools

Eric Kuyt
In reply to this post by Eric Kuyt

Because I want to use R which has multiple layers of dependencies rebuilding it locally seemed a bit labor intensive but this capsule thing sounds exactly what I need. I'll take a look into it.

Thanks for the fast response.


On 24 June 2014 15:00, Greg Von Kuster <[hidden email]> wrote:
your tool in your local Galaxy environment, using the local Tool Shed as a source code repository if necessary.
  6.  When development is finished, make sure the finished tool is available in hyour local Tool Shed's repository and execute your local Tool Shed's Install and Test framework to validate the tool - see the "Using the Install an




--
Central Veterinary Institute of Wageningen UR (CVI)
Department of Infection Biology
PO box 65, 8200 AB Lelystad, NL
Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad
 
Tel:  +31-(0)320-293391
Fax: +31-(0)320-238153
E-mail: [hidden email]
Web: http://www.cvi.wur.nl

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/