nested repositories and tools

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

nested repositories and tools

Simon Gladman
Hi all,

I've been trying to get my velvet optimiser wrapper to automatically install the tool and its dependencies to no avail.

Here is my problem in detail.


I have what I thought would be a simple wrapper and tool to pack into the new toolshed.


I have a tool_wrapper which points to a perl tool (VelvetOptimiser.pl) which in turn depends on some binaries being installed (velveth and velvetg). When I try and load these with tool dependencies and repository dependencies, galaxy cannot find the VelvetOptimiser.pl script nor the velvet binaries after they have been installed. (They're there, I've checked.)


It comprises of a repository (velvetoptimiser_gvl) with:

velvet_optimiser_vlsci.xml which points to a more complex velvet_optimiser_vlsci_wrapper.pl to create the actual command line. (This works when the dependencies are installed on the machine and available in the galaxy users path).


A repository_dependencies.xml file which points to an orphan repository (velvetoptimiser_orphan_225)


velvetoptimiser_orphan_225 has a tool_dependencies.xml to install the VelvetOptimiser.pl and associated modules and a repository_dependencies.xml file to point to the velvet orphan repository (velvet_1208).


velvet_1208 has a tool_dependencies.xml file which clones, installs and makes the velvet binaries and puts them in the right place.


All of the installations work when you install the tool from the toolshed. However, when I go to run the tool, it kept coming back saying it couldn’t find the VelvetOptimser.pl file. Then after the documentation was updated, I thought I may have to include the path as an environment like in the R_SCRIPT_PATH example in the wiki. So I set that up as well to no avail. I’ve left it in the files attached..


Both the toolshed that these repos are in and the galaxy being installed into are the latest versions as of 05/06/2013.


Please help!


I’ve attached all of the relevant files below.



Cheers,


Simon Gladman.

Victorian Bioinformatics Consortium

Monash University, Australia.


Respository: velvetoptimiser_gvl   -- the main repo.


File: velvet_optimiser_vlsci.xml (salient parts)


<tool id="velvetoptimiser_vlsci" name="Velvet Optimiser vlsci" version="1.0.0">

<description>Automatically optimise a de-novo assembly using Velvet.</description>

<command interpreter="perl">

    velvet_optimiser_wrapper_vlsci.pl

           \$VOPT_SCRIPT_PATH

           '$start_kmer'

           '$end_kmer'

           '$kmer_step'


           #for i in $inputs:

               '$i.__index__'

               '${i.filetype}'


</outputs>

   

   <requirments>

       <requirement type="set_environment">VOPT_SCRIPT_PATH</requirement>

       <requirement type="package" version="2.2.5">velvetoptimiser_orphan_225</requirement>

   </requirments>



File: repository_dependencies.xml


<?xml version="1.0"?>

<repositories description="Velvet Optimiser requires access to the VelvetOptimiser package.">

    <repository toolshed="http://galaxy-dev.genome.edu.au:9009/" name="velvetoptimiser_orphan_225" owner="slugger" changeset_revision="fc5a642b8741" />

</repositories>



Repository: velvetoptimiser_orphan_225


File: tool_dependencies.xml


<?xml version="1.0"?>

<tool_dependency>

   <set_environment version="1.0">

       <environment_variable name="VOPT_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable>   

   </set_environment>

   <package name="VelvetOptimiser-2.2.5" version="2.2.5">

       <install version="1.0">

           <actions>

               <action type="download_by_url">http://www.vicbioinformatics.com/VelvetOptimiser-2.2.5.tar.gz</action>

               <action type="move_directory_files">

                   <source_directory>.</source_directory>

                   <destination_directory>$INSTALL_DIR/bin</destination_directory>

               </action>

               <action type="set_environment">

                   <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable>

               </action>

           </actions>

       </install>

   </package>

</tool_dependency>


File: repository_dependencies.xml


<?xml version="1.0"?>

<repositories description="Velvet Optimiser requires access to the velvet binaries.">

    <repository toolshed="http://galaxy-dev.genome.edu.au:9009/" name="velvet_1208" owner="slugger" changeset_revision="2c95298637c9" />

</repositories>


Repository: velvet_1208


File: tool_dependencies.xml


<?xml version="1.0"?>

<tool_dependency>

   <package name="velvet" version="1.2.08">

       <install version="1.0">

           <actions>

               <action type="shell_command">git clone git://github.com/dzerbino/velvet.git</action>

               <action type="shell_command">make -i CATEGORIES=4 MAXKMERLENGTH=191 OPENMP=1</action>

               <action type="move_file">

                   <source>velvetg</source>

                   <destination>$INSTALL_DIR/bin</destination>

               </action>

               <action type="move_file">

                   <source>velveth</source>

                   <destination>$INSTALL_DIR/bin</destination>

               </action>

               <action type="set_environment">

                   <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable>

               </action>

           </actions>

       </install>

       <readme>

           Installs velvet latest version. (called 1.2.08 for Galaxy purposes.)

       </readme>

   </package>

</tool_dependency>



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: nested repositories and tools

Björn Grüning
Hi Simon!

Hi all,

I've been trying to get my velvet optimiser wrapper to automatically install the tool and its dependencies to no avail.

Nice!

Here is my problem in detail.
I have what I thought would be a simple wrapper and tool to pack into the new toolshed.


I have a tool_wrapper which points to a perl tool (VelvetOptimiser.pl) which in turn depends on some binaries being installed (velveth and velvetg). When I try and load these with tool dependencies and repository dependencies, galaxy cannot find the VelvetOptimiser.pl script nor the velvet binaries after they have been installed. (They're there, I've checked.)
It comprises of a repository (velvetoptimiser_gvl) with:

Are you installing your velvetoptimiser.pl (copying to some location) or is it shipped with your repository?
In the first case you do not need the "VOPT_SCRIPT_PATH" because its part of your PATH, in the latter case you do not need to install it.

velvet_optimiser_vlsci.xml which points to a more complex velvet_optimiser_vlsci_wrapper.pl to create the actual command line. (This works when the dependencies are installed on the machine and available in the galaxy users path).


A repository_dependencies.xml file which points to an orphan repository (velvetoptimiser_orphan_225)

Please name your repositories according to the naming scheme. Something like package_depname_majorversion_minorversion (like package_atlas_3_10, package_bwa_0_6_2, package_numpy_1_7).


velvetoptimiser_orphan_225 has a tool_dependencies.xml to install the VelvetOptimiser.pl and associated modules and a repository_dependencies.xml file to point to the velvet orphan repository (velvet_1208).

velvet_1208 has a tool_dependencies.xml file which clones, installs and makes the velvet binaries and puts them in the right place.
All of the installations work when you install the tool from the toolshed. However, when I go to run the tool, it kept coming back saying it couldn’t find the VelvetOptimser.pl file. Then after the documentation was updated, I thought I may have to include the path as an environment like in the R_SCRIPT_PATH example in the wiki. So I set that up as well to no avail. I’ve left it in the files attached..

Both the toolshed that these repos are in and the galaxy being installed into are the latest versions as of 05/06/2013.

I think you miss used the repository_dependencies.xml file.
Try to following:
- one orphan-tool-definition called package_velveth_1_2 (assuming 1_2 is stable and in point releases no new features are introduced, only bugfixed). Insert you velveth install script.
- place the following (please adopt, its not tested) in your main repository:

<?xml version="1.0"?>
<package name="velvet" version="0.1.18">
        <repository toolshed="http://galaxy-dev.genome.edu.au:9009/" name="package_velvet_1_2" owner="slugger" changeset_revision="2c95298637c9" />
</package>

<tool_dependency>
  <!-- im not sure that is needed,  see above -->
   <set_environment version="1.0">
       <environment_variable name="VOPT_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable>   
   </set_environment>
   <package name="velvetoptimiser" version="2.2.5">
       <install version="1.0">
           <actions>
               <action type="download_by_url">http://www.vicbioinformatics.com/VelvetOptimiser-2.2.5.tar.gz</action>
               <action type="move_directory_files">
                   <source_directory>.</source_directory>
                   <destination_directory>$INSTALL_DIR/bin</destination_directory>
               </action>
               <action type="set_environment">
                   <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable>
               </action>
           </actions>
       </install>
   </package>

</tool_dependency>

- your requirments tag in your tool.xml should look like that:

<requirments>
       <requirement type="set_environment">VOPT_SCRIPT_PATH</requirement>
       <requirement type="package" version="2.2.5">velvetoptimiser</requirement>
       <requirement type="package" version="0.1.18">velvet</requirement>
</requirments>

I hope it gets a little bit more clear.
Cheers,
Björn

Respository: velvetoptimiser_gvl   -- the main repo.


File: velvet_optimiser_vlsci.xml (salient parts)


<tool id="velvetoptimiser_vlsci" name="Velvet Optimiser vlsci" version="1.0.0">

<description>Automatically optimise a de-novo assembly using Velvet.</description>

<command interpreter="perl">

    velvet_optimiser_wrapper_vlsci.pl

           \$VOPT_SCRIPT_PATH

           '$start_kmer'

           '$end_kmer'

           '$kmer_step'


           #for i in $inputs:

               '$i.__index__'

               '${i.filetype}'






</outputs>

   

   <requirments>

       <requirement type="set_environment">VOPT_SCRIPT_PATH</requirement>

       <requirement type="package" version="2.2.5">velvetoptimiser_orphan_225</requirement>

   </requirments>





File: repository_dependencies.xml


<?xml version="1.0"?>

<repositories description="Velvet Optimiser requires access to the VelvetOptimiser package.">

    <repository toolshed="http://galaxy-dev.genome.edu.au:9009/" name="velvetoptimiser_orphan_225" owner="slugger" changeset_revision="fc5a642b8741" />

</repositories>



Repository: velvetoptimiser_orphan_225


File: tool_dependencies.xml


<?xml version="1.0"?>

<tool_dependency>

   <set_environment version="1.0">

       <environment_variable name="VOPT_SCRIPT_PATH" action="set_to">$REPOSITORY_INSTALL_DIR</environment_variable>   
   </set_environment>

   <package name="VelvetOptimiser-2.2.5" version="2.2.5">

       <install version="1.0">

           <actions>

               <action type="download_by_url">http://www.vicbioinformatics.com/VelvetOptimiser-2.2.5.tar.gz</action>

               <action type="move_directory_files">

                   <source_directory>.</source_directory>

                   <destination_directory>$INSTALL_DIR/bin</destination_directory>

               </action>

               <action type="set_environment">

                   <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable>

               </action>

           </actions>

       </install>

   </package>

</tool_dependency>


File: repository_dependencies.xml


<?xml version="1.0"?>

<repositories description="Velvet Optimiser requires access to the velvet binaries.">

    <repository toolshed="http://galaxy-dev.genome.edu.au:9009/" name="velvet_1208" owner="slugger" changeset_revision="2c95298637c9" />

</repositories>


Repository: velvet_1208


File: tool_dependencies.xml


<?xml version="1.0"?>

<tool_dependency>

   <package name="velvet" version="1.2.08">

       <install version="1.0">

           <actions>

               <action type="shell_command">git clone git://github.com/dzerbino/velvet.git</action>

               <action type="shell_command">make -i CATEGORIES=4 MAXKMERLENGTH=191 OPENMP=1</action>

               <action type="move_file">

                   <source>velvetg</source>

                   <destination>$INSTALL_DIR/bin</destination>

               </action>

               <action type="move_file">

                   <source>velveth</source>

                   <destination>$INSTALL_DIR/bin</destination>

               </action>

               <action type="set_environment">

                   <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable>

               </action>

           </actions>

       </install>

       <readme>

           Installs velvet latest version. (called 1.2.08 for Galaxy purposes.)

       </readme>

   </package>

</tool_dependency>



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/