tool data confusion

classic Classic list List threaded Threaded
5 messages Options
| Threaded
Open this post in threaded view
|

tool data confusion

Matthias Bernt
Dear list,

I still have problems to get my head around tool data. Lets consider
diamond for example (lets ignore data managers for the moment). After
installation it seems that there are two relevant xml files:

- `config/shed_tool_data_table_conf.xml`
-
`tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`

and two loc files:

- `tool-data/diamond_database.loc`
-
`tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`

In both xml files the latter loc file is referenced. And therefor the
tool-data/diamond_database.loc file is seemingly ignored.

What is the rational to have a loc and xml file for each tool version?
It seems that they are merged upon startup anyway.

What would be the best way to administrate a single tool loc file?

Cheers,
Matthias

--

-------------------------------------------
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone +49 341 235 482296,
[hidden email], www.ufz.de

Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
-------------------------------------------
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: tool data confusion

Peter Cock
Personally I tend to work with $GALAXY/tool-data/*.loc (and ignore
the tool shed installed copies in their cryptically named folders) but
this reflects in part the fact that this used to be the only copy of the file,
and that's just what I always did.

If you use a data manager, I don't know which copy it updates -
but the merging design is meant to hide these details I suppose.

Peter

On Mon, Oct 1, 2018 at 5:56 PM Matthias Bernt <[hidden email]> wrote:

>
> Dear list,
>
> I still have problems to get my head around tool data. Lets consider
> diamond for example (lets ignore data managers for the moment). After
> installation it seems that there are two relevant xml files:
>
> - `config/shed_tool_data_table_conf.xml`
> -
> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`
>
> and two loc files:
>
> - `tool-data/diamond_database.loc`
> -
> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`
>
> In both xml files the latter loc file is referenced. And therefor the
> tool-data/diamond_database.loc file is seemingly ignored.
>
> What is the rational to have a loc and xml file for each tool version?
> It seems that they are merged upon startup anyway.
>
> What would be the best way to administrate a single tool loc file?
>
> Cheers,
> Matthias
>
> --
>
> -------------------------------------------
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> [hidden email], www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
> MinDirig Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> -------------------------------------------
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: tool data confusion

Matthias Bernt
Hi Peter,

then the I need to update all the paths in:

`config/shed_tool_data_table_conf.xml`?

Because currently the $GALAXY/tool-data/*.loc files are ignored in my
instance.

For me it looks like a bug (caused by a misconfiguration?) that there
are all xml files refer to the same loc file.

Wouldn't this be more useful:

- config/shed_tool_data_table_conf.xml -> tool-data/diamond_database.loc

-
tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml
->
tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc

Best,
Matthias


On 02.10.2018 11:22, Peter Cock wrote:

> Personally I tend to work with $GALAXY/tool-data/*.loc (and ignore
> the tool shed installed copies in their cryptically named folders) but
> this reflects in part the fact that this used to be the only copy of the file,
> and that's just what I always did.
>
> If you use a data manager, I don't know which copy it updates -
> but the merging design is meant to hide these details I suppose.
>
> Peter
>
> On Mon, Oct 1, 2018 at 5:56 PM Matthias Bernt <[hidden email]> wrote:
>>
>> Dear list,
>>
>> I still have problems to get my head around tool data. Lets consider
>> diamond for example (lets ignore data managers for the moment). After
>> installation it seems that there are two relevant xml files:
>>
>> - `config/shed_tool_data_table_conf.xml`
>> -
>> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`
>>
>> and two loc files:
>>
>> - `tool-data/diamond_database.loc`
>> -
>> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`
>>
>> In both xml files the latter loc file is referenced. And therefor the
>> tool-data/diamond_database.loc file is seemingly ignored.
>>
>> What is the rational to have a loc and xml file for each tool version?
>> It seems that they are merged upon startup anyway.
>>
>> What would be the best way to administrate a single tool loc file?
>>
>> Cheers,
>> Matthias
>>
>> --
>>
>> -------------------------------------------
>> Matthias Bernt
>> Bioinformatics Service
>> Molekulare Systembiologie (MOLSYB)
>> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
>> Helmholtz Centre for Environmental Research GmbH - UFZ
>> Permoserstraße 15, 04318 Leipzig, Germany
>> Phone +49 341 235 482296,
>> [hidden email], www.ufz.de
>>
>> Sitz der Gesellschaft/Registered Office: Leipzig
>> Registergericht/Registration Office: Amtsgericht Leipzig
>> Handelsregister Nr./Trade Register Nr.: B 4703
>> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
>> MinDirig Wilfried Kraus
>> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
>> Prof. Dr. Dr. h.c. Georg Teutsch
>> Administrative Geschäftsführerin/ Administrative Managing Director:
>> Prof. Dr. Heike Graßmann
>> -------------------------------------------
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>    https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>    http://galaxyproject.org/search/

--

-------------------------------------------
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone +49 341 235 482296,
[hidden email], www.ufz.de

Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
-------------------------------------------
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: tool data confusion

Peter Cock
Hi Matthias,

I'm getting out of my depth here - but if tool-data/xxx.loc is being ignored,
you may need to enable this by adding the XML data table entries from
the relevant tool_data_table_conf.xml.sample file for tool xxx to
$GALAXY/config/tool_data_table_conf.xml.

At least, that's what I had to do and documented recently here:
https://github.com/abaizan/kodoja_galaxy/commit/2cd7579a15887ae4ffdc5ab3a346681ebb53b0a2

Again, this may be me sticking to old pre-tool shed habits - so I'd like
to hear how other people manage their *.loc files, especially when doing
hand editing to add entries.

Peter
On Tue, Oct 2, 2018 at 10:29 AM Matthias Bernt <[hidden email]> wrote:

>
> Hi Peter,
>
> then the I need to update all the paths in:
>
> `config/shed_tool_data_table_conf.xml`?
>
> Because currently the $GALAXY/tool-data/*.loc files are ignored in my
> instance.
>
> For me it looks like a bug (caused by a misconfiguration?) that there
> are all xml files refer to the same loc file.
>
> Wouldn't this be more useful:
>
> - config/shed_tool_data_table_conf.xml -> tool-data/diamond_database.loc
>
> -
> tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml
> ->
> tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc
>
> Best,
> Matthias
>
>
> On 02.10.2018 11:22, Peter Cock wrote:
> > Personally I tend to work with $GALAXY/tool-data/*.loc (and ignore
> > the tool shed installed copies in their cryptically named folders) but
> > this reflects in part the fact that this used to be the only copy of the file,
> > and that's just what I always did.
> >
> > If you use a data manager, I don't know which copy it updates -
> > but the merging design is meant to hide these details I suppose.
> >
> > Peter
> >
> > On Mon, Oct 1, 2018 at 5:56 PM Matthias Bernt <[hidden email]> wrote:
> >>
> >> Dear list,
> >>
> >> I still have problems to get my head around tool data. Lets consider
> >> diamond for example (lets ignore data managers for the moment). After
> >> installation it seems that there are two relevant xml files:
> >>
> >> - `config/shed_tool_data_table_conf.xml`
> >> -
> >> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/tool_data_table_conf.xml`
> >>
> >> and two loc files:
> >>
> >> - `tool-data/diamond_database.loc`
> >> -
> >> `tool-data/toolshed.g2.bx.psu.edu/repos/bgruening/diamond/64be1ac21109/diamond_database.loc`
> >>
> >> In both xml files the latter loc file is referenced. And therefor the
> >> tool-data/diamond_database.loc file is seemingly ignored.
> >>
> >> What is the rational to have a loc and xml file for each tool version?
> >> It seems that they are merged upon startup anyway.
> >>
> >> What would be the best way to administrate a single tool loc file?
> >>
> >> Cheers,
> >> Matthias
> >>
> >> --
> >>
> >> -------------------------------------------
> >> Matthias Bernt
> >> Bioinformatics Service
> >> Molekulare Systembiologie (MOLSYB)
> >> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> >> Helmholtz Centre for Environmental Research GmbH - UFZ
> >> Permoserstraße 15, 04318 Leipzig, Germany
> >> Phone +49 341 235 482296,
> >> [hidden email], www.ufz.de
> >>
> >> Sitz der Gesellschaft/Registered Office: Leipzig
> >> Registergericht/Registration Office: Amtsgericht Leipzig
> >> Handelsregister Nr./Trade Register Nr.: B 4703
> >> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
> >> MinDirig Wilfried Kraus
> >> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> >> Prof. Dr. Dr. h.c. Georg Teutsch
> >> Administrative Geschäftsführerin/ Administrative Managing Director:
> >> Prof. Dr. Heike Graßmann
> >> -------------------------------------------
> >> ___________________________________________________________
> >> Please keep all replies on the list by using "reply all"
> >> in your mail client.  To manage your subscriptions to this
> >> and other Galaxy lists, please use the interface at:
> >>    https://lists.galaxyproject.org/
> >>
> >> To search Galaxy mailing lists use the unified search at:
> >>    http://galaxyproject.org/search/
>
> --
>
> -------------------------------------------
> Matthias Bernt
> Bioinformatics Service
> Molekulare Systembiologie (MOLSYB)
> Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
> Helmholtz Centre for Environmental Research GmbH - UFZ
> Permoserstraße 15, 04318 Leipzig, Germany
> Phone +49 341 235 482296,
> [hidden email], www.ufz.de
>
> Sitz der Gesellschaft/Registered Office: Leipzig
> Registergericht/Registration Office: Amtsgericht Leipzig
> Handelsregister Nr./Trade Register Nr.: B 4703
> Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
> MinDirig Wilfried Kraus
> Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
> Prof. Dr. Dr. h.c. Georg Teutsch
> Administrative Geschäftsführerin/ Administrative Managing Director:
> Prof. Dr. Heike Graßmann
> -------------------------------------------
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: tool data confusion

Jelle Scholtalbers
Hi all,

Again, this may be me sticking to old pre-tool shed habits - so I'd like
to hear how other people manage their *.loc files, especially when doing
hand editing to add entries.

I **try** to stick to just editing the *.loc file in tool-data for those tools that do not have a data manager. This way I can also track them in my local git branch. But I do agree that it can be complicated to figure out which files to edit for which tool (bwa/bwamem...) and add the correct entry and in the right format.
The data managers make it on the one hand easier, but harder to tell if I was adding entries by hand or through a data manager - especially when not being consistent..

Part of this could be mitigated if the admin web interface would allow for editing these "Tool Data Table Entries" and not just view them -> https://github.com/galaxyproject/galaxy/issues/6819

Cheers,
Jelle


 

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/