Ensuring Python 2.7 when resolving conda dependencies for tool

classic Classic list List threaded Threaded
7 messages Options
| Threaded
Open this post in threaded view
|

Ensuring Python 2.7 when resolving conda dependencies for tool

Peter Briggs
Hello

I'm in the process of updating our local Galaxy tools to use conda dependency resolution, and I've hit a snag with a tool that requires Python 2.7 along with the Python 2.7-compatible version of Biopython 1.65.

I'd assumed that if I explicitly used the following in the "requirements" section of the tool XML:

    <requirement type="package" version="2.7">python</requirement>
    <requirement type="package" version="1.65">biopython</requirement>

that the biopython install would respect the specified Python version, and that the command execution environment would be based on Python 2.7.

But in practice it appears I'm getting Python3 as I'm seeing errors:

      File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py", line 129
        print "\n~~~~~~~~~~"
                           ^
    SyntaxError: Missing parentheses in call to 'print'

This is happening both when running the tool tests via the planemo tests, and also when I install the tool from a local toolshed instance.

Can anyone advise how to coerce the Python dependencies to be 2.7-compatible?

Thanks in advance for any help,

Best wishes

Peter

--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Peter Cock
Tricky, as I understand it the conda python packages are specific to
the conda version of Python - in this case Python 3 not 2.

It might actually be simpler to fix pal_finder/pal_filter.py to cope
with Python 3 - is the code online somewhere, I could probably cast an
eye over it.

Peter

On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
<[hidden email]> wrote:

> Hello
>
> I'm in the process of updating our local Galaxy tools to use conda dependency resolution, and I've hit a snag with a tool that requires Python 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>
> I'd assumed that if I explicitly used the following in the "requirements" section of the tool XML:
>
>     <requirement type="package" version="2.7">python</requirement>
>     <requirement type="package" version="1.65">biopython</requirement>
>
> that the biopython install would respect the specified Python version, and that the command execution environment would be based on Python 2.7.
>
> But in practice it appears I'm getting Python3 as I'm seeing errors:
>
>       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py", line 129
>         print "\n~~~~~~~~~~"
>                            ^
>     SyntaxError: Missing parentheses in call to 'print'
>
> This is happening both when running the tool tests via the planemo tests, and also when I install the tool from a local toolshed instance.
>
> Can anyone advise how to coerce the Python dependencies to be 2.7-compatible?
>
> Thanks in advance for any help,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs [hidden email]
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Marius van den Beek
This should actually work properly if you install the dependencies via
the Manage dependencies page in the admin menu or if you install the tool via the tool shed.
This translates more or less to the following conda command:
```
$ conda create -n mulled-v1xxxxxxxx python=2.7 biopython=1.65 -c iuc -c bioconda -c conda-forge -c defaults
Fetching package metadata ...................
Solving package specifications: .

Package plan for installation in environment /Users/mvandenb/miniconda3/envs/test_resolution:

The following NEW packages will be INSTALLED:

    biopython:       1.65-np19py27_0
    ca-certificates: 2018.1.18-0          conda-forge
    certifi:         2018.1.18-py27_0     conda-forge
    intel-openmp:    2018.0.0-h8158457_8
    libgfortran:     3.0.1-h93005f0_2
    mkl:             2018.0.1-hfbd8650_4
    ncurses:         5.9-10               conda-forge
    numpy:           1.9.3-py27hb3dd696_3
    openssl:         1.0.2n-0             conda-forge
    pip:             9.0.1-py27_1         conda-forge
    python:          2.7.14-4             conda-forge
    readline:        7.0-0                conda-forge
    setuptools:      38.5.1-py27_0        conda-forge
    sqlite:          3.20.1-2             conda-forge
    tk:              8.6.7-0              conda-forge
    wheel:           0.30.0-py27_2        conda-forge
    zlib:            1.2.11-0             conda-forge
```

So given a recent conda (latest version should work) and the correct channel order
it should work.

Best,
Marius

On 5 March 2018 at 17:39, Peter Cock <[hidden email]> wrote:
Tricky, as I understand it the conda python packages are specific to
the conda version of Python - in this case Python 3 not 2.

It might actually be simpler to fix pal_finder/pal_filter.py to cope
with Python 3 - is the code online somewhere, I could probably cast an
eye over it.

Peter

On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
<[hidden email]> wrote:
> Hello
>
> I'm in the process of updating our local Galaxy tools to use conda dependency resolution, and I've hit a snag with a tool that requires Python 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>
> I'd assumed that if I explicitly used the following in the "requirements" section of the tool XML:
>
>     <requirement type="package" version="2.7">python</requirement>
>     <requirement type="package" version="1.65">biopython</requirement>
>
> that the biopython install would respect the specified Python version, and that the command execution environment would be based on Python 2.7.
>
> But in practice it appears I'm getting Python3 as I'm seeing errors:
>
>       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py", line 129
>         print "\n~~~~~~~~~~"
>                            ^
>     SyntaxError: Missing parentheses in call to 'print'
>
> This is happening both when running the tool tests via the planemo tests, and also when I install the tool from a local toolshed instance.
>
> Can anyone advise how to coerce the Python dependencies to be 2.7-compatible?
>
> Thanks in advance for any help,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs [hidden email]
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Peter Cock
I stand corrected. Looking closer, there are conda packages
for both Python 2 and 3, e.g.

https://anaconda.org/conda-forge/biopython/files

However, in this case you're asking for an older Biopython
which has to date not been packaged in conda-forge or
bioconda, so I presume in Marius example it comes from
PyPI via pip?


Peter

On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
<[hidden email]> wrote:

> This should actually work properly if you install the dependencies via
> the Manage dependencies page in the admin menu or if you install the tool
> via the tool shed.
> This translates more or less to the following conda command:
> ```
> $ conda create -n mulled-v1xxxxxxxx python=2.7 biopython=1.65 -c iuc -c
> bioconda -c conda-forge -c defaults
> Fetching package metadata ...................
> Solving package specifications: .
>
> Package plan for installation in environment
> /Users/mvandenb/miniconda3/envs/test_resolution:
>
> The following NEW packages will be INSTALLED:
>
>     biopython:       1.65-np19py27_0
>     ca-certificates: 2018.1.18-0          conda-forge
>     certifi:         2018.1.18-py27_0     conda-forge
>     intel-openmp:    2018.0.0-h8158457_8
>     libgfortran:     3.0.1-h93005f0_2
>     mkl:             2018.0.1-hfbd8650_4
>     ncurses:         5.9-10               conda-forge
>     numpy:           1.9.3-py27hb3dd696_3
>     openssl:         1.0.2n-0             conda-forge
>     pip:             9.0.1-py27_1         conda-forge
>     python:          2.7.14-4             conda-forge
>     readline:        7.0-0                conda-forge
>     setuptools:      38.5.1-py27_0        conda-forge
>     sqlite:          3.20.1-2             conda-forge
>     tk:              8.6.7-0              conda-forge
>     wheel:           0.30.0-py27_2        conda-forge
>     zlib:            1.2.11-0             conda-forge
> ```
>
> So given a recent conda (latest version should work) and the correct channel
> order
> it should work.
>
> Best,
> Marius
>
> On 5 March 2018 at 17:39, Peter Cock <[hidden email]> wrote:
>>
>> Tricky, as I understand it the conda python packages are specific to
>> the conda version of Python - in this case Python 3 not 2.
>>
>> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> with Python 3 - is the code online somewhere, I could probably cast an
>> eye over it.
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> <[hidden email]> wrote:
>> > Hello
>> >
>> > I'm in the process of updating our local Galaxy tools to use conda
>> > dependency resolution, and I've hit a snag with a tool that requires Python
>> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >
>> > I'd assumed that if I explicitly used the following in the
>> > "requirements" section of the tool XML:
>> >
>> >     <requirement type="package" version="2.7">python</requirement>
>> >     <requirement type="package" version="1.65">biopython</requirement>
>> >
>> > that the biopython install would respect the specified Python version,
>> > and that the command execution environment would be based on Python 2.7.
>> >
>> > But in practice it appears I'm getting Python3 as I'm seeing errors:
>> >
>> >       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py", line
>> > 129
>> >         print "\n~~~~~~~~~~"
>> >                            ^
>> >     SyntaxError: Missing parentheses in call to 'print'
>> >
>> > This is happening both when running the tool tests via the planemo
>> > tests, and also when I install the tool from a local toolshed instance.
>> >
>> > Can anyone advise how to coerce the Python dependencies to be
>> > 2.7-compatible?
>> >
>> > Thanks in advance for any help,
>> >
>> > Best wishes
>> >
>> > Peter
>> >
>> > --
>> > Peter Briggs [hidden email]
>> > Bioinformatics Core Facility University of Manchester
>> > B.1083 Michael Smith Bldg Tel: (0161) 2751482
>> >
>> > ___________________________________________________________
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Marius van den Beek
It's coming from the defaults channel, I didn't check if this is actually working,
it may very well not be compatible with the remaining packages form conda-forge.
AFAIK you can't specify packages to install via the regular conda commands
(though you can use pip in a conda environemnt to do this manually).

Hope that helps,
Marius

On 5 March 2018 at 17:51, Peter Cock <[hidden email]> wrote:
I stand corrected. Looking closer, there are conda packages
for both Python 2 and 3, e.g.

https://anaconda.org/conda-forge/biopython/files

However, in this case you're asking for an older Biopython
which has to date not been packaged in conda-forge or
bioconda, so I presume in Marius example it comes from
PyPI via pip?


Peter

On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
<[hidden email]> wrote:
> This should actually work properly if you install the dependencies via
> the Manage dependencies page in the admin menu or if you install the tool
> via the tool shed.
> This translates more or less to the following conda command:
> ```
> $ conda create -n mulled-v1xxxxxxxx python=2.7 biopython=1.65 -c iuc -c
> bioconda -c conda-forge -c defaults
> Fetching package metadata ...................
> Solving package specifications: .
>
> Package plan for installation in environment
> /Users/mvandenb/miniconda3/envs/test_resolution:
>
> The following NEW packages will be INSTALLED:
>
>     biopython:       1.65-np19py27_0
>     ca-certificates: 2018.1.18-0          conda-forge
>     certifi:         2018.1.18-py27_0     conda-forge
>     intel-openmp:    2018.0.0-h8158457_8
>     libgfortran:     3.0.1-h93005f0_2
>     mkl:             2018.0.1-hfbd8650_4
>     ncurses:         5.9-10               conda-forge
>     numpy:           1.9.3-py27hb3dd696_3
>     openssl:         1.0.2n-0             conda-forge
>     pip:             9.0.1-py27_1         conda-forge
>     python:          2.7.14-4             conda-forge
>     readline:        7.0-0                conda-forge
>     setuptools:      38.5.1-py27_0        conda-forge
>     sqlite:          3.20.1-2             conda-forge
>     tk:              8.6.7-0              conda-forge
>     wheel:           0.30.0-py27_2        conda-forge
>     zlib:            1.2.11-0             conda-forge
> ```
>
> So given a recent conda (latest version should work) and the correct channel
> order
> it should work.
>
> Best,
> Marius
>
> On 5 March 2018 at 17:39, Peter Cock <[hidden email]> wrote:
>>
>> Tricky, as I understand it the conda python packages are specific to
>> the conda version of Python - in this case Python 3 not 2.
>>
>> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> with Python 3 - is the code online somewhere, I could probably cast an
>> eye over it.
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> <[hidden email]> wrote:
>> > Hello
>> >
>> > I'm in the process of updating our local Galaxy tools to use conda
>> > dependency resolution, and I've hit a snag with a tool that requires Python
>> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >
>> > I'd assumed that if I explicitly used the following in the
>> > "requirements" section of the tool XML:
>> >
>> >     <requirement type="package" version="2.7">python</requirement>
>> >     <requirement type="package" version="1.65">biopython</requirement>
>> >
>> > that the biopython install would respect the specified Python version,
>> > and that the command execution environment would be based on Python 2.7.
>> >
>> > But in practice it appears I'm getting Python3 as I'm seeing errors:
>> >
>> >       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py", line
>> > 129
>> >         print "\n~~~~~~~~~~"
>> >                            ^
>> >     SyntaxError: Missing parentheses in call to 'print'
>> >
>> > This is happening both when running the tool tests via the planemo
>> > tests, and also when I install the tool from a local toolshed instance.
>> >
>> > Can anyone advise how to coerce the Python dependencies to be
>> > 2.7-compatible?
>> >
>> > Thanks in advance for any help,
>> >
>> > Best wishes
>> >
>> > Peter
>> >
>> > --
>> > Peter Briggs [hidden email]
>> > Bioinformatics Core Facility University of Manchester
>> > B.1083 Michael Smith Bldg Tel: (0161) 2751482
>> >
>> > ___________________________________________________________
>> > Please keep all replies on the list by using "reply all"
>> > in your mail client.  To manage your subscriptions to this
>> > and other Galaxy lists, please use the interface at:
>> >   https://lists.galaxyproject.org/
>> >
>> > To search Galaxy mailing lists use the unified search at:
>> >   http://galaxyproject.org/search/
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/
>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Peter Cock
Oh yeah - thanks, I can see them in the default channel which does have
far older versions of Biopython packaged - currently 1.65 is on page 5:

https://anaconda.org/anaconda/biopython/files?sort=basename&sort_order=desc&page=5

Peter


On Mon, Mar 5, 2018 at 5:00 PM, Marius van den Beek
<[hidden email]> wrote:

> It's coming from the defaults channel, I didn't check if this is actually
> working,
> it may very well not be compatible with the remaining packages form
> conda-forge.
> AFAIK you can't specify packages to install via the regular conda commands
> (though you can use pip in a conda environemnt to do this manually).
>
> Hope that helps,
> Marius
>
> On 5 March 2018 at 17:51, Peter Cock <[hidden email]> wrote:
>>
>> I stand corrected. Looking closer, there are conda packages
>> for both Python 2 and 3, e.g.
>>
>> https://anaconda.org/conda-forge/biopython/files
>>
>> However, in this case you're asking for an older Biopython
>> which has to date not been packaged in conda-forge or
>> bioconda, so I presume in Marius example it comes from
>> PyPI via pip?
>>
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
>> <[hidden email]> wrote:
>> > This should actually work properly if you install the dependencies via
>> > the Manage dependencies page in the admin menu or if you install the
>> > tool
>> > via the tool shed.
>> > This translates more or less to the following conda command:
>> > ```
>> > $ conda create -n mulled-v1xxxxxxxx python=2.7 biopython=1.65 -c iuc -c
>> > bioconda -c conda-forge -c defaults
>> > Fetching package metadata ...................
>> > Solving package specifications: .
>> >
>> > Package plan for installation in environment
>> > /Users/mvandenb/miniconda3/envs/test_resolution:
>> >
>> > The following NEW packages will be INSTALLED:
>> >
>> >     biopython:       1.65-np19py27_0
>> >     ca-certificates: 2018.1.18-0          conda-forge
>> >     certifi:         2018.1.18-py27_0     conda-forge
>> >     intel-openmp:    2018.0.0-h8158457_8
>> >     libgfortran:     3.0.1-h93005f0_2
>> >     mkl:             2018.0.1-hfbd8650_4
>> >     ncurses:         5.9-10               conda-forge
>> >     numpy:           1.9.3-py27hb3dd696_3
>> >     openssl:         1.0.2n-0             conda-forge
>> >     pip:             9.0.1-py27_1         conda-forge
>> >     python:          2.7.14-4             conda-forge
>> >     readline:        7.0-0                conda-forge
>> >     setuptools:      38.5.1-py27_0        conda-forge
>> >     sqlite:          3.20.1-2             conda-forge
>> >     tk:              8.6.7-0              conda-forge
>> >     wheel:           0.30.0-py27_2        conda-forge
>> >     zlib:            1.2.11-0             conda-forge
>> > ```
>> >
>> > So given a recent conda (latest version should work) and the correct
>> > channel
>> > order
>> > it should work.
>> >
>> > Best,
>> > Marius
>> >
>> > On 5 March 2018 at 17:39, Peter Cock <[hidden email]> wrote:
>> >>
>> >> Tricky, as I understand it the conda python packages are specific to
>> >> the conda version of Python - in this case Python 3 not 2.
>> >>
>> >> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> >> with Python 3 - is the code online somewhere, I could probably cast an
>> >> eye over it.
>> >>
>> >> Peter
>> >>
>> >> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> >> <[hidden email]> wrote:
>> >> > Hello
>> >> >
>> >> > I'm in the process of updating our local Galaxy tools to use conda
>> >> > dependency resolution, and I've hit a snag with a tool that requires
>> >> > Python
>> >> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >> >
>> >> > I'd assumed that if I explicitly used the following in the
>> >> > "requirements" section of the tool XML:
>> >> >
>> >> >     <requirement type="package" version="2.7">python</requirement>
>> >> >     <requirement type="package"
>> >> > version="1.65">biopython</requirement>
>> >> >
>> >> > that the biopython install would respect the specified Python
>> >> > version,
>> >> > and that the command execution environment would be based on Python
>> >> > 2.7.
>> >> >
>> >> > But in practice it appears I'm getting Python3 as I'm seeing errors:
>> >> >
>> >> >       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py",
>> >> > line
>> >> > 129
>> >> >         print "\n~~~~~~~~~~"
>> >> >                            ^
>> >> >     SyntaxError: Missing parentheses in call to 'print'
>> >> >
>> >> > This is happening both when running the tool tests via the planemo
>> >> > tests, and also when I install the tool from a local toolshed
>> >> > instance.
>> >> >
>> >> > Can anyone advise how to coerce the Python dependencies to be
>> >> > 2.7-compatible?
>> >> >
>> >> > Thanks in advance for any help,
>> >> >
>> >> > Best wishes
>> >> >
>> >> > Peter
>> >> >
>> >> > --
>> >> > Peter Briggs [hidden email]
>> >> > Bioinformatics Core Facility University of Manchester
>> >> > B.1083 Michael Smith Bldg Tel: (0161) 2751482
>> >> >
>> >> > ___________________________________________________________
>> >> > Please keep all replies on the list by using "reply all"
>> >> > in your mail client.  To manage your subscriptions to this
>> >> > and other Galaxy lists, please use the interface at:
>> >> >   https://lists.galaxyproject.org/
>> >> >
>> >> > To search Galaxy mailing lists use the unified search at:
>> >> >   http://galaxyproject.org/search/
>> >> ___________________________________________________________
>> >> Please keep all replies on the list by using "reply all"
>> >> in your mail client.  To manage your subscriptions to this
>> >> and other Galaxy lists, please use the interface at:
>> >>   https://lists.galaxyproject.org/
>> >>
>> >> To search Galaxy mailing lists use the unified search at:
>> >>   http://galaxyproject.org/search/
>> >
>> >
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
| Threaded
Open this post in threaded view
|

Re: Ensuring Python 2.7 when resolving conda dependencies for tool

Peter Briggs
Hello Marius and Peter

Thanks for all the info & suggestions, that was really helpful and I can see what's happening now (and why it's not working in my specific case).

If you're interested in the details:

-- If the tool requires python==2.7 and biopython=1.65 only, then as Marius says the dependencies are resolved and Galaxy creates a "mulled" environment with the correct package versions installed.

-- In my case however I have some additional requirements defined, which are not yet in bioconda or the other channels (at this stage they've just been built locally, while I try to get everything updated). In this case conda fails to resolve some of the dependencies when trying to create the mulled environment, and bails out - instead it creates individual environments (e.g. __python@2.7, __biopython@1.65 etc) just for those packages it can find. But this fallback "standalone" biopython environment doesn't have the python==2.7 requirement transferred in, hence it defaults to python3.

I guess I didn't have this issue previously when migrating other tools (even when there were local dependencies) as there was no python dependency in those cases.

So it looks like I need to sort out the missing dependencies first, then come back to testing the tool.

Thanks again both for all your help,

Best wishes

Peter

--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482


________________________________________
From: Peter Cock [[hidden email]]
Sent: Monday, March 05, 2018 5:04 PM
To: Marius van den Beek
Cc: Peter Briggs; [hidden email]
Subject: Re: [galaxy-dev] Ensuring Python 2.7 when resolving conda dependencies for tool

Oh yeah - thanks, I can see them in the default channel which does have
far older versions of Biopython packaged - currently 1.65 is on page 5:

https://anaconda.org/anaconda/biopython/files?sort=basename&sort_order=desc&page=5

Peter


On Mon, Mar 5, 2018 at 5:00 PM, Marius van den Beek
<[hidden email]> wrote:

> It's coming from the defaults channel, I didn't check if this is actually
> working,
> it may very well not be compatible with the remaining packages form
> conda-forge.
> AFAIK you can't specify packages to install via the regular conda commands
> (though you can use pip in a conda environemnt to do this manually).
>
> Hope that helps,
> Marius
>
> On 5 March 2018 at 17:51, Peter Cock <[hidden email]> wrote:
>>
>> I stand corrected. Looking closer, there are conda packages
>> for both Python 2 and 3, e.g.
>>
>> https://anaconda.org/conda-forge/biopython/files
>>
>> However, in this case you're asking for an older Biopython
>> which has to date not been packaged in conda-forge or
>> bioconda, so I presume in Marius example it comes from
>> PyPI via pip?
>>
>>
>> Peter
>>
>> On Mon, Mar 5, 2018 at 4:46 PM, Marius van den Beek
>> <[hidden email]> wrote:
>> > This should actually work properly if you install the dependencies via
>> > the Manage dependencies page in the admin menu or if you install the
>> > tool
>> > via the tool shed.
>> > This translates more or less to the following conda command:
>> > ```
>> > $ conda create -n mulled-v1xxxxxxxx python=2.7 biopython=1.65 -c iuc -c
>> > bioconda -c conda-forge -c defaults
>> > Fetching package metadata ...................
>> > Solving package specifications: .
>> >
>> > Package plan for installation in environment
>> > /Users/mvandenb/miniconda3/envs/test_resolution:
>> >
>> > The following NEW packages will be INSTALLED:
>> >
>> >     biopython:       1.65-np19py27_0
>> >     ca-certificates: 2018.1.18-0          conda-forge
>> >     certifi:         2018.1.18-py27_0     conda-forge
>> >     intel-openmp:    2018.0.0-h8158457_8
>> >     libgfortran:     3.0.1-h93005f0_2
>> >     mkl:             2018.0.1-hfbd8650_4
>> >     ncurses:         5.9-10               conda-forge
>> >     numpy:           1.9.3-py27hb3dd696_3
>> >     openssl:         1.0.2n-0             conda-forge
>> >     pip:             9.0.1-py27_1         conda-forge
>> >     python:          2.7.14-4             conda-forge
>> >     readline:        7.0-0                conda-forge
>> >     setuptools:      38.5.1-py27_0        conda-forge
>> >     sqlite:          3.20.1-2             conda-forge
>> >     tk:              8.6.7-0              conda-forge
>> >     wheel:           0.30.0-py27_2        conda-forge
>> >     zlib:            1.2.11-0             conda-forge
>> > ```
>> >
>> > So given a recent conda (latest version should work) and the correct
>> > channel
>> > order
>> > it should work.
>> >
>> > Best,
>> > Marius
>> >
>> > On 5 March 2018 at 17:39, Peter Cock <[hidden email]> wrote:
>> >>
>> >> Tricky, as I understand it the conda python packages are specific to
>> >> the conda version of Python - in this case Python 3 not 2.
>> >>
>> >> It might actually be simpler to fix pal_finder/pal_filter.py to cope
>> >> with Python 3 - is the code online somewhere, I could probably cast an
>> >> eye over it.
>> >>
>> >> Peter
>> >>
>> >> On Mon, Mar 5, 2018 at 4:28 PM, Peter Briggs
>> >> <[hidden email]> wrote:
>> >> > Hello
>> >> >
>> >> > I'm in the process of updating our local Galaxy tools to use conda
>> >> > dependency resolution, and I've hit a snag with a tool that requires
>> >> > Python
>> >> > 2.7 along with the Python 2.7-compatible version of Biopython 1.65.
>> >> >
>> >> > I'd assumed that if I explicitly used the following in the
>> >> > "requirements" section of the tool XML:
>> >> >
>> >> >     <requirement type="package" version="2.7">python</requirement>
>> >> >     <requirement type="package"
>> >> > version="1.65">biopython</requirement>
>> >> >
>> >> > that the biopython install would respect the specified Python
>> >> > version,
>> >> > and that the command execution environment would be based on Python
>> >> > 2.7.
>> >> >
>> >> > But in practice it appears I'm getting Python3 as I'm seeing errors:
>> >> >
>> >> >       File "XXXXXXXX/galaxy-tools/tools/pal_finder/pal_filter.py",
>> >> > line
>> >> > 129
>> >> >         print "\n~~~~~~~~~~"
>> >> >                            ^
>> >> >     SyntaxError: Missing parentheses in call to 'print'
>> >> >
>> >> > This is happening both when running the tool tests via the planemo
>> >> > tests, and also when I install the tool from a local toolshed
>> >> > instance.
>> >> >
>> >> > Can anyone advise how to coerce the Python dependencies to be
>> >> > 2.7-compatible?
>> >> >
>> >> > Thanks in advance for any help,
>> >> >
>> >> > Best wishes
>> >> >
>> >> > Peter
>> >> >
>> >> > --
>> >> > Peter Briggs [hidden email]
>> >> > Bioinformatics Core Facility University of Manchester
>> >> > B.1083 Michael Smith Bldg Tel: (0161) 2751482
>> >> >
>> >> > ___________________________________________________________
>> >> > Please keep all replies on the list by using "reply all"
>> >> > in your mail client.  To manage your subscriptions to this
>> >> > and other Galaxy lists, please use the interface at:
>> >> >   https://lists.galaxyproject.org/
>> >> >
>> >> > To search Galaxy mailing lists use the unified search at:
>> >> >   http://galaxyproject.org/search/
>> >> ___________________________________________________________
>> >> Please keep all replies on the list by using "reply all"
>> >> in your mail client.  To manage your subscriptions to this
>> >> and other Galaxy lists, please use the interface at:
>> >>   https://lists.galaxyproject.org/
>> >>
>> >> To search Galaxy mailing lists use the unified search at:
>> >>   http://galaxyproject.org/search/
>> >
>> >
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/