Quantcast

Problem with installing tools using Conda

classic Classic list List threaded Threaded
7 messages Options
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Problem with installing tools using Conda

Ashok Varadharajan
Dear Team,

I have problem with the installation of tools using conda. I am getting following error message when i try to install tools from galaxy. I tried to install these tools manually with the same conda used by the galaxy and it worked. Can anyone help us to solve this issue ?

Traceback (most recent call last):
  File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
    from conda.cli.main import main
ImportError: No module named conda.cli.main
galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,759 Removing failed conda install of samtools, version '1.3.1'
Traceback (most recent call last):
  File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
    from conda.cli.main import main
ImportError: No module named conda.cli.main
galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,941 Removing failed conda install of samtools, version 'None'
Traceback (most recent call last):
  File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
    from conda.cli.main import main
ImportError: No module named conda.cli.main
galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:50,142 Removing failed conda install of bwa, version '0.7.15'
Traceback (most recent call last):
  File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
    from conda.cli.main import main
ImportError: No module named conda.cli.main

We are running galaxy-16.07 behind the proxy using nginx, uwsgi and supervisor in scientific linux 7. Below are the details of galaxy.ini and supervisor configuration.


### galaxy.ini 

conda_prefix = /xxxx/xxxxx/xxxx-xxxx/conda/
#conda_exec = 
conda_debug = False
conda_ensure_channels = r,bioconda,iuc
#conda_auto_install = False
#conda_verbose_install_check=False
conda_auto_init = True

### supervisor

[supervisord]

[program:galaxy_web]
command         = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = $GALAXY_PATH
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
numprocs = 1
stoplight = INT
startretries    = 15
process_name    = web%(process_num)s

[program:handler]
command         = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/
>galaxy_handler%(process_num)s.log
directory       = $GALAXY_PATH
process_name    = handler%(process_num)s
numprocs = 3
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
startretries    = 15

--
Thanks and Regards,
Ashok Vardharajan


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

John Chilton-4
I've never seen anything like this before - I am sorry. Hopefully
someone else will have a more concrete idea.

Does that Conda executable work on the command line?

I'd try removing the whole Conda folder and rebuilding it. Is it
possible the install was corrupted at some point - maybe during
install?

The just released Galaxy 17.01 has much better logging of Conda
problems and a script that can be used outside of Galaxy to explicitly
setup Conda.

-John

On Thu, Mar 2, 2017 at 11:47 AM, Ashok Varadharajan <[hidden email]> wrote:

> Dear Team,
>
> I have problem with the installation of tools using conda. I am getting
> following error message when i try to install tools from galaxy. I tried to
> install these tools manually with the same conda used by the galaxy and it
> worked. Can anyone help us to solve this issue ?
>
> Traceback (most recent call last):
>   File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
>     from conda.cli.main import main
> ImportError: No module named conda.cli.main
> galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,759 Removing
> failed conda install of samtools, version '1.3.1'
> Traceback (most recent call last):
>   File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
>     from conda.cli.main import main
> ImportError: No module named conda.cli.main
> galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,941 Removing
> failed conda install of samtools, version 'None'
> Traceback (most recent call last):
>   File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
>     from conda.cli.main import main
> ImportError: No module named conda.cli.main
> galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:50,142 Removing
> failed conda install of bwa, version '0.7.15'
> Traceback (most recent call last):
>   File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module>
>     from conda.cli.main import main
> ImportError: No module named conda.cli.main
>
> We are running galaxy-16.07 behind the proxy using nginx, uwsgi and
> supervisor in scientific linux 7. Below are the details of galaxy.ini and
> supervisor configuration.
>
>
> ### galaxy.ini
>
> conda_prefix = /xxxx/xxxxx/xxxx-xxxx/conda/
> #conda_exec =
> conda_debug = False
> conda_ensure_channels = r,bioconda,iuc
> #conda_auto_install = False
> #conda_verbose_install_check=False
> conda_auto_init = True
>
> ### supervisor
>
> [supervisord]
>
> [program:galaxy_web]
> command         = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv
> $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate
> --master --processes 3 --threads 6 --logto
> $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib
> --stats 127.0.0.1:9191
> directory       = $GALAXY_PATH
> umask           = 022
> autostart       = true
> autorestart     = true
> startsecs       = 20
> user            = galaxy
> environment     =
> PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
> numprocs = 1
> stoplight = INT
> startretries    = 15
> process_name    = web%(process_num)s
>
> [program:handler]
> command         = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c
> $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s
> --log-file=$GALAXY_PATH/logs/
>>galaxy_handler%(process_num)s.log
> directory       = $GALAXY_PATH
> process_name    = handler%(process_num)s
> numprocs = 3
> umask           = 022
> autostart       = true
> autorestart     = true
> startsecs       = 20
> user            = galaxy
> environment     =
> PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
> startretries    = 15
>
> --
> Thanks and Regards,
> Ashok Vardharajan
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

Björn Grüning-3
In reply to this post by Ashok Varadharajan
Hi Ashok,

this seems to be a problem with your conda installation.

See for example here: https://github.com/conda/conda/issues/2463
Can you update your conda manually?

Cheers,
Bjoern

Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
> ImportError: No module named conda.cli.main
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

Ashok Varadharajan
Thanks John and Bjoern for your reply. 

Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)

I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed. 

I also tried updating the conda using "conda update conda" but didnt help. 

Do i have to adjust the PYTHONPATH in my supervisor configuration ?

Regards,
Ashok



On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <[hidden email]> wrote:
Hi Ashok,

this seems to be a problem with your conda installation.

See for example here: https://github.com/conda/conda/issues/2463
Can you update your conda manually?

Cheers,
Bjoern

Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
> ImportError: No module named conda.cli.main



--
Thanks and Regards,
Ashok Varadharajan

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

Marius van den Beek

Hi Ashok,

You should not set the PYTHONPATH in supervisor.
That is indeed going to conflict with conda.
If you are using uwsgi you can use something like this:

[program:galaxy_web]
command         = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate
 --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = /bioinfo/guests/mvandenb/galaxy
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
numprocs        = 1
stopsignal      = INT
startretries    = 15

Best,
Marius


On 3 March 2017 at 10:29, Ashok Varadharajan <[hidden email]> wrote:
Thanks John and Bjoern for your reply. 

Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)

I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed. 

I also tried updating the conda using "conda update conda" but didnt help. 

Do i have to adjust the PYTHONPATH in my supervisor configuration ?

Regards,
Ashok



On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <[hidden email]> wrote:
Hi Ashok,

this seems to be a problem with your conda installation.

See for example here: https://github.com/conda/conda/issues/2463
Can you update your conda manually?

Cheers,
Bjoern

Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
> ImportError: No module named conda.cli.main



--
Thanks and Regards,
Ashok Varadharajan

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

Ashok Varadharajan
Hi Marius,

Thanks for your reply. 

Below is my supervisor configuration and it looks similar to yours except "PYTHONHOME" in the environment section. Do you think that would be the issue ?

[supervisord]

[program:galaxy_web]
command         = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = $GALAXY_PATH
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
numprocs = 1
stoplight = INT
startretries    = 15
process_name    = web%(process_num)s

[program:handler]
command         = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/
>galaxy_handler%(process_num)s.log
directory       = $GALAXY_PATH
process_name    = handler%(process_num)s
numprocs = 3
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
startretries    = 15


Thanks
Ashok



On Fri, Mar 3, 2017 at 10:37 AM, Marius van den Beek <[hidden email]> wrote:

Hi Ashok,

You should not set the PYTHONPATH in supervisor.
That is indeed going to conflict with conda.
If you are using uwsgi you can use something like this:

[program:galaxy_web]
command         = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate
 --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = /bioinfo/guests/mvandenb/galaxy
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
numprocs        = 1
stopsignal      = INT
startretries    = 15

Best,
Marius


On 3 March 2017 at 10:29, Ashok Varadharajan <[hidden email]> wrote:
Thanks John and Bjoern for your reply. 

Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)

I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed. 

I also tried updating the conda using "conda update conda" but didnt help. 

Do i have to adjust the PYTHONPATH in my supervisor configuration ?

Regards,
Ashok



On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <[hidden email]> wrote:
Hi Ashok,

this seems to be a problem with your conda installation.

See for example here: https://github.com/conda/conda/issues/2463
Can you update your conda manually?

Cheers,
Bjoern

Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
> ImportError: No module named conda.cli.main



--
Thanks and Regards,
Ashok Varadharajan

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




--
Thanks and Regards,
Ashok Varadharajan

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Problem with installing tools using Conda

Marius van den Beek

Yep:

PYTHONHOME=~/.venv conda --help
Traceback (most recent call last):
  File "/home/marius/conda/bin/conda", line 3, in <module>
    from conda.cli import main
ImportError: No module named conda.cli

On 3 March 2017 at 11:04, Ashok Varadharajan <[hidden email]> wrote:
Hi Marius,

Thanks for your reply. 

Below is my supervisor configuration and it looks similar to yours except "PYTHONHOME" in the environment section. Do you think that would be the issue ?

[supervisord]

[program:galaxy_web]
command         = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = $GALAXY_PATH
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
numprocs = 1
stoplight = INT
startretries    = 15
process_name    = web%(process_num)s

[program:handler]
command         = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/
>galaxy_handler%(process_num)s.log
directory       = $GALAXY_PATH
process_name    = handler%(process_num)s
numprocs = 3
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so
startretries    = 15


Thanks
Ashok



On Fri, Mar 3, 2017 at 10:37 AM, Marius van den Beek <[hidden email]> wrote:

Hi Ashok,

You should not set the PYTHONPATH in supervisor.
That is indeed going to conflict with conda.
If you are using uwsgi you can use something like this:

[program:galaxy_web]
command         = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate
 --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191
directory       = /bioinfo/guests/mvandenb/galaxy
umask           = 022
autostart       = true
autorestart     = true
startsecs       = 20
user            = galaxy
environment     = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
numprocs        = 1
stopsignal      = INT
startretries    = 15

Best,
Marius


On 3 March 2017 at 10:29, Ashok Varadharajan <[hidden email]> wrote:
Thanks John and Bjoern for your reply. 

Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)

I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed. 

I also tried updating the conda using "conda update conda" but didnt help. 

Do i have to adjust the PYTHONPATH in my supervisor configuration ?

Regards,
Ashok



On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <[hidden email]> wrote:
Hi Ashok,

this seems to be a problem with your conda installation.

See for example here: https://github.com/conda/conda/issues/2463
Can you update your conda manually?

Cheers,
Bjoern

Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
> ImportError: No module named conda.cli.main



--
Thanks and Regards,
Ashok Varadharajan

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




--
Thanks and Regards,
Ashok Varadharajan


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Loading...