Galaxy on CentOS?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Galaxy on CentOS?

Carlos Lijeron

Hello everyone,

 

We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager.   We are wondering if there is a guideline on how to install and support Galaxy on CentOS.  Does anyone have any info on this?

 

Thanks !

 


Carlos.

 


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

Peter Cock
Hi Carlos,

We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a
problem. Most of the effort was sorting out shared storage with the
cluster, which in our case is managed with SGE (fairly commonly
used with Galaxy).

In reply to your thread last month David Trudgian said he was using
Bright Cluster Manager:

http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager-tc4667015.html

Regards,

Peter


On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <[hidden email]> wrote:

> Hello everyone,
>
> We are soon acquiring our cluster which has CentOS on the master node along
> with Bright Cluster Manager and workload manager.   We are wondering if
> there is a guideline on how to install and support Galaxy on CentOS.  Does
> anyone have any info on this?
>
> Thanks !
>
> Carlos.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

Zuzanna K. Filutowska
In reply to this post by Carlos Lijeron
Dnia 2015-05-05, wto o godzinie 01:51 +0000, Carlos Lijeron pisze:
> Hello everyone,
>
>  
>
> We are soon acquiring our cluster which has CentOS on the master node
> along with Bright Cluster Manager and workload manager.   We are
> wondering if there is a guideline on how to install and support Galaxy
> on CentOS.  Does anyone have any info on this?

I am running Galaxy on Centos 6.6. But we are considering to move to
Fedora Server. Of course it is possible to run Galaxy on Centos, however
I had to compile python and several other packets to make all tools to
work.


--

Pozdrawiam,

--
Zuzanna K. Filutowska, HPC Systems Administrator
Poznan Supercomputing and Networking Center
Institute of Bioorganic Chemistry Polish Academy of Sciences
Seize the day boys! Make your lifes extraordinary! --John Keating

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

David Trudgian
In reply to this post by Peter Cock
Hi Carlos,

Our Galaxy is running on RedHat 6.6 on our cluster - which is basically identical to CentOS 6.6. Per Peter's email we haven't had any big issues with Galaxy on CentOS 6.x.

You are likely to want to use the postgresql packages from www.postrgresql.org, not the out-of-date versions that come with the distribution. I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6. In my case I did it to have a standard setup with other apps that do need python 2.7.

Which workload manager will your cluster manager be configured to use?

DT




-----Original Message-----
From: galaxy-dev [mailto:[hidden email]] On Behalf Of Peter Cock
Sent: Tuesday, May 05, 2015 3:13 AM
To: Carlos Lijeron
Cc: [hidden email]
Subject: Re: [galaxy-dev] Galaxy on CentOS?

Hi Carlos,

We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy).

In reply to your thread last month David Trudgian said he was using Bright Cluster Manager:

http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager-tc4667015.html

Regards,

Peter


On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <[hidden email]> wrote:

> Hello everyone,
>
> We are soon acquiring our cluster which has CentOS on the master node along
> with Bright Cluster Manager and workload manager.   We are wondering if
> there is a guideline on how to install and support Galaxy on CentOS.
> Does anyone have any info on this?
>
> Thanks !
>
> Carlos.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

________________________________

UT Southwestern


Medical Center



The future of medicine, today.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

Carlos Lijeron
In reply to this post by Zuzanna K. Filutowska
Thank you for the feedback Zuzanna,

Would you have documentation on scripts your updated or any lessons
learned during your deployment?  I¹m trying to get as much feedback as
possible to hit the ground running when the cluster is delivered in our
facility.

One more question, do you use SGI or SLURM for the workload manager?

Thanks again !

Carlos.


On 5/5/15, 9:41 AM, "Zuzanna K. Filutowska" <[hidden email]> wrote:

>>wondering if there is a guideline on how to install and support Galaxy
>>on CentOS.  Does anyone have any info on this?
>
>I am running Galaxy on Centos 6.6. But we are considering to move to
>Fedora Server. Of course it is possible to run Galaxy on Centos, however
>I had to compile python and several other packets to make all tools to
>work.
>
>
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

Carlos Lijeron
In reply to this post by Peter Cock
Thank you Peter, I am now getting a clear idea on how our deployment will
go.  Yes, I followed up with David and he provided additional insights
into his deployment.  What a great virtual team !!!  Thank you.


Carlos

On 5/5/15, 4:12 AM, "Peter Cock" <[hidden email]> wrote:

>Hi Carlos,
>
>We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a
>problem. Most of the effort was sorting out shared storage with the
>cluster, which in our case is managed with SGE (fairly commonly
>used with Galaxy).
>
>In reply to your thread last month David Trudgian said he was using
>Bright Cluster Manager:
>
>http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager
>-tc4667015.html
>
>Regards,
>
>Peter
>
>
>On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <[hidden email]>
>wrote:
>> Hello everyone,
>>
>> We are soon acquiring our cluster which has CentOS on the master node
>>along
>> with Bright Cluster Manager and workload manager.   We are wondering if
>> there is a guideline on how to install and support Galaxy on CentOS.
>>Does
>> anyone have any info on this?
>>
>> Thanks !
>>
>> Carlos.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

Carlos Lijeron
In reply to this post by David Trudgian
David,

We are in our final configuration stage and tomorrow we have a
conversation with Silicon Mechanics business integration team to discuss
the workload manager, and other details about the data center backbone
speed.

As per your suggestion I inquired about using SLURM instead of SGI, but I
haven¹t been able to make a good case as to why this is.  My only support
is your email / suggestion and your existing experience with an HPC,
Bright and SLURM.

My inexperience with this type of setups and lack of personnel at our
institution makes me rely 100% of everyone's feedback in this group

Thank you David and anyone else in the group for your contributions and
suggestions.


Attached is the diagram of our future implementation.  Another concern
that I have is that our sequencer will be located 0.5 miles from the HPC,
but there is a 10 Gpbs point to point connection between the 2 buildings.
I wanted to find a way to calculate / predict the average amount of
traffic that will go from the sequencer to the HPC so we can allocate
specific bandwidth for our use and leave the rest to other users.


Carlos.



On 5/5/15, 10:24 AM, "David Trudgian" <[hidden email]>
wrote:

>
>Our Galaxy is running on RedHat 6.6 on our cluster - which is basically
>identical to CentOS 6.6. Per Peter's email we haven't had any big issues
>with Galaxy on CentOS 6.x.
>
>You are likely to want to use the postgresql packages from
>www.postrgresql.org, not the out-of-date versions that come with the
>distribution. I also installed an up-to-date python 2.7.x for Galaxy
>using pyenv but this shouldn't be necessary - Galaxy should work fine on
>python 2.6. In my case I did it to have a standard setup with other apps
>that do need python 2.7.
>
>Which workload manager will your cluster manager be configured to use?
>
>DT
>
>
>
>
>-----Original Message-----
>From: galaxy-dev [mailto:[hidden email]] On
>Behalf Of Peter Cock
>Sent: Tuesday, May 05, 2015 3:13 AM
>To: Carlos Lijeron
>Cc: [hidden email]
>Subject: Re: [galaxy-dev] Galaxy on CentOS?
>
>Hi Carlos,
>
>We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a
>problem. Most of the effort was sorting out shared storage with the
>cluster, which in our case is managed with SGE (fairly commonly used with
>Galaxy).
>
>In reply to your thread last month David Trudgian said he was using
>Bright Cluster Manager:
>
>http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager
>-tc4667015.html
>
>Regards,
>
>Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Hunter HPC.pdf (684K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

David Trudgian
Hi Carlos,

I previously used Galaxy with a GridEngine cluster. Having GridEngine is no more complex than having SLURM.

R.E. you other email, recording cluster usage for labs' galaxy jobs, that requires the 'running jobs as real user' setup...

https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_Jobs_as_the_Real_User

There are caveats with that setup r.e. file permissions / privacy of data between groups. In our situation everything has to be kept private, so to ensure jobs run as a real cluster user we have a complex setup using Galaxy pulsar as well as SLURM. Some notes on that in a GitHub issue here:

https://github.com/galaxyproject/pulsar/issues/65

Cheers,

DT



-----Original Message-----
From: Carlos Lijeron [mailto:[hidden email]]
Sent: Tuesday, May 05, 2015 11:18 AM
To: David Trudgian; Peter Cock
Cc: [hidden email]
Subject: Re: [galaxy-dev] Galaxy on CentOS?

David,

We are in our final configuration stage and tomorrow we have a conversation with Silicon Mechanics business integration team to discuss the workload manager, and other details about the data center backbone speed.

As per your suggestion I inquired about using SLURM instead of SGI, but I haven¹t been able to make a good case as to why this is.  My only support is your email / suggestion and your existing experience with an HPC, Bright and SLURM.

My inexperience with this type of setups and lack of personnel at our institution makes me rely 100% of everyone's feedback in this group

Thank you David and anyone else in the group for your contributions and suggestions.


Attached is the diagram of our future implementation.  Another concern that I have is that our sequencer will be located 0.5 miles from the HPC, but there is a 10 Gpbs point to point connection between the 2 buildings.
I wanted to find a way to calculate / predict the average amount of traffic that will go from the sequencer to the HPC so we can allocate specific bandwidth for our use and leave the rest to other users.


Carlos.



On 5/5/15, 10:24 AM, "David Trudgian" <[hidden email]>
wrote:

>
>Our Galaxy is running on RedHat 6.6 on our cluster - which is basically
>identical to CentOS 6.6. Per Peter's email we haven't had any big
>issues with Galaxy on CentOS 6.x.
>
>You are likely to want to use the postgresql packages from
>www.postrgresql.org, not the out-of-date versions that come with the
>distribution. I also installed an up-to-date python 2.7.x for Galaxy
>using pyenv but this shouldn't be necessary - Galaxy should work fine
>on python 2.6. In my case I did it to have a standard setup with other
>apps that do need python 2.7.
>
>Which workload manager will your cluster manager be configured to use?
>
>DT
>
>
>
>
>-----Original Message-----
>From: galaxy-dev [mailto:[hidden email]] On
>Behalf Of Peter Cock
>Sent: Tuesday, May 05, 2015 3:13 AM
>To: Carlos Lijeron
>Cc: [hidden email]
>Subject: Re: [galaxy-dev] Galaxy on CentOS?
>
>Hi Carlos,
>
>We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a
>problem. Most of the effort was sorting out shared storage with the
>cluster, which in our case is managed with SGE (fairly commonly used
>with Galaxy).
>
>In reply to your thread last month David Trudgian said he was using
>Bright Cluster Manager:
>
>http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Mana
>ger
>-tc4667015.html
>
>Regards,
>
>Peter


________________________________

UT Southwestern


Medical Center



The future of medicine, today.


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Galaxy on CentOS?

James Taylor-4
In reply to this post by David Trudgian
On Tue, May 5, 2015 at 10:24 AM, David Trudgian
<[hidden email]> wrote:
> I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6.

I would strongly suggest using the latest Python 2.7 for any new
Galaxy deployment. Though there is no plan to right now, inevitably we
will have to deprecate 2.7 support. In particular, if we want to do a
Python 2 and 3 compatible Galaxy we would need to use 2.7 features.

--
James Taylor
Ralph S. O'Connor Associate Professor of Biology
Associate Professor of Computer Science
Johns Hopkins University
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/