Two instances on different server with only one postgres database ?

classic Classic list List threaded Threaded
5 messages Options
| Threaded
Open this post in threaded view
|

Two instances on different server with only one postgres database ?

Jean-Philippe Villemin-2

Hi,

Is this possible to set two instances of galaxy on server 1 and server 2, and to install only one database on server 1 for both of them.

I'm actually trying to do that and I don't know if it's a mad idea.

The two instances launch well but when i try to upload file , I get some errors about failing the erase of some metada ....

If I only run one of the instance (even instance on server 1 and database on server 2) it will works fine.

An idea about that ? thanks.


--
Jean-Philippe Villemin   Bioinformatics, Software Engineer -
IURC (Institut Universitaire de Recherche Clinique)
641 avenue du Doyen Gaston Giraud
34093 Montpellier Cedex 5, France
[hidden email]


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Two instances on different server with only one postgres database ?

Nicola Soranzo-2
Hi Jean-Philippe,
it's not clear to me what you are trying to do: are you trying to create a mirrored Galaxy server (same users, same data libraries, same tools, same histories for each user,...) or 2 different Galaxy servers using the same PostgreSQL database *server* as backend?

If it's the latter, then you just need to create 2 different databases (say galaxy_db1 and galaxy_db2) on the same PostgreSQL *server*. If you want to be more secure, the 2 database should be owned by different PostgreSQL users.

Cheers,
Nicola

On 09/12/15 10:32, Jean-Philippe Villemin wrote:

Hi,

Is this possible to set two instances of galaxy on server 1 and server 2, and to install only one database on server 1 for both of them.

I'm actually trying to do that and I don't know if it's a mad idea.

The two instances launch well but when i try to upload file , I get some errors about failing the erase of some metada ....

If I only run one of the instance (even instance on server 1 and database on server 2) it will works fine.

An idea about that ? thanks.


--
Jean-Philippe Villemin   Bioinformatics, Software Engineer -
IURC (Institut Universitaire de Recherche Clinique)
641 avenue du Doyen Gaston Giraud
34093 Montpellier Cedex 5, France
[hidden email]



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Two instances on different server with only one postgres database ?

Jean-Philippe Villemin-2
Hi,

Initially we have two galaxy. One instance for production on server 1, One instance for dev on server 2.
Now we want to move on because more users are using the production instance and our server is not very powerful.

We'd like to have two production instances using both servers 1 & 2.
Depending on the job charge, we want (anyway the instance is accessed by the user), make a redirection on the less overloaded server.
Imagine user go http://server1/galaxy, if the load is very intense, than we'd like the new job throws under server2. I suppose you already think it's more a balancing problem than a database access.(yes, I will take about that, we came to this one database idea after having tried web balancing with apache with no sucess)

So we begun by an installation of two instances on server 1 & 2. Each one have an postgres server with a galaxy database.
Each one have a main job handler, and two job handlers on port 8081 & 8082.

We tried to configure web balancing in apache, it's was not working so well. It's actually working well if I we stay on the same server and balance over port 8081 / 8082. In this case the balancing is made on the same server whereas it would be better if the load is shared between two distinct servers.
So we decided then to set one database to have the same tools, histories, users on both instances, and we hoped that it magically solves our problem of balancing.

I wrote a tutorial here on github about what we have done on the server , if you'd like to give a look...(but sorry it's in french :-/)

So maybe i should rewrite the question : How should configure galaxies instances to balance between servers taking account of the job/request charge ? ( and can you do the difference between web request charge and job loading ? )

I hope that is clear.

Thanks,



2015-12-09 12:23 GMT+01:00 Nicola Soranzo <[hidden email]>:
Hi Jean-Philippe,
it's not clear to me what you are trying to do: are you trying to create a mirrored Galaxy server (same users, same data libraries, same tools, same histories for each user,...) or 2 different Galaxy servers using the same PostgreSQL database *server* as backend?

If it's the latter, then you just need to create 2 different databases (say galaxy_db1 and galaxy_db2) on the same PostgreSQL *server*. If you want to be more secure, the 2 database should be owned by different PostgreSQL users.

Cheers,
Nicola


On 09/12/15 10:32, Jean-Philippe Villemin wrote:

Hi,

Is this possible to set two instances of galaxy on server 1 and server 2, and to install only one database on server 1 for both of them.

I'm actually trying to do that and I don't know if it's a mad idea.

The two instances launch well but when i try to upload file , I get some errors about failing the erase of some metada ....

If I only run one of the instance (even instance on server 1 and database on server 2) it will works fine.

An idea about that ? thanks.


--
Jean-Philippe Villemin   Bioinformatics, Software Engineer -
IURC (Institut Universitaire de Recherche Clinique)
641 avenue du Doyen Gaston Giraud
34093 Montpellier Cedex 5, France
[hidden email]



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




--
Jean-Philippe Villemin   Bioinformatics, Software Engineer -
IURC (Institut Universitaire de Recherche Clinique)
641 avenue du Doyen Gaston Giraud
34093 Montpellier Cedex 5, France
[hidden email]


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Pass Working Directory from xml

Todd Oakley
Greetings,
     I feel this must be simple, but I cannot find it.

I'm making a wrapper for a tool that requires the output directory to be
specified on the command line of a python script (otherwise it uses the
path of the input file for the multiple output files). How can I pass
the working directory to the tool from the xml file?

<command interpreter="python">
     the_tool.py -o $DIRECTORY_WHERE_GALAXY_RUNS_THE_TOOL

      This is similar to from_work_dir in the outputs -- but I need the
path of the directory that does not include any file names.


Thanks!

Todd Oakley, UCSB
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
| Threaded
Open this post in threaded view
|

Re: Pass Working Directory from xml

Daniel Blankenberg
Hi Todd,

Can you just pass in “.”?

<command interpreter="python">
   the_tool.py -o ‘.'


Thanks for using Galaxy,

Dan


On Dec 11, 2015, at 4:00 PM, Todd Oakley <[hidden email]> wrote:

> Greetings,
>    I feel this must be simple, but I cannot find it.
>
> I'm making a wrapper for a tool that requires the output directory to be specified on the command line of a python script (otherwise it uses the path of the input file for the multiple output files). How can I pass the working directory to the tool from the xml file?
>
> <command interpreter="python">
>    the_tool.py -o $DIRECTORY_WHERE_GALAXY_RUNS_THE_TOOL
>
>     This is similar to from_work_dir in the outputs -- but I need the path of the directory that does not include any file names.
>
>
> Thanks!
>
> Todd Oakley, UCSB
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/