Missing configuration and location files automatic creation

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Missing configuration and location files automatic creation

Julien Seiler-2
Hi all,

I have notice that the run.sh script is creating automatically all missing configuration and location files to make sure Galaxy can work properly at first launch. However, the script is not taking care of the actual settings of an existing universe_wsgi.ini file. As a result, if the universe_wsgi.ini file says that job_conf.xml is located at foo/bar location, the run.sh will create a job_conf.xml file at the root of the galaxy source code anyway.

I'm thinking of improving the run.sh behavior to respect file path declared in an existing universe_wsgi.ini file in order not to create unused config or location files. I have currently started to implement this behavior in a small Python script (more handy than pure sh).
What do you think about this concern ? Should I go for a pull request at any point ?

Regards,

--
Julien


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Missing configuration and location files automatic creation

Nate Coraor (nate@bx.psu.edu)
Hi Julien,

What timing for this email - we have just made significant changes to the default config layout, and this includes no longer copying sample configs to their non-sample locations. The changes are in our default branch right now and will be part of the stable release scheduled for October. Have a look, I'd be interested to hear your feedback since you've already put some thought into this.

--nate

On Fri, Sep 19, 2014 at 3:10 PM, Julien Seiler <[hidden email]> wrote:
Hi all,

I have notice that the run.sh script is creating automatically all missing configuration and location files to make sure Galaxy can work properly at first launch. However, the script is not taking care of the actual settings of an existing universe_wsgi.ini file. As a result, if the universe_wsgi.ini file says that job_conf.xml is located at foo/bar location, the run.sh will create a job_conf.xml file at the root of the galaxy source code anyway.

I'm thinking of improving the run.sh behavior to respect file path declared in an existing universe_wsgi.ini file in order not to create unused config or location files. I have currently started to implement this behavior in a small Python script (more handy than pure sh).
What do you think about this concern ? Should I go for a pull request at any point ?

Regards,

--
Julien


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Missing configuration and location files automatic creation

Julien Seiler-2
Hi Nate,

Indeed, I have noticed some significant improvements in the config files layout.
All config files have been moved to the config/ directory and the automatic creation of missing config/location files is now handled by the common_startup.sh script. This is great but it is not solving my main concern about creating « unused » config files at startup.

To be more specific, I will give you a short description of our Galaxy instance layout. We are currently sharing the sources of our Galaxy instance because we want to share our Vagrant setup with Galaxy tools developers. However, we actually don’t want to share all our config files but still want to keep them under version control. That’s why we decided to move all our config/location files in a private repository and point to this specific location in our main galaxy.ini config.
In my opinion, if a galaxy.ini file already existing and is pointing to non-default location for some config files ou tool-data path, the Galaxy startup script should not be creating the corresponding default config files to the default location.

My idea was to refactor the common_startup.sh script in order to parse an eventually existing galaxy.ini and then create each default config files to the location pointed by galaxy.ini.

--
Julien Seiler

Le 20 sept. 2014 à 03:10, Nate Coraor <[hidden email]> a écrit :

Hi Julien,

What timing for this email - we have just made significant changes to the default config layout, and this includes no longer copying sample configs to their non-sample locations. The changes are in our default branch right now and will be part of the stable release scheduled for October. Have a look, I'd be interested to hear your feedback since you've already put some thought into this.

--nate

On Fri, Sep 19, 2014 at 3:10 PM, Julien Seiler <[hidden email]> wrote:
Hi all,

I have notice that the run.sh script is creating automatically all missing configuration and location files to make sure Galaxy can work properly at first launch. However, the script is not taking care of the actual settings of an existing universe_wsgi.ini file. As a result, if the universe_wsgi.ini file says that job_conf.xml is located at foo/bar location, the run.sh will create a job_conf.xml file at the root of the galaxy source code anyway.

I'm thinking of improving the run.sh behavior to respect file path declared in an existing universe_wsgi.ini file in order not to create unused config or location files. I have currently started to implement this behavior in a small Python script (more handy than pure sh).
What do you think about this concern ? Should I go for a pull request at any point ?

Regards,

--
Julien


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Missing configuration and location files automatic creation

Nate Coraor (nate@bx.psu.edu)
Hi Julien,

I agree with your idea. As you say, there are still a number of defaults that we will set up with common_startup.sh even if they are configured in galaxy.ini. I had always assumed that most deployers who are using custom locations are bypassing run.sh anyway. For usegalaxy.org we just call fetch_eggs.py prior to starting and invoke Galaxy directly (via uWSGI for the web processes and individual calls to `python ./scripts/paster.py serve /path/to/galaxy.ini` for the handlers). run.sh is intended as more of a bootstrap script for uncustomized copies of Galaxy, but I do realize people use it as the primary method for controlling their servers.

We intend to address the remaining items in common_startup.sh in a similar fashion to the config files we have already moved to config/ - namely, attempting to use the .sample file if the non-.sample does not exist and the location is the default. The only case where this won't be possible is with the first four files, which Galaxy needs to be able to modify. So I am not trying to deter you, but I also don't want you to waste a lot of effort on this.

Thanks,
--nate

On Sat, Sep 20, 2014 at 12:56 AM, Julien Seiler <[hidden email]> wrote:
Hi Nate,

Indeed, I have noticed some significant improvements in the config files layout.
All config files have been moved to the config/ directory and the automatic creation of missing config/location files is now handled by the common_startup.sh script. This is great but it is not solving my main concern about creating « unused » config files at startup.

To be more specific, I will give you a short description of our Galaxy instance layout. We are currently sharing the sources of our Galaxy instance because we want to share our Vagrant setup with Galaxy tools developers. However, we actually don’t want to share all our config files but still want to keep them under version control. That’s why we decided to move all our config/location files in a private repository and point to this specific location in our main galaxy.ini config.
In my opinion, if a galaxy.ini file already existing and is pointing to non-default location for some config files ou tool-data path, the Galaxy startup script should not be creating the corresponding default config files to the default location.

My idea was to refactor the common_startup.sh script in order to parse an eventually existing galaxy.ini and then create each default config files to the location pointed by galaxy.ini.

--
Julien Seiler

Le 20 sept. 2014 à 03:10, Nate Coraor <[hidden email]> a écrit :

Hi Julien,

What timing for this email - we have just made significant changes to the default config layout, and this includes no longer copying sample configs to their non-sample locations. The changes are in our default branch right now and will be part of the stable release scheduled for October. Have a look, I'd be interested to hear your feedback since you've already put some thought into this.

--nate

On Fri, Sep 19, 2014 at 3:10 PM, Julien Seiler <[hidden email]> wrote:
Hi all,

I have notice that the run.sh script is creating automatically all missing configuration and location files to make sure Galaxy can work properly at first launch. However, the script is not taking care of the actual settings of an existing universe_wsgi.ini file. As a result, if the universe_wsgi.ini file says that job_conf.xml is located at foo/bar location, the run.sh will create a job_conf.xml file at the root of the galaxy source code anyway.

I'm thinking of improving the run.sh behavior to respect file path declared in an existing universe_wsgi.ini file in order not to create unused config or location files. I have currently started to implement this behavior in a small Python script (more handy than pure sh).
What do you think about this concern ? Should I go for a pull request at any point ?

Regards,

--
Julien


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/