$GALAXY_TEST_DB_TEMPLATE failing with KeyError: <VerNum(127)>

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

$GALAXY_TEST_DB_TEMPLATE failing with KeyError: <VerNum(127)>

Peter Cock
Hi all,

I've been using the SQLite template database feature to speed up
functional testing. Rather than starting for the initial schema and
applying all the incremental changes (as done when installing a
fresh Galaxy instance), this lets the test suite start with a recent
version of the database, by setting the environment variable
$GALAXY_TEST_DB_TEMPLATE to a URL to fetch a template
database, e.g. from:

https://github.com/jmchilton/galaxy-downloads

Database schema version 120 was working, changing to 127 broke:

https://github.com/peterjc/pico_galaxy/commit/412d3fe88b8011f0b690956d129fd72f5b5d4c5c

TravisCI using v120, worked:
https://travis-ci.org/peterjc/pico_galaxy/builds/52308064

TravisCI using v127, broke:
https://travis-ci.org/peterjc/pico_galaxy/builds/52398778

galaxy.model.migrate.check DEBUG 2015-02-27 10:05:33,274 pysqlite>=2
egg successfully loaded for sqlite dialect
Traceback (most recent call last):
  File "./scripts/functional_tests.py", line 602, in <module>
    sys.exit( main() )
  File "./scripts/functional_tests.py", line 409, in main
    app = UniverseApplication( **kwargs )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/app.py",
line 49, in __init__
    self._configure_models( check_migrate_databases=True,
check_migrate_tools=check_migrate_tools, config_file=config_file )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/config.py",
line 779, in _configure_models
    create_or_verify_database( db_url, config_file,
self.config.database_engine_options, app=self )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
line 61, in create_or_verify_database
    migrate()
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
line 57, in migrate
    migrate_to_current_version( engine, db_schema )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
line 117, in migrate_to_current_version
    changeset = schema.changeset( None )
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py",
line 80, in changeset
    changeset = self.repository.changeset(database, start_ver, version)
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py",
line 225, in changeset
    changes = [self.version(v).script(database, op) for v in versions]
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py",
line 189, in version
    return self.versions.version(*p, **k)
  File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/version.py",
line 140, in version
    return self.versions[VerNum(vernum)]
KeyError: <VerNum(127)>

This is now testing against the (beta) Galaxy on GitHub:
https://github.com/galaxyproject/galaxy

Is it possible this is out of synch with the SQLite templates John
posts here? https://github.com/jmchilton/galaxy-downloads

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: $GALAXY_TEST_DB_TEMPLATE failing with KeyError: <VerNum(127)>

John Chilton-4
Hey Peter,

  That database is pretty new (not yet in stable) - but you are
tracking the master branch of galaxy on github in your tests (which is
sortof like stable). You will want to either track the dev branch (so
you can continue to catch all the errors right away :)) or track
master and use an older database.

  Someday soon I will move this logic into Galaxy's run_tests.sh so
that Galaxy always knows what version of the database it needs and we
don't have to deal with these version incompatibilities. Sorry for the
confusion.

Thanks,
-John




On Fri, Feb 27, 2015 at 9:07 AM, Peter Cock <[hidden email]> wrote:

> Hi all,
>
> I've been using the SQLite template database feature to speed up
> functional testing. Rather than starting for the initial schema and
> applying all the incremental changes (as done when installing a
> fresh Galaxy instance), this lets the test suite start with a recent
> version of the database, by setting the environment variable
> $GALAXY_TEST_DB_TEMPLATE to a URL to fetch a template
> database, e.g. from:
>
> https://github.com/jmchilton/galaxy-downloads
>
> Database schema version 120 was working, changing to 127 broke:
>
> https://github.com/peterjc/pico_galaxy/commit/412d3fe88b8011f0b690956d129fd72f5b5d4c5c
>
> TravisCI using v120, worked:
> https://travis-ci.org/peterjc/pico_galaxy/builds/52308064
>
> TravisCI using v127, broke:
> https://travis-ci.org/peterjc/pico_galaxy/builds/52398778
>
> galaxy.model.migrate.check DEBUG 2015-02-27 10:05:33,274 pysqlite>=2
> egg successfully loaded for sqlite dialect
> Traceback (most recent call last):
>   File "./scripts/functional_tests.py", line 602, in <module>
>     sys.exit( main() )
>   File "./scripts/functional_tests.py", line 409, in main
>     app = UniverseApplication( **kwargs )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/app.py",
> line 49, in __init__
>     self._configure_models( check_migrate_databases=True,
> check_migrate_tools=check_migrate_tools, config_file=config_file )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/config.py",
> line 779, in _configure_models
>     create_or_verify_database( db_url, config_file,
> self.config.database_engine_options, app=self )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
> line 61, in create_or_verify_database
>     migrate()
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
> line 57, in migrate
>     migrate_to_current_version( engine, db_schema )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py",
> line 117, in migrate_to_current_version
>     changeset = schema.changeset( None )
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py",
> line 80, in changeset
>     changeset = self.repository.changeset(database, start_ver, version)
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py",
> line 225, in changeset
>     changes = [self.version(v).script(database, op) for v in versions]
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py",
> line 189, in version
>     return self.versions.version(*p, **k)
>   File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/version.py",
> line 140, in version
>     return self.versions[VerNum(vernum)]
> KeyError: <VerNum(127)>
>
> This is now testing against the (beta) Galaxy on GitHub:
> https://github.com/galaxyproject/galaxy
>
> Is it possible this is out of synch with the SQLite templates John
> posts here? https://github.com/jmchilton/galaxy-downloads
>
> Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: $GALAXY_TEST_DB_TEMPLATE failing with KeyError: <VerNum(127)>

Peter Cock
On Fri, Feb 27, 2015 at 2:15 PM, John Chilton <[hidden email]> wrote:
> Hey Peter,
>
>   That database is pretty new (not yet in stable) - but you are
> tracking the master branch of galaxy on github in your tests (which is
> sortof like stable). You will want to either track the dev branch (so
> you can continue to catch all the errors right away :)) or track
> master and use an older database.

Ah - that makes sense - wrongly assumed the master was
the default and development branch.

(This probably also explains why the galaxy_blast tests stopped
working, installing the blast.py custom datatypes was failing.)

>   Someday soon I will move this logic into Galaxy's run_tests.sh so
> that Galaxy always knows what version of the database it needs and we
> don't have to deal with these version incompatibilities. Sorry for the
> confusion.

Yes please - an explicit error message for "Your database schema
(version XXX) is too new for this version of Galaxy" should do the
trick (not just for run_tests.sh but one once in the database code
to cover all of Galaxy).

Thanks,

Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/