concatenate_datasets docker image can't create output

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

concatenate_datasets docker image can't create output

jeltje
Hi,
When I try to run the concatenate-datasets on two input files, two odd things happen. First, TWO nearly identical commands are generated in the History panel, only differing in their output filename. In paster.log, only the second one shows up. Second, the program fails, with this info in the History:

An error occurred with this dataset:
Galaxy slots passed through contain as 1 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh: line 2: can't create /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat

This happens for both commands in the History panel, only the output filename in the other error is dataset_15.dat

Oddly, the dataset_14.dat and dataset_15.dat are both created during this command, they just end up empty.

Paster.log:

galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker inspect busybox:ubuntu-14.04 > /dev/null 2>&1
[ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1

docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat ; fi; sh -c "exit $return_code"
galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing job script: /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job destination (destination id: docker_local)


What could be happening here, and how do I fix it?

Thanks,

-Jeltje



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

John Chilton-4
My first thought is the bug mentioned here -
https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
along with potential work arounds. Are you able to confirm that this
is or is not the problem at all?

-John

On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
<[hidden email]> wrote:

> Hi,
>
> I'm following instructions at
> https://github.com/apetkau/galaxy-hackathon-2014.
>
> When I try to run the concatenate-datasets on two input files, two odd
> things happen. First, TWO nearly identical commands are generated in the
> History panel, only differing in their output filename. In paster.log, only
> the second one shows up. Second, the program fails, with this info in the
> History:
>
> An error occurred with this dataset:
> Galaxy slots passed through contain as 1
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
> line 2: can't create
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>
> This happens for both commands in the History panel, only the output
> filename in the other error is dataset_15.dat
>
> Oddly, the dataset_14.dat and dataset_15.dat are both created during this
> command, they just end up empty.
>
> Paster.log:
>
> galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker
> inspect busybox:ubuntu-14.04 > /dev/null 2>&1
> [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>
> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
> -w
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
> --net none busybox:ubuntu-14.04
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
> return_code=$?; if [ -f
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> ] ; then cp
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
> ; fi; sh -c "exit $return_code"
> galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing job
> script:
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
> galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job destination
> (destination id: docker_local)
>
>
> What could be happening here, and how do I fix it?
>
> Thanks,
>
> -Jeltje
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

jeltje
I tried the second solution (setting everything to rw) and while I can confirm that the command is changed accordingly, the results are the same: Two empty output files and the same error message.

Command from paster.log:
docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat ; fi; sh -c "exit $return_code"

-Jeltje

On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]> wrote:
My first thought is the bug mentioned here -
https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
along with potential work arounds. Are you able to confirm that this
is or is not the problem at all?

-John

On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
<[hidden email]> wrote:
> Hi,
>
> I'm following instructions at
> https://github.com/apetkau/galaxy-hackathon-2014.
>
> When I try to run the concatenate-datasets on two input files, two odd
> things happen. First, TWO nearly identical commands are generated in the
> History panel, only differing in their output filename. In paster.log, only
> the second one shows up. Second, the program fails, with this info in the
> History:
>
> An error occurred with this dataset:
> Galaxy slots passed through contain as 1
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
> line 2: can't create
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>
> This happens for both commands in the History panel, only the output
> filename in the other error is dataset_15.dat
>
> Oddly, the dataset_14.dat and dataset_15.dat are both created during this
> command, they just end up empty.
>
> Paster.log:
>
> galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker
> inspect busybox:ubuntu-14.04 > /dev/null 2>&1
> [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>
> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
> -w
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
> --net none busybox:ubuntu-14.04
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
> return_code=$?; if [ -f
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> ] ; then cp
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
> ; fi; sh -c "exit $return_code"
> galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing job
> script:
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
> galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job destination
> (destination id: docker_local)
>
>
> What could be happening here, and how do I fix it?
>
> Thanks,
>
> -Jeltje
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

jeltje
I'm still trying to debug this.

How do I change the working directory? I can't seem to find out how all those directory mounts get passed to docker - the catDocker.xml appears to generate a 'cat' command that I assume ends up in
()/job_working_directory/000/12/container.sh
but I can't find where that's happening - container.sh isn't listed in any file under galaxy-dist.
job_conf.xml lists the directories that get passed as variables - again, when and where are those defined?

Thanks,

-Jeltje


On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren <[hidden email]> wrote:
I tried the second solution (setting everything to rw) and while I can confirm that the command is changed accordingly, the results are the same: Two empty output files and the same error message.

Command from paster.log:
docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw -v /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw -w /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12 --net none busybox:ubuntu-14.04 /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh; return_code=$?; if [ -f /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file ] ; then cp /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat ; fi; sh -c "exit $return_code"

-Jeltje

On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]> wrote:
My first thought is the bug mentioned here -
https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
along with potential work arounds. Are you able to confirm that this
is or is not the problem at all?

-John

On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
<[hidden email]> wrote:
> Hi,
>
> I'm following instructions at
> https://github.com/apetkau/galaxy-hackathon-2014.
>
> When I try to run the concatenate-datasets on two input files, two odd
> things happen. First, TWO nearly identical commands are generated in the
> History panel, only differing in their output filename. In paster.log, only
> the second one shows up. Second, the program fails, with this info in the
> History:
>
> An error occurred with this dataset:
> Galaxy slots passed through contain as 1
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
> line 2: can't create
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>
> This happens for both commands in the History panel, only the output
> filename in the other error is dataset_15.dat
>
> Oddly, the dataset_14.dat and dataset_15.dat are both created during this
> command, they just end up empty.
>
> Paster.log:
>
> galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is: docker
> inspect busybox:ubuntu-14.04 > /dev/null 2>&1
> [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>
> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
> -v
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
> -w
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
> --net none busybox:ubuntu-14.04
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
> return_code=$?; if [ -f
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> ] ; then cp
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
> ; fi; sh -c "exit $return_code"
> galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing job
> script:
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
> galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job destination
> (destination id: docker_local)
>
>
> What could be happening here, and how do I fix it?
>
> Thanks,
>
> -Jeltje
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

John Chilton-4
Can you send me a screenshot of the Tool form right before you hit
submit. It is odd that two commands are executed barely differing at
all. There were some recent UI changes and I want to make sure that
you are passing two inputs once and not submitting one input twice in
the newly named "batch mode"?

container.sh is generated in command_factory.py using containers.py -
here are some relevant links.

https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default

-John


On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren
<[hidden email]> wrote:

> I'm still trying to debug this.
>
> How do I change the working directory? I can't seem to find out how all
> those directory mounts get passed to docker - the catDocker.xml appears to
> generate a 'cat' command that I assume ends up in
> ()/job_working_directory/000/12/container.sh
> but I can't find where that's happening - container.sh isn't listed in any
> file under galaxy-dist.
> job_conf.xml lists the directories that get passed as variables - again,
> when and where are those defined?
>
> Thanks,
>
> -Jeltje
>
>
> On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren
> <[hidden email]> wrote:
>>
>> I tried the second solution (setting everything to rw) and while I can
>> confirm that the command is changed accordingly, the results are the same:
>> Two empty output files and the same error message.
>>
>> Command from paster.log:
>> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>> -w
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12
>> --net none busybox:ubuntu-14.04
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh;
>> return_code=$?; if [ -f
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> ] ; then cp
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat
>> ; fi; sh -c "exit $return_code"
>>
>> -Jeltje
>>
>> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]>
>> wrote:
>>>
>>> My first thought is the bug mentioned here -
>>>
>>> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
>>> along with potential work arounds. Are you able to confirm that this
>>> is or is not the problem at all?
>>>
>>> -John
>>>
>>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
>>> <[hidden email]> wrote:
>>> > Hi,
>>> >
>>> > I'm following instructions at
>>> > https://github.com/apetkau/galaxy-hackathon-2014.
>>> >
>>> > When I try to run the concatenate-datasets on two input files, two odd
>>> > things happen. First, TWO nearly identical commands are generated in
>>> > the
>>> > History panel, only differing in their output filename. In paster.log,
>>> > only
>>> > the second one shows up. Second, the program fails, with this info in
>>> > the
>>> > History:
>>> >
>>> > An error occurred with this dataset:
>>> > Galaxy slots passed through contain as 1
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
>>> > line 2: can't create
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>>> >
>>> > This happens for both commands in the History panel, only the output
>>> > filename in the other error is dataset_15.dat
>>> >
>>> > Oddly, the dataset_14.dat and dataset_15.dat are both created during
>>> > this
>>> > command, they just end up empty.
>>> >
>>> > Paster.log:
>>> >
>>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is:
>>> > docker
>>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1
>>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >
>>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> > -w
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
>>> > --net none busybox:ubuntu-14.04
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
>>> > return_code=$?; if [ -f
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> > ] ; then cp
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
>>> > ; fi; sh -c "exit $return_code"
>>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing
>>> > job
>>> > script:
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
>>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job
>>> > destination
>>> > (destination id: docker_local)
>>> >
>>> >
>>> > What could be happening here, and how do I fix it?
>>> >
>>> > Thanks,
>>> >
>>> > -Jeltje
>>> >
>>> >
>>> >
>>> > ___________________________________________________________
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>
>>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

jeltje
See attached for screenshot!

Thanks for the links.



On Mon, Jan 26, 2015 at 11:37 AM, John Chilton <[hidden email]> wrote:
Can you send me a screenshot of the Tool form right before you hit
submit. It is odd that two commands are executed barely differing at
all. There were some recent UI changes and I want to make sure that
you are passing two inputs once and not submitting one input twice in
the newly named "batch mode"?

container.sh is generated in command_factory.py using containers.py -
here are some relevant links.

https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default

-John


On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren
<[hidden email]> wrote:
> I'm still trying to debug this.
>
> How do I change the working directory? I can't seem to find out how all
> those directory mounts get passed to docker - the catDocker.xml appears to
> generate a 'cat' command that I assume ends up in
> ()/job_working_directory/000/12/container.sh
> but I can't find where that's happening - container.sh isn't listed in any
> file under galaxy-dist.
> job_conf.xml lists the directories that get passed as variables - again,
> when and where are those defined?
>
> Thanks,
>
> -Jeltje
>
>
> On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren
> <[hidden email]> wrote:
>>
>> I tried the second solution (setting everything to rw) and while I can
>> confirm that the command is changed accordingly, the results are the same:
>> Two empty output files and the same error message.
>>
>> Command from paster.log:
>> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>> -w
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12
>> --net none busybox:ubuntu-14.04
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh;
>> return_code=$?; if [ -f
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> ] ; then cp
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat
>> ; fi; sh -c "exit $return_code"
>>
>> -Jeltje
>>
>> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]>
>> wrote:
>>>
>>> My first thought is the bug mentioned here -
>>>
>>> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
>>> along with potential work arounds. Are you able to confirm that this
>>> is or is not the problem at all?
>>>
>>> -John
>>>
>>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
>>> <[hidden email]> wrote:
>>> > Hi,
>>> >
>>> > I'm following instructions at
>>> > https://github.com/apetkau/galaxy-hackathon-2014.
>>> >
>>> > When I try to run the concatenate-datasets on two input files, two odd
>>> > things happen. First, TWO nearly identical commands are generated in
>>> > the
>>> > History panel, only differing in their output filename. In paster.log,
>>> > only
>>> > the second one shows up. Second, the program fails, with this info in
>>> > the
>>> > History:
>>> >
>>> > An error occurred with this dataset:
>>> > Galaxy slots passed through contain as 1
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
>>> > line 2: can't create
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>>> >
>>> > This happens for both commands in the History panel, only the output
>>> > filename in the other error is dataset_15.dat
>>> >
>>> > Oddly, the dataset_14.dat and dataset_15.dat are both created during
>>> > this
>>> > command, they just end up empty.
>>> >
>>> > Paster.log:
>>> >
>>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is:
>>> > docker
>>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1
>>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >
>>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> > -w
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
>>> > --net none busybox:ubuntu-14.04
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
>>> > return_code=$?; if [ -f
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> > ] ; then cp
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
>>> > ; fi; sh -c "exit $return_code"
>>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing
>>> > job
>>> > script:
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
>>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job
>>> > destination
>>> > (destination id: docker_local)
>>> >
>>> >
>>> > What could be happening here, and how do I fix it?
>>> >
>>> > Thanks,
>>> >
>>> > -Jeltje
>>> >
>>> >
>>> >
>>> > ___________________________________________________________
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Galaxy_-_2015-01-26_11.45.51.png (73K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

jeltje
Our sysadmin just notified me docker was taking up large amounts of space in
/var/lib/docker/devicemapper/devicemapper/data
and
/var/lib/docker/devicemapper/devicemapper/metadata

I noticed the following DEBUG message in paster.log:

galaxy.jobs.runners.local DEBUG 2015-01-23 13:30:19,503 (12) executing job script: /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/galaxy_12.sh
galaxy.jobs DEBUG 2015-01-23 13:30:19,552 (12) Persisting job destination (destination id: docker_local)

Are these related? (the files I'm trying to concatenate are small text files so that's not it)


On Mon, Jan 26, 2015 at 11:47 AM, Jeltje van Baren <[hidden email]> wrote:
See attached for screenshot!

Thanks for the links.



On Mon, Jan 26, 2015 at 11:37 AM, John Chilton <[hidden email]> wrote:
Can you send me a screenshot of the Tool form right before you hit
submit. It is odd that two commands are executed barely differing at
all. There were some recent UI changes and I want to make sure that
you are passing two inputs once and not submitting one input twice in
the newly named "batch mode"?

container.sh is generated in command_factory.py using containers.py -
here are some relevant links.

https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default
https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default

-John


On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren
<[hidden email]> wrote:
> I'm still trying to debug this.
>
> How do I change the working directory? I can't seem to find out how all
> those directory mounts get passed to docker - the catDocker.xml appears to
> generate a 'cat' command that I assume ends up in
> ()/job_working_directory/000/12/container.sh
> but I can't find where that's happening - container.sh isn't listed in any
> file under galaxy-dist.
> job_conf.xml lists the directories that get passed as variables - again,
> when and where are those defined?
>
> Thanks,
>
> -Jeltje
>
>
> On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren
> <[hidden email]> wrote:
>>
>> I tried the second solution (setting everything to rw) and while I can
>> confirm that the command is changed accordingly, the results are the same:
>> Two empty output files and the same error message.
>>
>> Command from paster.log:
>> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw
>> -v
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>> -w
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12
>> --net none busybox:ubuntu-14.04
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh;
>> return_code=$?; if [ -f
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> ] ; then cp
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat
>> ; fi; sh -c "exit $return_code"
>>
>> -Jeltje
>>
>> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]>
>> wrote:
>>>
>>> My first thought is the bug mentioned here -
>>>
>>> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
>>> along with potential work arounds. Are you able to confirm that this
>>> is or is not the problem at all?
>>>
>>> -John
>>>
>>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
>>> <[hidden email]> wrote:
>>> > Hi,
>>> >
>>> > I'm following instructions at
>>> > https://github.com/apetkau/galaxy-hackathon-2014.
>>> >
>>> > When I try to run the concatenate-datasets on two input files, two odd
>>> > things happen. First, TWO nearly identical commands are generated in
>>> > the
>>> > History panel, only differing in their output filename. In paster.log,
>>> > only
>>> > the second one shows up. Second, the program fails, with this info in
>>> > the
>>> > History:
>>> >
>>> > An error occurred with this dataset:
>>> > Galaxy slots passed through contain as 1
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
>>> > line 2: can't create
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>>> >
>>> > This happens for both commands in the History panel, only the output
>>> > filename in the other error is dataset_15.dat
>>> >
>>> > Oddly, the dataset_14.dat and dataset_15.dat are both created during
>>> > this
>>> > command, they just end up empty.
>>> >
>>> > Paster.log:
>>> >
>>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is:
>>> > docker
>>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1
>>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >
>>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
>>> > -v
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> > -w
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
>>> > --net none busybox:ubuntu-14.04
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
>>> > return_code=$?; if [ -f
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> > ] ; then cp
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
>>> > ; fi; sh -c "exit $return_code"
>>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10) executing
>>> > job
>>> > script:
>>> >
>>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
>>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job
>>> > destination
>>> > (destination id: docker_local)
>>> >
>>> >
>>> > What could be happening here, and how do I fix it?
>>> >
>>> > Thanks,
>>> >
>>> > -Jeltje
>>> >
>>> >
>>> >
>>> > ___________________________________________________________
>>> > Please keep all replies on the list by using "reply all"
>>> > in your mail client.  To manage your subscriptions to this
>>> > and other Galaxy lists, please use the interface at:
>>> >   https://lists.galaxyproject.org/
>>> >
>>> > To search Galaxy mailing lists use the unified search at:
>>> >   http://galaxyproject.org/search/mailinglists/
>>
>>
>



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

John Chilton-4
Based on the screenshot you are calling the program correctly.

Can you tell me what version of Docker you are running and Galaxy you
are running? There have been a number of fixes in both programs over
the last six months to a year. Docker is known to consume a lot of
space in /var - it probably doesn't have much to do with Galaxy or
this tool right. It looks like you Docker command line does not
include -rm - which was added recently to have Docker dispose of
instances after it done with them. This might help with your /var
space issue - though in general Docker does use a lot a relatively
large amount of disk under /var - it is the nature of the technology
and many traditional HPC clusters for instance may allocate relatively
small /var partitions.

Upgrading Galaxy might also help the permission thing - I don't see a
'-u <uid>' argument in your command-line - which is another newer
default to help ensure the resulting files have the correct
permissions (especially when sudo is disabled - which it seems it is
in your configuration).

Hope this helps,

-John



On Mon, Jan 26, 2015 at 7:02 PM, Jeltje van Baren
<[hidden email]> wrote:

> Our sysadmin just notified me docker was taking up large amounts of space in
> /var/lib/docker/devicemapper/devicemapper/data
> and
> /var/lib/docker/devicemapper/devicemapper/metadata
>
> I noticed the following DEBUG message in paster.log:
>
> galaxy.jobs.runners.local DEBUG 2015-01-23 13:30:19,503 (12) executing job
> script:
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/galaxy_12.sh
> galaxy.jobs DEBUG 2015-01-23 13:30:19,552 (12) Persisting job destination
> (destination id: docker_local)
>
> Are these related? (the files I'm trying to concatenate are small text files
> so that's not it)
>
>
> On Mon, Jan 26, 2015 at 11:47 AM, Jeltje van Baren
> <[hidden email]> wrote:
>>
>> See attached for screenshot!
>>
>> Thanks for the links.
>>
>>
>>
>> On Mon, Jan 26, 2015 at 11:37 AM, John Chilton <[hidden email]>
>> wrote:
>>>
>>> Can you send me a screenshot of the Tool form right before you hit
>>> submit. It is odd that two commands are executed barely differing at
>>> all. There were some recent UI changes and I want to make sure that
>>> you are passing two inputs once and not submitting one input twice in
>>> the newly named "batch mode"?
>>>
>>> container.sh is generated in command_factory.py using containers.py -
>>> here are some relevant links.
>>>
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default
>>>
>>> -John
>>>
>>>
>>> On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren
>>> <[hidden email]> wrote:
>>> > I'm still trying to debug this.
>>> >
>>> > How do I change the working directory? I can't seem to find out how all
>>> > those directory mounts get passed to docker - the catDocker.xml appears
>>> > to
>>> > generate a 'cat' command that I assume ends up in
>>> > ()/job_working_directory/000/12/container.sh
>>> > but I can't find where that's happening - container.sh isn't listed in
>>> > any
>>> > file under galaxy-dist.
>>> > job_conf.xml lists the directories that get passed as variables -
>>> > again,
>>> > when and where are those defined?
>>> >
>>> > Thanks,
>>> >
>>> > -Jeltje
>>> >
>>> >
>>> > On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren
>>> > <[hidden email]> wrote:
>>> >>
>>> >> I tried the second solution (setting everything to rw) and while I can
>>> >> confirm that the command is changed accordingly, the results are the
>>> >> same:
>>> >> Two empty output files and the same error message.
>>> >>
>>> >> Command from paster.log:
>>> >> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> >> -w
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12
>>> >> --net none busybox:ubuntu-14.04
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh;
>>> >> return_code=$?; if [ -f
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>>> >> ] ; then cp
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat
>>> >> ; fi; sh -c "exit $return_code"
>>> >>
>>> >> -Jeltje
>>> >>
>>> >> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]>
>>> >> wrote:
>>> >>>
>>> >>> My first thought is the bug mentioned here -
>>> >>>
>>> >>>
>>> >>> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
>>> >>> along with potential work arounds. Are you able to confirm that this
>>> >>> is or is not the problem at all?
>>> >>>
>>> >>> -John
>>> >>>
>>> >>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
>>> >>> <[hidden email]> wrote:
>>> >>> > Hi,
>>> >>> >
>>> >>> > I'm following instructions at
>>> >>> > https://github.com/apetkau/galaxy-hackathon-2014.
>>> >>> >
>>> >>> > When I try to run the concatenate-datasets on two input files, two
>>> >>> > odd
>>> >>> > things happen. First, TWO nearly identical commands are generated
>>> >>> > in
>>> >>> > the
>>> >>> > History panel, only differing in their output filename. In
>>> >>> > paster.log,
>>> >>> > only
>>> >>> > the second one shows up. Second, the program fails, with this info
>>> >>> > in
>>> >>> > the
>>> >>> > History:
>>> >>> >
>>> >>> > An error occurred with this dataset:
>>> >>> > Galaxy slots passed through contain as 1
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
>>> >>> > line 2: can't create
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>>> >>> >
>>> >>> > This happens for both commands in the History panel, only the
>>> >>> > output
>>> >>> > filename in the other error is dataset_15.dat
>>> >>> >
>>> >>> > Oddly, the dataset_14.dat and dataset_15.dat are both created
>>> >>> > during
>>> >>> > this
>>> >>> > command, they just end up empty.
>>> >>> >
>>> >>> > Paster.log:
>>> >>> >
>>> >>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is:
>>> >>> > docker
>>> >>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >>> >
>>> >>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> >>> > -w
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
>>> >>> > --net none busybox:ubuntu-14.04
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
>>> >>> > return_code=$?; if [ -f
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >>> > ] ; then cp
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
>>> >>> > ; fi; sh -c "exit $return_code"
>>> >>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10)
>>> >>> > executing
>>> >>> > job
>>> >>> > script:
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
>>> >>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job
>>> >>> > destination
>>> >>> > (destination id: docker_local)
>>> >>> >
>>> >>> >
>>> >>> > What could be happening here, and how do I fix it?
>>> >>> >
>>> >>> > Thanks,
>>> >>> >
>>> >>> > -Jeltje
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > ___________________________________________________________
>>> >>> > Please keep all replies on the list by using "reply all"
>>> >>> > in your mail client.  To manage your subscriptions to this
>>> >>> > and other Galaxy lists, please use the interface at:
>>> >>> >   https://lists.galaxyproject.org/
>>> >>> >
>>> >>> > To search Galaxy mailing lists use the unified search at:
>>> >>> >   http://galaxyproject.org/search/mailinglists/
>>> >>
>>> >>
>>> >
>>
>>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: concatenate_datasets docker image can't create output

jeltje
The docker version is 1.1.2.
I'm not entirely sure how to get the galaxy version, but hg sum resulted in
parent: 14955:c25df9f3d31e tip
Update tag latest_2014.10.06 for changeset ff6e36d7a238
branch: stable

So docker may be a bit too old. To get some more control over my OS, I did a fresh Ubuntu install on a separate laptop and got docker (1.41) and galaxy (downloaded stable release on Friday). Now I can get the docker image to run, but I STILL get two jobs in the history panel. One results in the concatenated files, the other has the text 'Work dir output'  (and nothing else). Screenshot attached.

I'll try to talk the sysadmins into upgrading docker and see if that fixes things.

Thanks!

-Jeltje

On Fri, Jan 30, 2015 at 10:22 AM, John Chilton <[hidden email]> wrote:
Based on the screenshot you are calling the program correctly.

Can you tell me what version of Docker you are running and Galaxy you
are running? There have been a number of fixes in both programs over
the last six months to a year. Docker is known to consume a lot of
space in /var - it probably doesn't have much to do with Galaxy or
this tool right. It looks like you Docker command line does not
include -rm - which was added recently to have Docker dispose of
instances after it done with them. This might help with your /var
space issue - though in general Docker does use a lot a relatively
large amount of disk under /var - it is the nature of the technology
and many traditional HPC clusters for instance may allocate relatively
small /var partitions.

Upgrading Galaxy might also help the permission thing - I don't see a
'-u <uid>' argument in your command-line - which is another newer
default to help ensure the resulting files have the correct
permissions (especially when sudo is disabled - which it seems it is
in your configuration).

Hope this helps,

-John



On Mon, Jan 26, 2015 at 7:02 PM, Jeltje van Baren
<[hidden email]> wrote:
> Our sysadmin just notified me docker was taking up large amounts of space in
> /var/lib/docker/devicemapper/devicemapper/data
> and
> /var/lib/docker/devicemapper/devicemapper/metadata
>
> I noticed the following DEBUG message in paster.log:
>
> galaxy.jobs.runners.local DEBUG 2015-01-23 13:30:19,503 (12) executing job
> script:
> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/galaxy_12.sh
> galaxy.jobs DEBUG 2015-01-23 13:30:19,552 (12) Persisting job destination
> (destination id: docker_local)
>
> Are these related? (the files I'm trying to concatenate are small text files
> so that's not it)
>
>
> On Mon, Jan 26, 2015 at 11:47 AM, Jeltje van Baren
> <[hidden email]> wrote:
>>
>> See attached for screenshot!
>>
>> Thanks for the links.
>>
>>
>>
>> On Mon, Jan 26, 2015 at 11:37 AM, John Chilton <[hidden email]>
>> wrote:
>>>
>>> Can you send me a screenshot of the Tool form right before you hit
>>> submit. It is odd that two commands are executed barely differing at
>>> all. There were some recent UI changes and I want to make sure that
>>> you are passing two inputs once and not submitting one input twice in
>>> the newly named "batch mode"?
>>>
>>> container.sh is generated in command_factory.py using containers.py -
>>> here are some relevant links.
>>>
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/jobs/command_factory.py?at=default
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/containers.py?at=default
>>>
>>> https://bitbucket.org/galaxy/galaxy-central/src/tip/lib/galaxy/tools/deps/docker_util.py?at=default
>>>
>>> -John
>>>
>>>
>>> On Mon, Jan 26, 2015 at 2:17 PM, Jeltje van Baren
>>> <[hidden email]> wrote:
>>> > I'm still trying to debug this.
>>> >
>>> > How do I change the working directory? I can't seem to find out how all
>>> > those directory mounts get passed to docker - the catDocker.xml appears
>>> > to
>>> > generate a 'cat' command that I assume ends up in
>>> > ()/job_working_directory/000/12/container.sh
>>> > but I can't find where that's happening - container.sh isn't listed in
>>> > any
>>> > file under galaxy-dist.
>>> > job_conf.xml lists the directories that get passed as variables -
>>> > again,
>>> > when and where are those defined?
>>> >
>>> > Thanks,
>>> >
>>> > -Jeltje
>>> >
>>> >
>>> > On Fri, Jan 23, 2015 at 1:35 PM, Jeltje van Baren
>>> > <[hidden email]> wrote:
>>> >>
>>> >> I tried the second solution (setting everything to rw) and while I can
>>> >> confirm that the command is changed accordingly, the results are the
>>> >> same:
>>> >> Two empty output files and the same error message.
>>> >>
>>> >> Command from paster.log:
>>> >> docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12:rw
>>> >> -v
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> >> -w
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12
>>> >> --net none busybox:ubuntu-14.04
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/container.sh;
>>> >> return_code=$?; if [ -f
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>>> >> ] ; then cp
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/12/working_file
>>> >>
>>> >> /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_19.dat
>>> >> ; fi; sh -c "exit $return_code"
>>> >>
>>> >> -Jeltje
>>> >>
>>> >> On Fri, Jan 23, 2015 at 12:24 PM, John Chilton <[hidden email]>
>>> >> wrote:
>>> >>>
>>> >>> My first thought is the bug mentioned here -
>>> >>>
>>> >>>
>>> >>> https://lists.galaxyproject.org/pipermail/galaxy-dev/2014-November/020892.html
>>> >>> along with potential work arounds. Are you able to confirm that this
>>> >>> is or is not the problem at all?
>>> >>>
>>> >>> -John
>>> >>>
>>> >>> On Fri, Jan 23, 2015 at 2:37 PM, Jeltje van Baren
>>> >>> <[hidden email]> wrote:
>>> >>> > Hi,
>>> >>> >
>>> >>> > I'm following instructions at
>>> >>> > https://github.com/apetkau/galaxy-hackathon-2014.
>>> >>> >
>>> >>> > When I try to run the concatenate-datasets on two input files, two
>>> >>> > odd
>>> >>> > things happen. First, TWO nearly identical commands are generated
>>> >>> > in
>>> >>> > the
>>> >>> > History panel, only differing in their output filename. In
>>> >>> > paster.log,
>>> >>> > only
>>> >>> > the second one shows up. Second, the program fails, with this info
>>> >>> > in
>>> >>> > the
>>> >>> > History:
>>> >>> >
>>> >>> > An error occurred with this dataset:
>>> >>> > Galaxy slots passed through contain as 1
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh:
>>> >>> > line 2: can't create
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_14.dat
>>> >>> >
>>> >>> > This happens for both commands in the History panel, only the
>>> >>> > output
>>> >>> > filename in the other error is dataset_15.dat
>>> >>> >
>>> >>> > Oddly, the dataset_14.dat and dataset_15.dat are both created
>>> >>> > during
>>> >>> > this
>>> >>> > command, they just end up empty.
>>> >>> >
>>> >>> > Paster.log:
>>> >>> >
>>> >>> > galaxy.jobs.runners DEBUG 2015-01-23 11:08:35,973 (10) command is:
>>> >>> > docker
>>> >>> > inspect busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >>> > [ $? -ne 0 ] && docker pull busybox:ubuntu-14.04 > /dev/null 2>&1
>>> >>> >
>>> >>> > docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist:ro
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/tools/docker:ro
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10:rw
>>> >>> > -v
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:/inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files:rw
>>> >>> > -w
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10
>>> >>> > --net none busybox:ubuntu-14.04
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/container.sh;
>>> >>> > return_code=$?; if [ -f
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >>> > ] ; then cp
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/working_file
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/files/000/dataset_15.dat
>>> >>> > ; fi; sh -c "exit $return_code"
>>> >>> > galaxy.jobs.runners.local DEBUG 2015-01-23 11:08:36,040 (10)
>>> >>> > executing
>>> >>> > job
>>> >>> > script:
>>> >>> >
>>> >>> >
>>> >>> > /inside/home/jeltje/exp/varscan2/programs/galaxy-dist/database/job_working_directory/000/10/galaxy_10.sh
>>> >>> > galaxy.jobs DEBUG 2015-01-23 11:08:36,091 (10) Persisting job
>>> >>> > destination
>>> >>> > (destination id: docker_local)
>>> >>> >
>>> >>> >
>>> >>> > What could be happening here, and how do I fix it?
>>> >>> >
>>> >>> > Thanks,
>>> >>> >
>>> >>> > -Jeltje
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > ___________________________________________________________
>>> >>> > Please keep all replies on the list by using "reply all"
>>> >>> > in your mail client.  To manage your subscriptions to this
>>> >>> > and other Galaxy lists, please use the interface at:
>>> >>> >   https://lists.galaxyproject.org/
>>> >>> >
>>> >>> > To search Galaxy mailing lists use the unified search at:
>>> >>> >   http://galaxyproject.org/search/mailinglists/
>>> >>
>>> >>
>>> >
>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Galaxy_-_2015-02-02_02.07.47.png (112K) Download Attachment