upload error

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

upload error

Rui Wang
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Aysam Guerler
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Martin Čech
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Martin Čech
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/





___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

hgstatus (36K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/






___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Martin Čech
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/







___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
I followed this instruction and the server started successfully. The whole thing starts with:

python path is: /home/bioinfoadmin/app/galaxy-dist/eggs/mercurial-2.2.3-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/amqplib-0.6.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pexpect-2.4-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Babel-0.9.4-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/WebHelpers-1.3-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/MarkupSafe-0.12-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/WebOb-0.8.5-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Routes-1.12.3-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/wchartype-0.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/docutils-0.7-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/PasteDeploy-1.5.0-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/lib, /usr/local/lib/python2.7/dist-packages/distribute-0.6.49-py2.7.egg, /usr/local/lib/python2.7/dist-packages/BETA_Package-0.1.5-py2.7.egg, /home/bioinfoadmin/app/dep/macs/1.4.2/lib/python2.7/site-packages, /home/bioinfoadmin/app/galaxy-dist, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib-dynload, /usr/local/lib/python2.7/dist-packages, /usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info, /usr/lib/python2.7/dist-packages/PIL, /usr/lib/python2.7/dist-packages/gst-0.10, /usr/lib/python2.7/dist-packages/gtk-2.0, /usr/lib/python2.7/dist-packages/ubuntu-sso-client, /usr/lib/python2.7/dist-packages/ubuntuone-client, /usr/lib/python2.7/dist-packages/ubuntuone-control-panel, /usr/lib/python2.7/dist-packages/ubuntuone-couch, /usr/lib/python2.7/dist-packages/ubuntuone-installer, /usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol, /usr/lib/python2.7/dist-packages
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loading references to tool sheds from tool_sheds_conf.xml
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loaded reference to tool shed: Galaxy main tool shed
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loaded reference to tool shed: Galaxy test tool shed
migrate.versioning.repository DEBUG 2014-02-28 10:32:53,427 Loading repository lib/galaxy/model/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 10:32:53,428 Loading script lib/galaxy/model/migrate/versions/0001_initial_tables.py...
migrate.versioning.script.base DEBUG 2014-02-28 10:32:53,428 Script lib/galaxy/model/migrate/versions/0001_initial_tables.py loaded successfully

Seems it did start the migration.

The server started with a few errors like:

galaxy.datatypes.registry DEBUG 2014-02-28 10:32:54,630 Loading datatypes from datatypes_conf.xml
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,630 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Eset'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'Eset'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'CelZip'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'CelZip'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'XysZip'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'XysZip'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Cel'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'Cel'

I think this might be related to a customized tool. however, the history panel is still showing 'loading...' and tool panel is empty. same as the picture I sent last night.

Thanks,
Rui


On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/








___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Martin,

So, after some effort, it started all ok now, and tool panel loaded. It was probably due to cistrome.

However, when we upload file(using 'load data' or 'upload file from computer'), we have the following warning. After I make the dir to be 755, it worked. Seems I'm ready to go. Is this a right approach to solve it?

Thanks,
Rui

/usr/local/lib/python2.7/dist-packages/distribute-0.6.49-py2.7.egg/pkg_resources.py:1025: UserWarning: /home/bioinfoadmin/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
  warnings.warn(msg, UserWarning)


On Fri, Feb 28, 2014 at 10:41 AM, ruiwang.sz <[hidden email]> wrote:
I followed this instruction and the server started successfully. The whole thing starts with:

python path is: /home/bioinfoadmin/app/galaxy-dist/eggs/mercurial-2.2.3-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/amqplib-0.6.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/pexpect-2.4-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Babel-0.9.4-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Mako-0.4.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/WebHelpers-1.3-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/MarkupSafe-0.12-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/WebOb-0.8.5-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Routes-1.12.3-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/wchartype-0.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/docutils-0.7-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/PasteDeploy-1.5.0-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg, /home/bioinfoadmin/app/galaxy-dist/lib, /usr/local/lib/python2.7/dist-packages/distribute-0.6.49-py2.7.egg, /usr/local/lib/python2.7/dist-packages/BETA_Package-0.1.5-py2.7.egg, /home/bioinfoadmin/app/dep/macs/1.4.2/lib/python2.7/site-packages, /home/bioinfoadmin/app/galaxy-dist, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib-dynload, /usr/local/lib/python2.7/dist-packages, /usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info, /usr/lib/python2.7/dist-packages/PIL, /usr/lib/python2.7/dist-packages/gst-0.10, /usr/lib/python2.7/dist-packages/gtk-2.0, /usr/lib/python2.7/dist-packages/ubuntu-sso-client, /usr/lib/python2.7/dist-packages/ubuntuone-client, /usr/lib/python2.7/dist-packages/ubuntuone-control-panel, /usr/lib/python2.7/dist-packages/ubuntuone-couch, /usr/lib/python2.7/dist-packages/ubuntuone-installer, /usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol, /usr/lib/python2.7/dist-packages
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loading references to tool sheds from tool_sheds_conf.xml
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loaded reference to tool shed: Galaxy main tool shed
tool_shed.tool_shed_registry DEBUG 2014-02-28 10:32:53,360 Loaded reference to tool shed: Galaxy test tool shed
migrate.versioning.repository DEBUG 2014-02-28 10:32:53,427 Loading repository lib/galaxy/model/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 10:32:53,428 Loading script lib/galaxy/model/migrate/versions/0001_initial_tables.py...
migrate.versioning.script.base DEBUG 2014-02-28 10:32:53,428 Script lib/galaxy/model/migrate/versions/0001_initial_tables.py loaded successfully

Seems it did start the migration.

The server started with a few errors like:

galaxy.datatypes.registry DEBUG 2014-02-28 10:32:54,630 Loading datatypes from datatypes_conf.xml
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,630 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Eset'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'Eset'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'CelZip'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'CelZip'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'XysZip'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'XysZip'
galaxy.datatypes.registry ERROR 2014-02-28 10:32:54,631 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Cel'
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes
    datatype_class = getattr( module, datatype_class_name )
AttributeError: 'module' object has no attribute 'Cel'

I think this might be related to a customized tool. however, the history panel is still showing 'loading...' and tool panel is empty. same as the picture I sent last night.

Thanks,
Rui


On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/









___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
In reply to this post by Martin Čech
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/








___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Martin Čech
Rui,

it might be that your shed_tool_conf.xml is inconsistent and doesn't actually reflect tools installed in Galaxy. Please log in as an admin and check 'Manage installed tool shed repositories' menu. Try to find a tool that is installed and in tool_conf.xml but is not showing up in the tool panel. Uninstall and install this tool again. That should create entry in shed_tool_conf.xml and the tool should display properly.

M.


On Fri, Feb 28, 2014 at 3:26 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/









___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Martin,

So, after some effort, things seem to be working. Thanks for the help along the way.

However, one more error happened. 

Originally, I installed macs dependency for galaxy myself. The version is 1.4.2. It worked well. 

Now, after galaxy upgrading removed it, and I installed it back from tool_shed, things broke. In the galaxy tool panel, it looks the same like before(as expected), but when I click on macs, it will show an option of MACS 'version 1.0.0' and 'version 1.0.1'. Be default it is 1.0.1, which is defined in migrated_tools_conf.xml:

<section id="peak_calling" name="NGS: Peak Calling" version="">
      <tool_shed>toolshed.g2.bx.psu.edu</tool_shed>
        <repository_name>macs</repository_name>
        <repository_owner>devteam</repository_owner>
        <installed_changeset_revision>ae2ec275332a</installed_changeset_revision>
        <version>1.0.1</version>
    </tool>
</section>

if we dive in this file toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs/macs_wrapper.xml, we could see that in this dir /home/bioinfoadmin/app/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs, there is a tool_dependency xml file that has following content:

<?xml version="1.0"?>
<tool_dependency>
  <package name="macs" version="1.3.7.1">
      <repository changeset_revision="a7ea583a35d2" name="package_macs_1_3_7_1" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
    <package name="R" version="2.15.0">
      <repository changeset_revision="8ab0d08a3da1" name="package_r_2_15_0" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
</tool_dependency>

These versions are both older than what we installed before(1.4.2 and 3.0.2). could we manually modify that to use our own? wouldn't that be a bad practice?

while in the same dir, macs_wrapper.xml has:

  <requirements>
    <requirement type="package" version="1.3.7.1">macs</requirement>
    <requirement type="package" version="2.15.0">R</requirement>
  </requirements>

Seems that this is the tool_shed version of macs(which is identical to the one on main galaxy server, I think?). However this one does not work with our input data. Additionally, when I click to switch from 1.0.1 to 1.0.0 of macs, it immediately reports error. 

So, what if I want to keep using the one I installed myself? I know that tool_shed is a cleaner way to manage tools, but in our situation the upgrade and elimination actually broke the thing that worked before. I want to ask the proper way to handle this kind of situation. could I simply drop the entry from migrated_tools_conf.xml about macs, and restore the old tools_conf.xml entry? in that way we could continue using the 1.4.2 we have. However next time when we do the upgrade, things will break again I guess? 

I think that I shouldn't be the only person that has this concern? how to handle multiple version/dependency version for a tool? is there an option to not use the tool_shed but keep own version?

Thanks,
Rui




On Fri, Feb 28, 2014 at 12:44 PM, Martin Čech <[hidden email]> wrote:
Rui,

it might be that your shed_tool_conf.xml is inconsistent and doesn't actually reflect tools installed in Galaxy. Please log in as an admin and check 'Manage installed tool shed repositories' menu. Try to find a tool that is installed and in tool_conf.xml but is not showing up in the tool panel. Uninstall and install this tool again. That should create entry in shed_tool_conf.xml and the tool should display properly.

M.


On Fri, Feb 28, 2014 at 3:26 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/










___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Nate Coraor (nate@bx.psu.edu)
Hi Rui,

I'm not sure what version of the MACS wrapper for Galaxy you were using prior to upgrading, but the version that previously came with the Galaxy distribution, and that is now in the Tool Shed under the devteam account, is only compatible with MACS 1.3.7.1. There is a 1.4 version available in the Tool Shed, but I don't believe anyone on our team has used it.

--nate


On Tue, Mar 4, 2014 at 8:01 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

So, after some effort, things seem to be working. Thanks for the help along the way.

However, one more error happened. 

Originally, I installed macs dependency for galaxy myself. The version is 1.4.2. It worked well. 

Now, after galaxy upgrading removed it, and I installed it back from tool_shed, things broke. In the galaxy tool panel, it looks the same like before(as expected), but when I click on macs, it will show an option of MACS 'version 1.0.0' and 'version 1.0.1'. Be default it is 1.0.1, which is defined in migrated_tools_conf.xml:

<section id="peak_calling" name="NGS: Peak Calling" version="">
      <tool_shed>toolshed.g2.bx.psu.edu</tool_shed>
        <repository_name>macs</repository_name>
        <repository_owner>devteam</repository_owner>
        <installed_changeset_revision>ae2ec275332a</installed_changeset_revision>
        <version>1.0.1</version>
    </tool>
</section>

if we dive in this file toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs/macs_wrapper.xml, we could see that in this dir /home/bioinfoadmin/app/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs, there is a tool_dependency xml file that has following content:

<?xml version="1.0"?>
<tool_dependency>
  <package name="macs" version="1.3.7.1">
      <repository changeset_revision="a7ea583a35d2" name="package_macs_1_3_7_1" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
    <package name="R" version="2.15.0">
      <repository changeset_revision="8ab0d08a3da1" name="package_r_2_15_0" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
</tool_dependency>

These versions are both older than what we installed before(1.4.2 and 3.0.2). could we manually modify that to use our own? wouldn't that be a bad practice?

while in the same dir, macs_wrapper.xml has:

  <requirements>
    <requirement type="package" version="1.3.7.1">macs</requirement>
    <requirement type="package" version="2.15.0">R</requirement>
  </requirements>

Seems that this is the tool_shed version of macs(which is identical to the one on main galaxy server, I think?). However this one does not work with our input data. Additionally, when I click to switch from 1.0.1 to 1.0.0 of macs, it immediately reports error. 

So, what if I want to keep using the one I installed myself? I know that tool_shed is a cleaner way to manage tools, but in our situation the upgrade and elimination actually broke the thing that worked before. I want to ask the proper way to handle this kind of situation. could I simply drop the entry from migrated_tools_conf.xml about macs, and restore the old tools_conf.xml entry? in that way we could continue using the 1.4.2 we have. However next time when we do the upgrade, things will break again I guess? 

I think that I shouldn't be the only person that has this concern? how to handle multiple version/dependency version for a tool? is there an option to not use the tool_shed but keep own version?

Thanks,
Rui




On Fri, Feb 28, 2014 at 12:44 PM, Martin Čech <[hidden email]> wrote:
Rui,

it might be that your shed_tool_conf.xml is inconsistent and doesn't actually reflect tools installed in Galaxy. Please log in as an admin and check 'Manage installed tool shed repositories' menu. Try to find a tool that is installed and in tool_conf.xml but is not showing up in the tool panel. Uninstall and install this tool again. That should create entry in shed_tool_conf.xml and the tool should display properly.

M.


On Fri, Feb 28, 2014 at 3:26 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/










___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

Rui Wang
Hi Nate,

Thanks for the note!!

The wrapper we were using prior to upgrading was 1.0.0, and now the wrapper in toolshed is 1.0.1. I could not switch between them -- an error occurred. 

However, the MACS binary package I installed had the name of executable file 'macs14', I thought that it is 1.4? Here people use it all the time.

So in short, now I still have the 1.4 installed, but shed_tool has the 1.3.7.1. How could I let Galaxy still use 1.4? Do I need to remove the shed_tool version by uninstalling that repo? I guess this might be a general question: how do I manage multiple version of tools from different sources(shed_tool and own installation)?

Best Regards,
Rui


On Fri, Mar 7, 2014 at 6:10 AM, Nate Coraor <[hidden email]> wrote:
Hi Rui,

I'm not sure what version of the MACS wrapper for Galaxy you were using prior to upgrading, but the version that previously came with the Galaxy distribution, and that is now in the Tool Shed under the devteam account, is only compatible with MACS 1.3.7.1. There is a 1.4 version available in the Tool Shed, but I don't believe anyone on our team has used it.

--nate


On Tue, Mar 4, 2014 at 8:01 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

So, after some effort, things seem to be working. Thanks for the help along the way.

However, one more error happened. 

Originally, I installed macs dependency for galaxy myself. The version is 1.4.2. It worked well. 

Now, after galaxy upgrading removed it, and I installed it back from tool_shed, things broke. In the galaxy tool panel, it looks the same like before(as expected), but when I click on macs, it will show an option of MACS 'version 1.0.0' and 'version 1.0.1'. Be default it is 1.0.1, which is defined in migrated_tools_conf.xml:

<section id="peak_calling" name="NGS: Peak Calling" version="">
      <tool_shed>toolshed.g2.bx.psu.edu</tool_shed>
        <repository_name>macs</repository_name>
        <repository_owner>devteam</repository_owner>
        <installed_changeset_revision>ae2ec275332a</installed_changeset_revision>
        <version>1.0.1</version>
    </tool>
</section>

if we dive in this file toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs/macs_wrapper.xml, we could see that in this dir /home/bioinfoadmin/app/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs, there is a tool_dependency xml file that has following content:

<?xml version="1.0"?>
<tool_dependency>
  <package name="macs" version="1.3.7.1">
      <repository changeset_revision="a7ea583a35d2" name="package_macs_1_3_7_1" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
    <package name="R" version="2.15.0">
      <repository changeset_revision="8ab0d08a3da1" name="package_r_2_15_0" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
</tool_dependency>

These versions are both older than what we installed before(1.4.2 and 3.0.2). could we manually modify that to use our own? wouldn't that be a bad practice?

while in the same dir, macs_wrapper.xml has:

  <requirements>
    <requirement type="package" version="1.3.7.1">macs</requirement>
    <requirement type="package" version="2.15.0">R</requirement>
  </requirements>

Seems that this is the tool_shed version of macs(which is identical to the one on main galaxy server, I think?). However this one does not work with our input data. Additionally, when I click to switch from 1.0.1 to 1.0.0 of macs, it immediately reports error. 

So, what if I want to keep using the one I installed myself? I know that tool_shed is a cleaner way to manage tools, but in our situation the upgrade and elimination actually broke the thing that worked before. I want to ask the proper way to handle this kind of situation. could I simply drop the entry from migrated_tools_conf.xml about macs, and restore the old tools_conf.xml entry? in that way we could continue using the 1.4.2 we have. However next time when we do the upgrade, things will break again I guess? 

I think that I shouldn't be the only person that has this concern? how to handle multiple version/dependency version for a tool? is there an option to not use the tool_shed but keep own version?

Thanks,
Rui




On Fri, Feb 28, 2014 at 12:44 PM, Martin Čech <[hidden email]> wrote:
Rui,

it might be that your shed_tool_conf.xml is inconsistent and doesn't actually reflect tools installed in Galaxy. Please log in as an admin and check 'Manage installed tool shed repositories' menu. Try to find a tool that is installed and in tool_conf.xml but is not showing up in the tool panel. Uninstall and install this tool again. That should create entry in shed_tool_conf.xml and the tool should display properly.

M.


On Fri, Feb 28, 2014 at 3:26 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------




___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/










___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload error

John Chilton-4
Hmm... I am not certain but it sounds like the migration script ripped something out of your Galaxy tool panel that you did not want it to? This is unfortunate - but I don't think it would happen again now that the migration process is complete. Some future update to Galaxy may include additional migrations scripts but now that macs is out it won't ever be migrated again.  imagine you can just update your tool_conf.xml file to point to your existing macs wrapper you set up manually and you can either leave the devteam version of macs in place or uninstall it if you prefer your own. Is it not this easy?

If there is some subtler problem with an incorrect version of MACS being used by a tool - that is definitely fixable, Galaxy can resolve requirements in a very configurable manner so let me know - but that is not sounding like the problem you are having.

-John


On Fri, Mar 7, 2014 at 10:43 AM, ruiwang.sz <[hidden email]> wrote:
Hi Nate,

Thanks for the note!!

The wrapper we were using prior to upgrading was 1.0.0, and now the wrapper in toolshed is 1.0.1. I could not switch between them -- an error occurred. 

However, the MACS binary package I installed had the name of executable file 'macs14', I thought that it is 1.4? Here people use it all the time.

So in short, now I still have the 1.4 installed, but shed_tool has the 1.3.7.1. How could I let Galaxy still use 1.4? Do I need to remove the shed_tool version by uninstalling that repo? I guess this might be a general question: how do I manage multiple version of tools from different sources(shed_tool and own installation)?

Best Regards,
Rui


On Fri, Mar 7, 2014 at 6:10 AM, Nate Coraor <[hidden email]> wrote:
Hi Rui,

I'm not sure what version of the MACS wrapper for Galaxy you were using prior to upgrading, but the version that previously came with the Galaxy distribution, and that is now in the Tool Shed under the devteam account, is only compatible with MACS 1.3.7.1. There is a 1.4 version available in the Tool Shed, but I don't believe anyone on our team has used it.

--nate


On Tue, Mar 4, 2014 at 8:01 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

So, after some effort, things seem to be working. Thanks for the help along the way.

However, one more error happened. 

Originally, I installed macs dependency for galaxy myself. The version is 1.4.2. It worked well. 

Now, after galaxy upgrading removed it, and I installed it back from tool_shed, things broke. In the galaxy tool panel, it looks the same like before(as expected), but when I click on macs, it will show an option of MACS 'version 1.0.0' and 'version 1.0.1'. Be default it is 1.0.1, which is defined in migrated_tools_conf.xml:

<section id="peak_calling" name="NGS: Peak Calling" version="">
      <tool_shed>toolshed.g2.bx.psu.edu</tool_shed>
        <repository_name>macs</repository_name>
        <repository_owner>devteam</repository_owner>
        <installed_changeset_revision>ae2ec275332a</installed_changeset_revision>
        <version>1.0.1</version>
    </tool>
</section>

if we dive in this file toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs/macs_wrapper.xml, we could see that in this dir /home/bioinfoadmin/app/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs, there is a tool_dependency xml file that has following content:

<?xml version="1.0"?>
<tool_dependency>
  <package name="macs" version="1.3.7.1">
      <repository changeset_revision="a7ea583a35d2" name="package_macs_1_3_7_1" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
    <package name="R" version="2.15.0">
      <repository changeset_revision="8ab0d08a3da1" name="package_r_2_15_0" owner="devteam" prior_installation_required="False" toolshed="http://toolshed.g2.bx.psu.edu" />
    </package>
</tool_dependency>

These versions are both older than what we installed before(1.4.2 and 3.0.2). could we manually modify that to use our own? wouldn't that be a bad practice?

while in the same dir, macs_wrapper.xml has:

  <requirements>
    <requirement type="package" version="1.3.7.1">macs</requirement>
    <requirement type="package" version="2.15.0">R</requirement>
  </requirements>

Seems that this is the tool_shed version of macs(which is identical to the one on main galaxy server, I think?). However this one does not work with our input data. Additionally, when I click to switch from 1.0.1 to 1.0.0 of macs, it immediately reports error. 

So, what if I want to keep using the one I installed myself? I know that tool_shed is a cleaner way to manage tools, but in our situation the upgrade and elimination actually broke the thing that worked before. I want to ask the proper way to handle this kind of situation. could I simply drop the entry from migrated_tools_conf.xml about macs, and restore the old tools_conf.xml entry? in that way we could continue using the 1.4.2 we have. However next time when we do the upgrade, things will break again I guess? 

I think that I shouldn't be the only person that has this concern? how to handle multiple version/dependency version for a tool? is there an option to not use the tool_shed but keep own version?

Thanks,
Rui




On Fri, Feb 28, 2014 at 12:44 PM, Martin Čech <[hidden email]> wrote:
Rui,

it might be that your shed_tool_conf.xml is inconsistent and doesn't actually reflect tools installed in Galaxy. Please log in as an admin and check 'Manage installed tool shed repositories' menu. Try to find a tool that is installed and in tool_conf.xml but is not showing up in the tool panel. Uninstall and install this tool again. That should create entry in shed_tool_conf.xml and the tool should display properly.

M.


On Fri, Feb 28, 2014 at 3:26 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

I'm seeing a lot of messages like:

galaxy.model.migrate.check INFO 2014-02-28 12:10:18,201 At database version 118
migrate.versioning.repository DEBUG 2014-02-28 12:10:18,220 Loading repository lib/tool_shed/galaxy_install/migrate...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0001_tools.py loaded successfully
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Loading script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py...
migrate.versioning.script.base DEBUG 2014-02-28 12:10:18,220 Script lib/tool_shed/galaxy_install/migrate/versions/0002_tools.py loaded successfully

and then

tool_shed.galaxy_install.installed_repository_manager DEBUG 2014-02-28 12:10:18,683 Adding an entry for revision 96d2e31a3938 of repository bowtie2 owned by devteam to repository_dependencies_of_installed_repositories.

I noticed that some tools disappeared from the side panel, like bowtie and samtools, etc. So, should I remove the corresponding entries in the tool_conf.xml? Now seems many of the tools were installed back by the toolshed but I don't see them on the panel, how to enable them?  I thought this would be done by the migration script?

Thanks,
Rui



On Fri, Feb 28, 2014 at 10:24 AM, Martin Čech <[hidden email]> wrote:
Question mark means 'untracked' so no problems there.

Please try removing tool_conf.xml (make backup) and run Galaxy again. It might trigger the migration again.

M.


On Fri, Feb 28, 2014 at 1:08 PM, ruiwang.sz <[hidden email]> wrote:
oh, I took that back, I think these are the tools that I installed but does not come with Galaxy. probably that's the ? means -- it is not part of the distribution?

thanks,
Rui


On Fri, Feb 28, 2014 at 10:07 AM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Here you go. there are 493 files missing, seems.

Thanks,
Rui


On Fri, Feb 28, 2014 at 9:59 AM, Martin Čech <[hidden email]> wrote:
Rui,

could you please try 'hg status' and tell us what it outputs? Thanks

Martin


On Fri, Feb 28, 2014 at 12:27 PM, ruiwang.sz <[hidden email]> wrote:
Hi Martin,

Thanks. I'll read the manual.

Best,
Rui


On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <[hidden email]> wrote:
Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin


On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <[hidden email]> wrote:
Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u
2. ./run.sh --daemon    (many error msg, follow the suggestion,then )
3. sh manage_db.sh upgrade      (update db to 118 version, start again)
4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)
5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

Inline image 1

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for
this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be
automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,
be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should
be removed from your file named ./tool_conf.xml

...migrate 7->8
...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu
and will be installed into your local Galaxy instance at the
location discussed above by running the following command.


vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'
option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sh ./scripts/migrate_tools/0009_tools.sh install_dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,
entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these
tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been
removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml
execute_dwt_var_perFeature.xml
GMAJ.xml
rgWebLogo3.xml
rgClustalw.xml
gi2taxonomy.xml
t2t_report.xml
t2ps_wrapper.xml
find_diag_hits.xml
lca.xml
poisson2test.xml
rgFastQC.xml
fastq_groomer.xml
fastq_paired_end_splitter.xml
fastq_paired_end_joiner.xml
fastq_stats.xml
fastq_combiner.xml
fastq_filter.xml
fastq_trimmer.xml
fastq_trimmer_by_quality.xml
fastq_masker_by_quality.xml
fastq_paired_end_interlacer.xml
fastq_paired_end_deinterlacer.xml
fastq_manipulation.xml
fastq_to_tabular.xml
tabular_to_fastq.xml
bowtie2_wrapper.xml
tophat_wrapper.xml
tophat2_wrapper.xml
express_wrapper.xml
filter_transcripts_via_tracking.xml
macs_wrapper.xml
sicer_wrapper.xml
ccat_wrapper.xml
freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml
Traceback (most recent call last):
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set
    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool
    tree = load_tool( config_file )
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool
    tree = parse_xml(path)
  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml
    tree = ElementTree.parse(fname)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse
    tree.parse(source, parser)
  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'
IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'
IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'
IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'
IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'
IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'
IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'
IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'
IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'
IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'
IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'
IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'
IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'
IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'
IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,
Rui








On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <[hidden email]> wrote:
Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam


On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <[hidden email]> wrote:
Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!
Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent ca