Running out of memory on data download

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Running out of memory on data download

Michael Römer
Dear all,

We run our Galaxy server in a virtual machine with less than 1GB RAM.
When I'm trying to download results via the Download button in the
corresponding history entry that are larger than 1GB, the results file
is loaded into memory. This leads to the galaxy process being killed,
due to memory issues.

Is there a way to omit loading the result into memory?

Regards,
Michael

--
Michael Römer, MSc. Bioinf.
Abteilung Kognitive Systeme
Fachbereich Informatik
Wilhelm-Schickard-Institut
Sand 1, Raum A312
72076 Tübingen
Tel.: +497071/2978970
E-Mail: [hidden email]

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Running out of memory on data download

Dannon Baker-2
My first guess is that it's likely this is due to 'debug = True' being set in your universe_wsgi.ini.  If you set this to false and restart, the entire request will no longer be loaded into memory.

-Dannon

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Running out of memory on data download

Björn Grüning-3
In reply to this post by Michael Römer

Hi Michael,

are you compressing the results during download?

Cheers,
Bjoern

> Dear all,
>
> We run our Galaxy server in a virtual machine with less than 1GB RAM.
> When I'm trying to download results via the Download button in the
> corresponding history entry that are larger than 1GB, the results file
> is loaded into memory. This leads to the galaxy process being killed,
> due to memory issues.
>
> Is there a way to omit loading the result into memory?
>
> Regards,
> Michael
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Running out of memory on data download

Michael Römer
Hi Bjoern,

No, I'm not compressing the results. However, Dannon pointed out the problem:

My first guess is that it's likely this is due to 'debug = True' being set in your universe_wsgi.ini.  If you set this to false and restart, the entire request will no longer be loaded into memory.

-Dannon
That solved my problem.

Thanks,
Michael

On 02/20/2014 01:50 PM, Björn Grüning wrote:

Hi Michael,

are you compressing the results during download?

Cheers,
Bjoern

Dear all,

We run our Galaxy server in a virtual machine with less than 1GB RAM.
When I'm trying to download results via the Download button in the
corresponding history entry that are larger than 1GB, the results file
is loaded into memory. This leads to the galaxy process being killed,
due to memory issues.

Is there a way to omit loading the result into memory?

Regards,
Michael




-- 
Michael Römer, MSc. Bioinf.
Abteilung Kognitive Systeme
Fachbereich Informatik
Wilhelm-Schickard-Institut
Sand 1, Raum A312
72076 Tübingen
Tel.: +497071/2978970
E-Mail: [hidden email]

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/