Memory leak in reading files from Google Cloud Storage at Google App Engine (python) -

below part of python code running @ google app engine. fetches file google cloud storage using cloudstorage client.

the problem each time code reads big file(about 10m), memory used in instance increase linearly. soon, process terminated due "exceeded soft private memory limit of 128 mb 134 mb after servicing 40 requests total".

class readgsfile(webapp2.requesthandler):     def get(self):         import cloudstorage gcs          self.response.headers['content-type'] = "file type"         read_path = "path/to/file"          gcs.open(read_path, 'r') fp:             buf = fp.read(1000000)             while buf:                 self.response.out.write(buf)                 buf = fp.read(1000000)             fp.close() 

if comment out following line, memory usage in instance change. should problem of webapp2.

  self.response.out.write(buf) 

it supposed webapp2 release memory space after finishing response. in code, not.

suggested above user voscausa's comment, changed scheme file downloading, is, serve file downloading using blobstore. problem of memory leak solved.

reference: https://cloud.google.com/appengine/docs/python/blobstore/#python_using_the_blobstore_api_with_google_cloud_storage

from google.appengine.ext import blobstore google.appengine.ext.webapp import blobstore_handlers  class gcsservinghandler(blobstore_handlers.blobstoredownloadhandler):   def get(self):     read_path = "/path/to/gcs file/"  # leading chars should not "/gs/"     blob_key  = blobstore.create_gs_key("/gs/" + read_path)      f_name = "file name"     f_type = "file type" # such 'text/plain'      self.response.headers['content-type'] = f_type     self.response.headers['content-disposition'] = "attachment; filename=\"%s\";"%f_name     self.response.headers['content-disposition'] += " filename*=utf-8''" + urllib2.quote(f_name.encode("utf8"))      self.send_blob(blob_key)