Django file upload

by Roberto on 29-Jul-2007

Some interesting examples on how to handle file uploads using Django:

  • Code snippet posted on Djangosnippets. The file is saved by declaring a save() method in the form class. This method is invoked when calling form.save(), which is standard Django newforms practice. (Note that this snipped uses clean_data. As of Django version 0.96, clean_data has been renamed to cleaned_data, so you will have to change the code or it won’t work)
  • Django image upload and validation. The author uses a model for the file and its related data. The uploaded file is saved by calling the save_FOO_file method. (This method is automatically provided by Django for fields declared as models.ImageField or models.FileField in the model. See the db-api documentation.)
  • Django image upload, form_for_instance and monkey-patching. The example code creates a form class from request.user by calling form_for_instance. The resulting class in then monkey-patched to insert the avatar image validation code. (Although the code is interesting the monkey patch seems unnecessary. I wouldn’t mind inserting the avatar validation method in a UserProfileForm class derived from form.Forms. The code would be certainly clearer: I think KISS takes precedence over DRY in this case.)

Interesting, there seems to be no easy way of limiting the uploaded file size. The file can be rejected at validation time, but the data would have already been transfered.

A file upload recipe

After reading those posts, I think that a good recipe for handling file uploads in Django would be:

  • Write a django model for the uploaded file and its related data. Using a Django model makes sense, because it is usually necessary for the application to keep track of the uploaded files.
  • Write a subclass of form.Forms and declare a clean_FOO method for each models.FileInput or models.ImageInput fields declared in the model class. These clean_FOO methods are used to validate the uploaded files.
  • use a django view to receive the POST data, or display the form if no data is posted or errors are found.
  • validate the uploaded file or files by triggering the standard django newforms validation mechanism: is_valid().
  • save the file or files getting the data directly from the request.FILES object, by writing a save() method for the subclassed form or by calling save_FOO_file for the model instance.

A simpler way to upload a file

The following short Django example uses no data models, does no data validation, and saves the file directly to disk using python standard file functions. It is just a simple test I wrote to get familiar with the request.FILES object. This is not production code: it could be used to execute an arbitrary script on the server. The directory where the file is to be saved must be writable by the user that is running the Django server script. (The example uses MEDIA_ROOT as defined in settings.py.)

file: views.py

from django import http
from django import newforms as forms
from django.shortcuts import render_to_response
from djangotest.settings import MEDIA_ROOT



class SimpleFileForm(forms.Form):
    file = forms.Field(widget=forms.FileInput, required=False)


def directupload(request):
    """
    Saves the file directly from the request object. 
    Disclaimer:  This is code is just an example, and should
    not be used on a real website.  It does not validate
    file uploaded:  it could be used to execute an 
    arbitrary script on the server. 
    """

    template = 'fileupload.html'

    if request['method'] == 'POST':
        if 'file' in request.FILES:
            file = request.FILES['file']

            # Other data on the request.FILES dictionary:
            #   filesize = len(file['content'])   
            #   filetype = file['content-type'] 

            filename = file['filename']

            fd = open('%s/%s' % (MEDIA_ROOT, filename), 'wb')
            fd.write(file['content'])
            fd.close()

            return http.HttpResponseRedirect(' upload_success.html')
    else:
        # display the form
        form = SimpleFileForm()
        return render_to_response(template, { 'form': form })

file: fileupload.html

{% extends "base.html" %}

{% block body  %}
     <h1>Upload a file</h1>
     <form action="." method="post" enctype="multipart/form-data">
             {{ form }}
             <input type="submit" value="Upload" />
     </form>
{% endblock %}

{ 18 comments… read them below or add one }

lorenzo August 22, 2007 at 9:25 am

Excellent code. It helps me a lot, thank you

Reply

Anonymous April 14, 2008 at 1:12 pm

Seems like it doesn’t work with 0.96.1. Any ideas why?

Reply

Roberto April 14, 2008 at 3:22 pm

What exactly does not work? Do you get any error messages?

Reply

Alexander Solovyov May 6, 2008 at 2:47 am

I think it’s better to use request.method instead of request['method']. At least it is documented in first place:

http://www.djangoproject.com/documentation/request_response/#attributes

Reply

SAn May 28, 2008 at 10:37 pm

Many thanks!!!!

Reply

mojave June 13, 2008 at 1:38 pm

Actually using django to handle file uploads at all is a very bad idea. The problem is that django doesn’t buffer the file out to the disk it just loads the whole thing into RAM. Get a couple of people uploading a 5 meg file at the same time…. and well you’re totally screwed unless your running one of those Sun T2′s with 32 GB or RAM. You can mitigate this to some extent by setting the apache POST limit but that’s only if you have control over that and know for certain that you’ll never get a post larger than X.

There is work on a patch taking place…
http://code.djangoproject.com/ticket/2070 but alas it’s been 2 years… and it appears the core team has little to no interest in it.

Reply

Zack July 31, 2008 at 8:16 am

for the function save_FOO_file(filename, content) the ‘content’ attribute does not exist anymore, use the attribute ‘data’

the request.File is now handled by the InMemoryUploadedFile class, and the raw file is called ‘data’

Reply

Alf August 21, 2008 at 1:21 pm

With regards to MOJAVEs comment, this has been fixed for Django 1.0. Check the documentation at http://www.djangoproject.com/documentation/upload_handling/

Reply

Dhruv June 5, 2009 at 4:15 pm

I get this error,

Error: One or more models did not validate:
polls.poll: “file”: To use ImageFields, you need to install the Python Imaging Library. Get it at http://www.pythonware.com/products/pil/ .

Reply

Roberto June 5, 2009 at 8:54 pm

It is as the error message says. You need to install the Python Imaging Library (PIL).

Dhruv June 8, 2009 at 5:21 pm

I did install PIL library through python setup.py install command. It still says me the same thing.

Any thoughts?

Reply

Roberto June 9, 2009 at 9:37 am

Let me do some testing…

brandon October 30, 2009 at 2:47 pm

@mojave

it only puts the whole file in RAM if you use .read()

this is from the docs and will work better.

UploadedFile.chunks()

A generator returning chunks of the file. If multiple_chunks() is True, you should use this method in a loop instead of read().

In practice, it’s often easiest simply to use chunks() all the time; see the example below.

Reply

karthikr March 1, 2011 at 11:56 am

This is a more updated version. I am working on Django 1.2 and the file upload syntax looks like this:

contents = file.file
filename = file.name

Reply

greatghoul August 18, 2011 at 10:15 am

If the file is a bit large, is there a better way to upload it..

Reply

Roberto August 19, 2011 at 6:01 pm

You can use chunks() to upload the file in 2.5MB chunks, as explained in the Django documentation: https://docs.djangoproject.com/en/dev/topics/http/file-uploads/

zoubeida September 18, 2011 at 4:40 am

Hello,

Langue source : français
Saisissez du texte, l’adresse d’un site Web ou importez un document à traduire.
Annuler

I a am beginner in django, I cretate a form with forms.FileField
If I have a lot of people that can upload files at the same time it would be better to save on the disk then to manage it in RAM?
else if I manage my XML file in memory how can i manage the content as xml and not as a string, because in the view.py file when i validate the field :
XMLfile = form.cleaned_data ['XML_file']
it returns a string then i can’t parse it whith xml parser.what shall i do ??
regards

Reply

Roberto September 18, 2011 at 8:31 am

What library are you using to process the XML data? If you are using xml.dom.minidom, for example, you could use xml.dom.minidom.parseString(string). (See http://docs.python.org/library/xml.dom.minidom.html).

About serveral people uploading files at the same time, by defualt, during the upload, Django saves to a temporary file on disk any file bigger than 2.5MB, but this behavior can be modified, as explained in the Django documentation (see https://docs.djangoproject.com/en/dev/topics/http/file-uploads/).

If there are several users uploading files at the same time, it means you have several forks of your webserver running, each using some amount of memory. If you expect a high number of concurrent uploads and force Django to upload all the data to memory, it could consume a lot of RAM on your web server. I would rather save to disk. But I haven’t done any tests myself, maybe you should ask on stackoverflow.com. For example, see this thread: http://stackoverflow.com/questions/7060869/does-django-file-upload-occupy-process-for-duration-of-upload

Leave a Comment

{ 3 trackbacks }

Previous post:

Next post: