Programmatically Uploading Large Files Client Side with SharePoint 2013

In this blog post we will detail how it is possible to programmatically upload large files with client side script (JavaScript) using the dedicated start, continue and finish upload functions with SharePoint 2013 Online.

There are plenty of examples about how to use these methods on the server side, but very little on how the same can be achieved with a client side only approach using JavaScript.

As it turns out this is possible, but there are a few things you need to consider and keep in mind before doing so:

  • This will only work on modern web browsers (due to the need for HTML 5 APIs)
  • You will need to build in failure tolerances into your code, to retry if a part of the upload does not work

Form UI

Your form will need an input file control (HTML 5), so that files can be selected. Here is a link to a good article getting you up to speed on how to embed one in your HTML. In this walkthrough, we will hook onto any file being selected by the input control, using the onchange handler and this will allow us to upload the file at the point that it has been selected.

Optionally, you can also include a div or span element which contains a progress indication of how well the upload is going.

JavaScript

The example code that we use in this article is taken from a SharePoint provider hosted app and uses the Request Executor object to communicate with SharePoint. If you were building a script to run directly with SharePoint, you would be able to avoid this step and use a direct REST or JSOM call (in the normal way).

The other point to note is it assumes that you have knowledge of ES6 (let, export etc..) and you will be able to cobble together a Utils script that provides the App Web and Host Web URLs.

 

Walkthrough Guide

Step 1 – File Input HTML Element

The first thing we need to do is hook up the file input control with an onchange event, so that it will call our entry point in our code.

We are assuming here that you will be storing your JS code in a file called fileUploader and that will have a public function called upload (more on this in a second).

Step 2 – Utilities Function

We now need to setup a little utilities function to allow us to grab the App Web and Host Web URLs. If you do not need this functionality in a SharePoint provider Hosted App / Addin, then you can remove these parts from the script (and replace references with your target site URL).

This script will also have a helper method to generate a GUID string to uniquely identify our upload session.

Create a new Utils.js file and add the following code.

Step 3 – Including Utilities and Backwards Capability

So now we need to create the fileUploader.js script and once we have created it, we will need to add an import reference for the utils.js file (that we created in the previous step) and then finally add a fail safe shim for IE, to make sure we have access to the ArrayBuffer.slice method:

Step 4 – Creating a Placeholder File

Now we have everything in place to start writing the code to upload the large file.

The first thing that we need to do is to create a temporary dummy file with the file name that you need as a place holder for when you start the actual upload process – thankfully this is pretty straight forward, we do this using the normal upload process (comments inline):

Note how we have also included method for actioning the request (executeAsync). You could replace this with a straight request to SharePoint if you were not working within a Provider Hosted App / Addin.

At the bottom of the extract, we also have the entry point to our code – the public facing upload function.

Step 5 – Getting Context

You may have noticed that in the example above, we reference a context object that we haven’t established or invoked yet. We need to add a couple of functions which for simplicity, we will add to the top of FileUploader script (underneath the shim) – However in a real world scenario we would have a dedicated file for this…

Step 6 – Beginning the Upload

So now we have a dummy placeholder file sitting in the document library which allows us to start the process of uploading the large file, replacing this temporary one.

First of all we create a new method called getUploadMethod. This will take a look at the length of the file, the current position of the offset in the upload process and the chunk length and work out which of the three upload methods we need to use for the current part of the file (chunk) being uploaded (e.g. are we at the start, end or somewhere in between in the upload process).

Following this, we get into the guts of reading the file from the input object, working out how big our file parts are going to be (held in the chunks array) and we keep track of how much of the file has been uploaded.

Step 7 – Processing the File Part (Working with Chunks)

In this last step of the walk through we will look at the code needed to actually upload the chunk to the library. We do this by recursively calling a new uploadFile function until the upload is complete.

Please Note: To simplify things, we have left out the convertFileToBlobChunks and setLoaderMessage functions from the extract below. You can view these functions in the full extract at the bottom of this article.

One point to note about the extract above, is the additional getFileInformation function which is called at the end of the upload process to get the list item details.

Final Thoughts

That’s really all there is to it (although you will need to build in some failure retry mechanism to make sure you catch any upload failures).

The full fileUploader.js example is shown below:

 

 

 

Programmatically Uploading Large Files Client Side with SharePoint 2013

Leave a Reply

%d bloggers like this: