How to Upload Large Files Using Multipart Upload

Multipart upload allows you to divide a large file into smaller chunks and upload them in parts to S3 Object Storage. When all parts are successfully uploaded, the storage service assembles them into the completed object.

This is a good method to use if you are uploading large videos, archives, backups or other files. If one part fails, it might be possible to retry that part instead of uploading the entire file again.

How Multipart Upload Works

Typically, multipart upload involves three phases:

  1. Start upload The app creates a new multipart upload with the storage service.

  2. Split and upload the pieces: The file is divided and uploaded in small pieces one by one. Where supported by the client the parts can be uploaded in parallel .

  3. Upload complete: The application then sends a request for completion when all the required parts have been uploaded, to assemble the parts into the final object.

Once the multipart upload is successfully completed, the uploaded file will be available as a completed object.

How to Upload a Large File

Exact steps will depend on your storage panel, S3-compatible client or application.

1. Check Multipart Upload Support

Ensure your storage service and upload tool supports multipart uploads.

Some storage panels handle large uploads automatically; others require an S3-compatible client, SDK, or application that supports multipart uploads.

2. Select the File

Select the file you wish to upload, enter the destination Space and object key.

Before you start, make sure you have the right permissions to upload the object, and that you are uploading it to the correct place.

3. Configure the Part Size

The upload tool splits the file into small chunks. The best size of part is determined by the file size, connection reliability, and capabilities of the storage service.

Small parts might make for more requests. Bigger parts will reduce the number of requests but might take longer to upload one by one.

Some S3-compatible services also have minimum and maximum part sizes and a maximum number of parts.

4. Upload the Parts

The client uploads each piece and tracks its progress.

A supported client can retry the failed parts without starting the whole upload again if the connection is lost. If the network and storage service can efficiently handle multiple requests, parallel uploads can increase throughput.

5. Complete the Upload

The client then requests completion following a successful upload of all parts.

wait for the multipart upload to finish Then, make sure the object is in your Space and is downloadable/openable.

How to Fix Failed Multipart Uploads

Upload Stops Before Completion

Why: unstable connection, timeout, failed part.

What to do Check your network connection and the message of error the upload tool gives you. If the client supports resuming multipart uploads, retry the failed upload or part.

Invalid Part Size

Possible causes The specified part size does not comply with the storage service requirements.

What to do Find out the application-specific limits for multipart uploads and set the part size accordingly in your upload tool.

Access Denied

Possible reasons The credentials do not have permission to upload to the object or to complete a multipart upload.

How to: Check credentials, permissions, destination Space, object key. Do not give out secret access keys while troubleshooting.

Upload Completes but the File Is Missing

Possible reasons: The multipart upload was not completed successfully or the object was uploaded to a different destination.

What to do: Verify the target Space and object key, the final completion response. Make sure the upload tool reported success.

Multipart Upload Remains Incomplete

Possible reasons The upload was cancelled before the completion request or it was partially uploaded and then cancelled.

What to do: Verify that the existing upload can be picked up by your client. If it is not you might have to cancel the upload that did not complete and start again. “Do this only when you’re sure that the upload is not needed anymore.

Important Things to Know

When Should You Use Multipart Upload?

Multipart upload is useful for large files, poor network connections, or applications that need to retry individual parts if they fail. A regular upload will suffice for small files.

If your storage panel doesn’t have controls for multipart upload, check to see if large uploads are automatically handled, or if you need an S3-compatible client or application.