For AI agents: the complete documentation index is available at https://docs.ovhcloud.com/en/llms.txt, the full documentation bundle is available at https://docs.ovhcloud.com/en/llms-full.txt, and this page is available as Markdown at https://docs.ovhcloud.com/en/guides/storage-and-backup/object-storage/pcs-optimised-method-for-uploading-files-to-object-storage.md.

Object Storage Swift - Optimised method for uploading files to Object Storage

View as Markdown

Objective

When you want to upload large files to Object Storage (including videos or disk images for example), you can use the Swift OpenStack client in order to optimise the file transfer by breaking down the files.

This guide explains how you can use this feature to upload your files to Object Storage more quickly.

Prerequisites

Instructions

Swift OpenStack lets you store files of any size by breaking them down into several segments.

When a Swift client is used to upload a file, the Swift proxy server determines the correct storage node responsible for the data (based on a hash of the object name). Therefore, it is highly likely that the segments will be stored in several storage nodes, which means you can write your data at a higher speed.

As a result you can upload a 10 GB file in 100 X 100 MB segments:

root@server:~$ swift upload --segment-size 104857600 --segment-threads 100
container_name 10Gio.dat
ArgumentDescription
--segment-sizeSegment size in bytes
--segment-threadsNumber of segments

You can measure the upload speed using iftop.

Go further

For training or technical assistance implementing our solutions, contact your sales representative or visit our Professional Services page to request a quote and have your project analysed by our experts.

Join our community of users.  

Was this page helpful?