Object Storage - Get started with Private Endpoint (Beta)

View as Markdown

Find out how to access your Object Storage buckets privately from a Public Cloud Private Network, with no public connectivity needed, using a Private Endpoint

Warning

Beta feature. Private Endpoint for Object Storage is currently in Beta. Features, API routes and field names may change before General Availability, and this guide will be updated accordingly. During the Beta, you can manage your Private Endpoints from the OVHcloud Console, through the OVHcloud API or with Terraform. This guide covers the Console and the API.

Objective

A Private Endpoint lets the resources on a Public Cloud Private Network subnet reach your S3¹-compatible Object Storage over the OVHcloud network, without a Gateway. Your traffic stays private and needs no public connectivity.

This guide explains how to create a Private Endpoint, connect your S3 client to it, and make sure your buckets can only be reached through this private path.

Info

Private Endpoint is only available for S3-compatible Object Storage. Swift-based Object Storage is not supported.

Requirements

A Public Cloud project in your OVHcloud account
A Public Cloud Private Network and subnet, active in the region where you want to create the Private Endpoint, with one free IP address
An Object Storage user and bucket
Access to the OVHcloud API, the OVHcloud Console or Terraform

The clients that use the Private Endpoint are the resources in this subnet, such as instances or Bare Metal servers connected through the vRack.

OVHcloud Console Access

Direct link:
Navigation path: Public Cloud > Select your project > Network > Private Endpoint

Instructions

How it works

When you create a Private Endpoint, OVHcloud places it in the subnet of your choice and gives it:

a private IP address (VIP) in this subnet;
a hostname, in the form pe-<pe_id>.s3.{region}.endpoint.cloud.ovh.net.

Your instances then send their S3 requests to this hostname instead of the public Object Storage endpoint. Your credentials (access key and secret key) do not change.

Traffic is encrypted with TLS all the way from your S3 client to Object Storage. The Private Endpoint forwards it without decrypting it.

A Private Endpoint has two identifiers. Make sure you use the right one:

Identifier	Example	Where to find it	Used for
Private Endpoint ID	`pe-a1b2c3d4`	In the Console, the ID field of the Private Endpoint details page. In the API, the `currentState.endpointId` field. It is also the first part of the hostname, before `.s3.`: `pe-` followed by 8 hexadecimal characters generated by OVHcloud, for example `pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net`	Access policies (see Step 5)
Resource ID (UUID)	`5d297126-f4bd-46f9-a4c4-2ffd134121b6`	The `id` field of the API response	API calls only: this is the `{privateEndpointId}` in the routes

Using the API: how it works and routes used in this guide

Private Endpoint uses OVHcloud API v2. Instead of performing actions one by one, you describe the configuration you want in a targetSpec object. OVHcloud then applies it in the background. You can follow its progress with the resourceStatus field, and the currentTasks list shows the operations in progress.

The routes used in this guide are the following:

Method	Route	Description
`GET`		List the services and connectivity tiers available
`GET`		List the Private Endpoints of a project
`POST`		Create a Private Endpoint
`GET`		Get the details of a Private Endpoint
`PUT`		Rename a Private Endpoint or change its description
`DELETE`		Delete a Private Endpoint
`GET`		Get the availability of one service

Info

You do not need any local tool to run these calls: you can run them directly from the . Open the route, fill in projectId and the request body, then click Execute. In the same way, you can follow the creation of your Private Endpoint with the GET route, and list the available service names and connectivity tiers with the serviceAvailability route.

Step 1 - Choose a connectivity tier

Each Private Endpoint has a connectivity tier, which determines its bandwidth:

Tier	Bandwidth	Recommended for
S	500 Mbps	Workloads with moderate throughput (default tier)
M	2 Gbps	Workloads with higher throughput
L	4 Gbps	Heavy data transfers: backups, analytics, machine learning pipelines

The bandwidth values apply to the Beta and may change.

Warning

Beta limitations. During the Beta:

only the S tier is available;
you can create up to 2 Private Endpoints per project. If you need more, contact us;
Private Endpoints are free, and no SLA applies;
Private Endpoint is available in the following regions: GRA, RBX, SBG, EU-WEST-PAR, EU-SOUTH-MIL, LIM and BHS.

Via the OVHcloud Console

Via the OVHcloud API

You choose the tier when creating the Private Endpoint (see Step 2), in the Service & size section of the creation form:

In Service, select S3. The list shows the services available in the region of the Private Network you selected.
In Size, select the tier. Tiers are shown as sizes: during the Beta, select S.

Enter the tier in service.tier when creating the Private Endpoint (see Step 2), in uppercase: S, M or L.

To check which services and tiers are available in each region, use the following call:

GET/publicCloud/project/{projectId}/reference/privateEndpoint/serviceAvailability

To display Object Storage only, add /s3 at the end of the route.

Each service lists its available tiers in tiers[]. For each tier, the response gives:

name: the value to enter in service.tier;
bandwidth: the bandwidth of the tier, in Mbps;
location: the region where the tier is available (region, and availabilityZone for regions with several availability zones);
port and transport: the port and protocol used through the Private Endpoint.

Availability is given per tier and per region: look for your region in the response before choosing a tier. For example:

[
  {
    "name": "s3",
    "description": "OVHcloud Object Storage",
    "tiers": [
      { "name": "S", "bandwidth": 500, "location": { "region": "GRA11" }, "port": 443, "transport": "TCP" }
    ]
  }
]

Info

Regions do not have the same name everywhere. In the API (location.region), use the Public Cloud region name, for example GRA11. In the hostname and in the region setting of your S3 client, use the Object Storage region, for example gra.

Step 2 - Create the Private Endpoint

Before you start, decide how the Private Endpoint hostname will be resolved. You make this choice when creating the Private Endpoint, with the Public DNS option (publicDnsEnabled in the API):

Your situation	Public DNS option	What you do next
Your instances can use a public DNS resolver	On	Nothing: OVHcloud publishes and manages the DNS record for you. The record points to a private IP address, which can only be reached from your Private Network.
Your instances cannot use a public DNS resolver, or you do not want the record to be public	Off (default)	You manage resolution yourself, for example in a private DNS zone.

The details for each option are given in Step 3.

Via the OVHcloud Console

Via the OVHcloud API

Open Public Cloud and select your project, then open Network > Private Endpoint.
Click Create a Private Endpoint.
In the Private Endpoint name section, enter a name for your Private Endpoint.
In the Connectivity section:
- select your Private network. The region of the Private Endpoint is the region of this Private Network;
- select the Subnet in which to place the Private Endpoint;
- optionally, enter an IP address of your choice in VIP selection. If you leave it empty, an address in the subnet is assigned automatically.
In the Service & size section, select the service S3 and the connectivity tier (see Step 1). During the Beta, only S is available.
In the Public DNS section, turn the switch on or leave it off, according to your choice above.
Optionally, add a Description.
Check the summary on the right, then click Create a Private Endpoint.

The Private Endpoint is created in the background, and is listed on the Private Endpoints page. It is ready when its status is READY.

Click its name to open its details page, and write down the following values: you will need them in the next steps.

ID, in the General information section: the Private Endpoint ID (pe-…);
Endpoint VIP, in the Connection details section: its IP address in the subnet;
Hostname, in the Connection details section.

Each value has a copy button next to it.

Use the following call:

POST/publicCloud/project/{projectId}/privateEndpoint

with a request body similar to this one:

{
  "targetSpec": {
    "name": "my-s3-endpoint",
    "description": "Private S3 access for the backup subnet",
    "location": { "region": "GRA11" },
    "service": {
      "name": "s3",
      "tier": "S"
    },
    "network": { "id": "<network_id>" },
    "subnet": { "id": "<subnet_id>" },
    "publicDnsEnabled": false
  }
}

Field	Description
`name`, `description`	A name and an optional description for your Private Endpoint.
`location.region`	The region of your Private Network.
`service.name`	The service to reach. Only `s3` is available.
`service.tier`	The connectivity tier (see Step 1). During the Beta, only `S` is available.
`network.id`, `subnet.id`	The IDs of the Private Network and subnet in which to place the Private Endpoint. Both are required.
`publicDnsEnabled`	`true` to publish the Private Endpoint hostname in public DNS, `false` to manage resolution yourself (see the choice above).
`vipAddress`	Optional. A private IP address of your choice in the subnet. It must be within a private range (RFC 1918). If you leave it out, an address is assigned automatically.

The Private Endpoint is created in the background: its resourceStatus is first CREATING. To follow its progress, retrieve its details:

GET/publicCloud/project/{projectId}/privateEndpoint/{privateEndpointId}

The Private Endpoint is ready when resourceStatus is READY and currentTasks is empty. Its hostname is then shown in the currentState.hostname field, and its Private Endpoint ID in the currentState.endpointId field. Write down its IP address (currentState.vipAddress), hostname and Private Endpoint ID: you will need them in the next steps.

Step 3 - Set up name resolution

The standard public endpoint of Object Storage (s3.{region}.io.cloud.ovh.net) is not automatically redirected to your Private Endpoint. Your applications must explicitly use the Private Endpoint hostname, for example pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net.

What you do depends on the Public DNS choice you made in Step 2:

Public DNS on: there is nothing to configure. Go to Step 4.
Public DNS off: in your DNS service, for example a private DNS zone, point the following names to the IP address of the Private Endpoint (Endpoint VIP in the Console, currentState.vipAddress in the API):
- pe-<pe_id>.s3.{region}.endpoint.cloud.ovh.net;
- *.pe-<pe_id>.s3.{region}.endpoint.cloud.ovh.net, which covers URLs that include the bucket name.

Step 4 - Connect your S3 client to the Private Endpoint

Once the Private Endpoint is READY, replace the Object Storage endpoint in your S3 client with the Private Endpoint hostname. In the examples below, replace pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net with your own hostname.

AWS CLI

boto3 (Python)

s3cmd

aws --endpoint-url https://pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net s3 ls
aws --endpoint-url https://pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net s3 cp /data/myfile.txt s3://<bucket_name>

To avoid adding --endpoint-url to each command, you can save the endpoint in a profile in ~/.aws/config:

[profile private-endpoint]
region = gra
endpoint_url = https://pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="https://pe-a1b2c3d4.s3.gra.endpoint.cloud.ovh.net",
    aws_access_key_id="<access_key>",
    aws_secret_access_key="<secret_key>",
    region_name="gra",
)
s3.list_buckets()

You can keep using the access key and secret key of your existing Object Storage user.

Step 5 - Restrict access to your buckets to the Private Endpoint

Creating a Private Endpoint gives your instances a private path to Object Storage, but it does not block access from the public endpoint. To make the Private Endpoint the only way to reach your data, add a restriction to the policy of your Object Storage user.

The following statement denies every request that does not go through your Private Endpoint. Replace pe-<pe_id> with your Private Endpoint ID, shown in the ID field of its details page in the Console, or in the currentState.endpointId field of the API:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyUnlessViaPrivateEndpoint",
      "Effect": "Deny",
      "Action": "s3:*",
      "Resource": "*",
      "Condition": {
        "StringNotEquals": {
          "aws:SourceVpce": "pe-<pe_id>"
        }
      }
    }
  ]
}

Warning

In aws:SourceVpce, use the Private Endpoint ID, including the pe- prefix. Copy the value of the ID field in the Console, or of currentState.endpointId as it is: it already includes the prefix. For example, with "endpointId": "pe-9d34106b", use "aws:SourceVpce": "pe-9d34106b".

Do not use the resource ID (UUID) returned by the API, and do not leave out the pe- prefix. Otherwise, the condition never matches and the user is blocked on every path, including through the Private Endpoint.

If your user already has a policy, add this statement to its existing Statement list rather than replacing it.

To apply the policy, use one of the following methods.

Via the OVHcloud Console

Via the AWS CLI

Via the OVHcloud API

The Console does not yet offer a dedicated option for Private Endpoints, but you can edit the user's policy as a JSON file:

In the Object Storage section, open the users tab and download the current policy of your user (Download the policy).
Add the statement above to the downloaded file.
Import the updated file on the same user (Import a policy).

For more details, see our guide on identity and access management.

Save the policy in a policy.json file, then use the put-user-policy command with the --endpoint-url option, so that the request goes to OVHcloud Object Storage rather than AWS:

aws iam put-user-policy \
  --endpoint-url https://s3.<region>.io.cloud.ovh.net \
  --user-name <user_name> \
  --policy-name private-endpoint-only \
  --policy-document file://policy.json

Parameter	Description
`--endpoint-url`	Your Object Storage endpoint, for example `https://s3.gra.io.cloud.ovh.net`.
`--user-name`	The name of the Object Storage user to restrict.
`--policy-name`	A name of your choice for this policy.
`--policy-document`	The path to your JSON file, preceded by `file://`.

Warning

Once this policy is applied, the user can no longer access Object Storage through the public endpoint, from anywhere. Before applying it to a production user, test it on a dedicated user and check that your applications reach Object Storage through the Private Endpoint.

Step 6 (optional) - Reach the Private Endpoint from several subnets

A Private Endpoint is placed in a single subnet. If instances in other subnets of the same Private Network also need private access to Object Storage, you have two options:

Create one Private Endpoint per subnet. This is the simplest option and requires no routing. During the Beta, Private Endpoints are free, but each one counts against the limit of 2 Private Endpoints per project.
Use a single Private Endpoint and a router instance. Place the Private Endpoint in one subnet, and route traffic from the other subnets through a dedicated router instance connected to these subnets. A single Private Endpoint serves all your subnets, but you have to manage the router instance yourself.

Instances in Subnet A reach the Private Endpoint directly. Instances in Subnet B reach it through the router instance that connects the two subnets.

Manage your Private Endpoints

OVHcloud IAM controls who can view, create, edit and delete Private Endpoints. The actions apply to the Public Cloud project (publicCloudProject resource type):

publicCloudProject:apiovh:privateEndpoint/get
publicCloudProject:apiovh:privateEndpoint/create
publicCloudProject:apiovh:privateEndpoint/edit
publicCloudProject:apiovh:privateEndpoint/delete
publicCloudProject:apiovh:reference/privateEndpoint/serviceAvailability/get

Only the name and description can be changed. The tier, the Private Network and the subnet cannot be changed after creation: to change them, create a new Private Endpoint, then delete the old one.

Via the OVHcloud Console

Via the OVHcloud API

Open Network > Private Endpoint. The page lists the Private Endpoints of your project, with their region, Private Network, subnet, service, size, VIP, hostname and status. Use Search to find one.

Change the description: click the name of the Private Endpoint, then click the pencil icon next to Description in the General information section.

Delete a Private Endpoint: click the bin icon in the Actions column of the list, or click Delete on the details page of the Private Endpoint.

List the Private Endpoints of a project:

GET/publicCloud/project/{projectId}/privateEndpoint

Rename a Private Endpoint or change its description:

First retrieve the details of the Private Endpoint with a GET call and write down the value of its checksum field. Then send this value back with your changes:

PUT/publicCloud/project/{projectId}/privateEndpoint/{privateEndpointId}

{
  "checksum": "<checksum>",
  "targetSpec": {
    "name": "my-s3-endpoint-renamed",
    "description": "Updated description"
  }
}

The checksum ensures that nobody has modified the Private Endpoint in the meantime. If it has been modified, the request is rejected with a 409 Conflict error: retrieve the details again and repeat the request with the new checksum.

Delete a Private Endpoint:

DELETE/publicCloud/project/{projectId}/privateEndpoint/{privateEndpointId}

The deletion takes place in the background: resourceStatus is DELETING until the operation is complete.

Warning

If your Object Storage user policy only allows access through this Private Endpoint (see Step 5), update the policy before deleting the Private Endpoint. Otherwise, the user will no longer be able to access your buckets.

Coming soon

The following features are planned after the Beta:

restricting access to your buckets with bucket policies;
restricting access to your buckets with OVHcloud IAM policies, once Object Storage is integrated into OVHcloud IAM.

Go further

Object Storage - Identity and access management

Object Storage - How to connect Object Storage buckets with other resources in a vRack

For training or technical assistance implementing our solutions, contact your sales representative or visit our Professional Services page to request a quote and have your project analysed by our experts.

Join our community of users.

¹: S3 is a trademark of Amazon Technologies, Inc. OVHcloud's service is not sponsored by, endorsed by, or otherwise affiliated with Amazon Technologies, Inc.

Was this page helpful?