For AI agents: the complete documentation index is available at https://docs.ovhcloud.com/it/llms.txt, the full documentation bundle is available at https://docs.ovhcloud.com/it/llms-full.txt, and this page is available as Markdown at https://docs.ovhcloud.com/it/guides/public-cloud/data-analytics/analytics/kafka-enable-rest-api.md.
Vai al contenuto

Kafka - How to enable the REST API

Vedi come Markdown

Find out how to enable the REST API of your Kafka service and produce and consume messages over HTTPS

Objective

Apache Kafka is an open-source, distributed event streaming platform designed for real-time, large-scale data processing with high scalability, durability, and low latency.

The Kafka REST API lets applications produce and consume messages over HTTPS, without a Kafka client library.

This guide explains how to enable the REST API of your Kafka service from the OVHcloud Console or the OVHcloud API, then how to produce and consume a first message with curl.

Requirements


OVHcloud Console Access

  • Direct link:
  • Navigation path: Public Cloud > Select your project > Analytics

Instructions

Enabling the REST API

OVHcloud Console
OVHcloud API

Go to the Configuration tab of your Kafka service. In the Kafka features section, turn on the REST API switch.

The REST API can be disabled with the same switch.

Retrieving the REST API endpoint

Once the REST API is enabled, a kafkaRestApi endpoint is added to your service. You can find it in the endpoints list of the service details:

{
  "component": "kafkaRestApi",
  "domain": "kafka-xxxxxxxx-xxxxxxxxx.database.cloud.ovh.net",
  "port": 20188,
  "scheme": "https",
  "ssl": true,
  "uri": "https://<username>:<password>@kafka-xxxxxxxx-xxxxxxxxx.database.cloud.ovh.net:20188"
}

The REST API uses HTTPS with the CA certificate of your service, and the username and password of a Kafka user. Download the CA certificate as ca.pem (see Connect to a Kafka cluster with CLI).

Info

A user you have just created can be rejected with 401 Unauthorized for about a minute, until it is available on the REST API.

Listing topics

curl --cacert ca.pem -u 'app-orders:<password>' https://<host>:<port>/topics
["orders","__consumer_offsets"]

Producing a message

Send a JSON message with the key order-1 to the orders topic:

curl --cacert ca.pem -u 'app-orders:<password>' \
  -H 'Content-Type: application/vnd.kafka.json.v2+json' \
  -X POST https://<host>:<port>/topics/orders \
  -d '{"records":[{"key":"order-1","value":{"amount":42}}]}'

The REST API returns the partition and offset of the message:

{"key_schema_id":null,"offsets":[{"offset":0,"partition":0}],"value_schema_id":null}

Consuming messages

Consuming takes three steps: create a consumer instance in a consumer group, subscribe it to the topic, then read the records.

# 1. Create the consumer instance "c1" in the consumer group "orders-readers"
curl --cacert ca.pem -u 'app-orders:<password>' \
  -H 'Content-Type: application/vnd.kafka.v2+json' \
  -X POST https://<host>:<port>/consumers/orders-readers \
  -d '{"name":"c1","format":"json","auto.offset.reset":"earliest"}'

# 2. Subscribe it to the "orders" topic
curl --cacert ca.pem -u 'app-orders:<password>' \
  -H 'Content-Type: application/vnd.kafka.v2+json' \
  -X POST https://<host>:<port>/consumers/orders-readers/instances/c1/subscription \
  -d '{"topics":["orders"]}'

# 3. Read the records
curl --cacert ca.pem -u 'app-orders:<password>' \
  -H 'Accept: application/vnd.kafka.json.v2+json' \
  https://<host>:<port>/consumers/orders-readers/instances/c1/records

The REST API returns the messages of the topic:

[{"key":"order-1","offset":0,"partition":0,"timestamp":1791971865910,"topic":"orders","value":{"amount":42}}]

When you are done, delete the consumer instance:

curl --cacert ca.pem -u 'app-orders:<password>' \
  -H 'Content-Type: application/vnd.kafka.v2+json' \
  -X DELETE https://<host>:<port>/consumers/orders-readers/instances/c1

Go further

Kafka - How to use Access Control Lists (ACLs)

Kafka - How to enable the Schema Registry

We want your feedback!

We would love to help answer questions and appreciate any feedback you may have.

For training or technical assistance implementing our solutions, contact your sales representative or visit our Professional Services page to request a quote and have your project analysed by our experts.

Are you on Discord? Connect to our channel at https://discord.gg/ovhcloud and interact directly with the team that builds our Analytics service!

Join our community of users.

Questa pagina ti è stata utile?