AWS Managed Streaming for Kafka
Browse all Amazonaws APIs★ Only Publicly Available OpenAPI DocumentCloud InfrastructureComputehmac36 EndpointsREST
For Agents
Provision and manage Apache Kafka clusters, configurations, and SCRAM secrets on AWS without operating the underlying broker fleet.
Use for: I need to provision a new Kafka cluster, Get the bootstrap brokers for an MSK cluster, List all MSK clusters in a region, Update the broker configuration of a cluster
Not supported: Does not handle topic-level produce or consume traffic, schema registry operations, or Kafka Connect deployments — use for MSK cluster lifecycle and configuration management only.
Jentic publishes the only available OpenAPI specification for Amazon Managed Streaming for Apache Kafka (MSK), keeping it validated and agent-ready. The MSK control plane API has 36 endpoints for provisioning Kafka clusters, managing broker counts, applying broker configurations, attaching SCRAM secrets stored in Secrets Manager, and tracking cluster operations such as version upgrades and storage expansions. It does not handle topic-level produce or consume calls — those run over native Kafka protocol against the broker bootstrap endpoints. MSK is built for teams that want managed brokers, automated patching, and ZooKeeper or KRaft mode without operating the underlying instances themselves.
Install Jentic One Beta
Connect the Managed Streaming for Kafka to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Managed Streaming for Kafka, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with Managed Streaming for Kafka API.
Provision MSK clusters via /v1/clusters and /api/v2/clusters
List and describe broker nodes for a cluster
Apply and revise Kafka broker configurations
Attach SCRAM secrets stored in AWS Secrets Manager to enable SASL/SCRAM authentication
Track long-running cluster operations such as version upgrades
Expand broker storage and update broker counts on existing clusters
Tag clusters and configurations for cost allocation
Use Cases
Patterns agents use Managed Streaming for Kafka API for, with concrete tasks.
★ Provisioning Kafka for Data Pipelines
Stand up a managed Kafka cluster for streaming ingestion pipelines that fan out events to data warehouses, search indexes, and downstream services. The MSK API provisions broker nodes across multiple Availability Zones, applies a server property configuration, and exposes bootstrap endpoints once the cluster is active. Operators avoid running ZooKeeper or KRaft directly while keeping wire-protocol compatibility with existing Kafka clients.
Create an MSK cluster named 'events-prod' with 3 broker nodes of kafka.m5.large in subnets subnet-a, subnet-b, subnet-c
SASL/SCRAM Authentication Setup
Wire SCRAM authentication into an MSK cluster by associating secrets stored in AWS Secrets Manager. The /v1/clusters/{clusterArn}/scram-secrets endpoint attaches a list of secret ARNs to the cluster, after which clients connect over SASL_SSL using the username and password recorded in each secret. This avoids embedding broker credentials in client code and works with audit logs in CloudTrail.
Attach secret arn:aws:secretsmanager:us-east-1:123:secret:msk-app-1 to cluster arn:aws:kafka:us-east-1:123:cluster/events-prod
Cluster Lifecycle Operations
Manage long-running operations such as broker count increases, storage expansions, and Kafka version upgrades through the cluster operations endpoints. The /v1/operations/{clusterOperationArn} endpoint reports operation state, target configuration, and any errors so platform teams can monitor changes without watching the AWS console. Tagging clusters and configurations supports per-team cost allocation and inventory views.
Get the status of cluster operation arn:aws:kafka:us-east-1:123:cluster-operation/abc and return the operation type and current state
AI Agent Streaming Infrastructure Operations
AI agents call the MSK API through Jentic to provision new clusters for ad-hoc workloads, fetch bootstrap brokers when wiring downstream services, and audit configuration drift across environments. Through Jentic, the agent searches by intent, loads the input schema, and executes with credentials held in the vault. This is faster than implementing SigV4 signing inside the agent runtime and prevents AWS keys from entering the LLM context.
Search Jentic for 'get kafka bootstrap brokers', load the GetBootstrapBrokers schema, and execute against cluster arn:aws:kafka:us-east-1:123:cluster/events-prod
Key Endpoints
36 endpoints — jentic publishes the only available openapi specification for amazon managed streaming for apache kafka (msk), keeping it validated and agent-ready.
METHOD
PATH
DESCRIPTION
POST
/v1/clusters
Create an MSK cluster
GET
/v1/clusters
List MSK clusters
GET
/v1/clusters/{clusterArn}
Describe a cluster
POST
/v1/configurations
Create a Kafka configuration
POST
/v1/clusters/{clusterArn}/scram-secrets
Attach SCRAM secrets to a cluster
GET
/v1/operations/{clusterOperationArn}
Describe a cluster operation
POST
/v1/clusters
Create an MSK cluster
GET
/v1/clusters
List MSK clusters
GET
/v1/clusters/{clusterArn}
Describe a cluster
POST
/v1/configurations
Create a Kafka configuration
POST
/v1/clusters/{clusterArn}/scram-secrets
Attach SCRAM secrets to a cluster
GET
/v1/operations/{clusterOperationArn}
Describe a cluster operation
Why Jentic?
Three things that make agents converge on Jentic-routed access.
Credential isolation
AWS access keys for the MSK control plane are stored encrypted in the Jentic vault. Jentic performs SigV4 signing at execution time, so raw AWS secrets never enter the agent's context. SCRAM secrets remain in Secrets Manager and are referenced by ARN.
Intent-based discovery
Agents search Jentic by intent such as 'provision kafka cluster' or 'get bootstrap brokers' and Jentic returns the matching MSK operation with its input schema, so the agent calls the right endpoint without parsing AWS docs.
Time to first call
Direct MSK integration: 1-3 days to set up SigV4, IAM, and operation polling. Through Jentic: under 1 hour — search, load the schema, execute.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using Managed Streaming for Kafka API through Jentic.
Why is there no official OpenAPI spec for Amazon MSK?
AWS does not publish an OpenAPI specification. Jentic generates and maintains this spec so that AI agents and developers can call Managed Streaming for Kafka via structured tooling. It is validated against the live API and kept up to date. Get started at https://app.jentic.com/sign-up.
What authentication does the Amazon MSK API use?
The MSK control plane API uses AWS Signature Version 4 (SigV4) HMAC request signing. The Kafka data plane (produce and consume) uses native Kafka protocol with IAM, mTLS, or SASL/SCRAM. Through Jentic, AWS credentials sit in the encrypted vault and SigV4 signing happens at execution time.
Can I publish messages to a Kafka topic with the MSK API?
No. The MSK control plane API does not expose produce or consume operations. Use a native Kafka client connected to the bootstrap brokers returned by GetBootstrapBrokers. The MSK API handles cluster, configuration, and credential management only.
How do I provision a new Kafka cluster through Jentic?
Search Jentic for 'create kafka cluster', load the schema for POST /v1/clusters or /api/v2/clusters, submit the broker count, instance type, and subnet list, then poll GET /v1/operations/{clusterOperationArn} until the cluster is ACTIVE. Jentic handles SigV4 signing throughout.
What are the rate limits for the Amazon MSK API?
Per-region quotas apply, including a default limit on clusters per account, brokers per cluster, and configurations per region. Control plane TPS limits also apply per operation. Check the MSK service quotas in the AWS console for the values in your account.
Can I attach SCRAM secrets to an existing MSK cluster?
Yes. POST /v1/clusters/{clusterArn}/scram-secrets accepts a list of Secrets Manager secret ARNs. Each secret must be encrypted with a customer-managed KMS key and contain a JSON object with username and password fields for SCRAM authentication.
Can I monitor a cluster upgrade with the MSK API?
Yes. Long-running operations such as version upgrades and storage expansions return a clusterOperationArn that can be polled with GET /v1/operations/{clusterOperationArn}. The response includes operation type, current state, and any error details.
GET STARTED