Configure Egress Private Endpoints

On this page

Establish a secure network connection from a CockroachDB Advanced cluster to send changefeeds to your private cloud infrastructure with egress private endpoints. You can use the CockroachDB Cloud API to create and manage egress private endpoints on a CockroachDB Advanced cluster.

CockroachDB Cloud supports egress private endpoints with the following cloud services:

• Amazon Virtual Private Cloud (AWS VPC)
• Amazon Managed Streaming for Apache Kafka (MSK) (MSK Provisioned only. MSK Serverless is not supported.)
• Google Cloud VPC Private Service Connect (GCP PSC)
• Azure Private Link Service
• Confluent Cloud on GCP, AWS, or Azure

Prerequisites

Refer to the following sections for prerequisites that apply to the corresponding cloud service:

AWS VPC

The CockroachDB Cloud AWS account must be added as a principal on the endpoint service. Using your CockroachDB Cloud account_id, add the arn:aws:iam::<CC_ACCOUNT_ID>:root principal to the endpoint service definition.

You can use the following API call to retrieve your CockroachDB Cloud account_id:

curl --request GET \
  --url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id} \
  --header "Authorization: Bearer {secret_key}" | jq .account_id

AWS MSK

The following prerequisites apply to the MSK service:

• The cluster must not use kafka.t3.small instances.
• If the cluster is not using IAM authentication, set the allow.everyone.if.no.acl.found=false ACL.
• Multi-VPC Connectivity must be enabled.

• Using the account_id returned from the GET /api/v1/clusters/{cluster_id}, include the following in the cluster policy:

  {
    "Effect": "Allow",
    "Principal": {
      "AWS": "arn:aws:iam::<CC_ACCOUNT_ID>:root"
    },
    "Action": [
      "kafka:CreateVpcConnection",
      "kafka:GetBootstrapBrokers",
      "kafka:DescribeCluster",
      "kafka:DescribeClusterV2"
    ],
    "Resource": "<CUSTOMER_MSK_CLUSTER_ARN>"
  }
  

GCP PSC

The following prerequisites apply to the Google Cloud VPC service:

• The CockroachDB Cloud account's GCP project must be granted explicit approval as a consumer. Follow the Google Cloud PSC documentation, and follow the steps to Accept connections for selected projects with your CockroachDB Cloud GCP account ID.

  You can use the following API call to retrieve your CockroachDB Cloud account_id:

  curl --request GET \
    --url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id} \
    --header "Authorization: Bearer {secret_key}" | jq .account_id
  

• Enable consumer global access on the service load balancer or forwarding rule.

Azure Private Link Service

The following prerequisites apply to Azure Private Link Service:

• An Azure Private Link Service backed by a Standard SKU Internal Load Balancer must be created in your Azure subscription. Basic SKU load balancers do not support the Private Link Service.

• The CockroachDB Cloud Azure subscription does not need to be pre-authorized on your Private Link Service. By default, after CockroachDB Cloud creates the private endpoint, the connection appears in Pending state on your PLS and must be approved manually. To prevent the need for manual approval, you can add CockroachDB Cloud's Azure subscription ID to your PLS auto-approval list during PLS creation.

  You can use the following API call to retrieve your CockroachDB Cloud Azure subscription ID:

  curl --request GET \
    --url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id} \
    --header "Authorization: Bearer {secret_key}" | jq .account_id
  

• Approve the connection via:

  • Azure Portal: Private Link Center > Private Link Services > select your service > Private endpoint connections > Approve
  • Azure CLI: az network private-endpoint-connection approve
  • See the Azure Private Link documentation for detailed steps.

• The Private Link Service must be in the same Azure region as the CockroachDB Cloud cluster region where the endpoint is created. Cross-region connections are not supported in Azure.

Confluent Cloud

You can configure egress private endpoints to connect to an AWS, GCP, or Azure private service configured in a Confluent account. Endpoint creation follows the same process and syntax as for AWS, GCP, or Azure.

Confluent Cloud requires a custom DNS configuration due to the TLS certificates provisioned for their Kafka clusters. Collect the required domain names from Confluent. After the endpoint is created, configure custom DNS records for the cluster.

Create an egress private endpoint

A user with the Cluster Admin, Cluster Operator, or Cluster Creator roles can create an egress private endpoint by sending a POST request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints endpoint with the following payload information:

• cluster_id: The CockroachDB Cloud cluster ID.
• region: The region code where the target service identifier is located (e.g. us-east-1).
• target_service_identifier: The unique identifier of the target service, which is a different value depending on the service:
  • AWS VPC: The AWS private service name.
  • MSK: The MSK-provisioned cluster's Amazon Resource Name (ARN).
  • GCP PSC: The GCP service attachment.
  • Azure Private Link Service: Either the Azure Private Link Service resource ID (recommended) or the PLS alias.
    • Resource ID format: /subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Network/privateLinkServices/{pls-name}
    • Alias format: {prefix}.{guid}.{region}.azure.privatelinkservice
    • To find these values:
      • Portal: Navigate to your Private Link Service > Settings > Properties for the resource ID, or Overview > Alias for the alias.
      • CLI: az network private-link-service show --name {pls-name} --resource-group {rg} --query '{id:id, alias:alias}'
• target_service_type: Description of the service type, dependent on the service and authentication method:
  • AWS VPC, GCP PSC, or Azure Private Link Service: Set to PRIVATE_SERVICE.
  • MSK with SASL/SCRAM authentication: Set to MSK_SASL_SCRAM.
  • MSK with IAM access control: Set to MSK_SASL_IAM.
  • MSK with mutual TLS authentication: Set to MSK_TLS.

Once this request is sent, the CockroachDB Cloud cluster enters a maintenance mode where other configuration changes (cluster scaling, feature configuration, upgrades, etc) cannot be made until the operation is complete. The operation is complete when the endpoint status is AVAILABLE and both the endpoint_id and endpoint_address fields are populated.

Example endpoint creation requests

The following example POST requests assume that an API key has been created for a user with the appropriate role, such as Cluster Operator:

AWS private service endpoint

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east-1",
  "target_service_identifier": "com.amazonaws.vpce.us-east-1.vpce-svc-example",
  "target_service_type": "PRIVATE_SERVICE"
}'

Amazon CloudWatch logs export endpoint

Log export to Amazon CloudWatch requires that you create a private service endpoint for each CockroachDB Cloud region, populating target_service_identifier with the domain name of a CloudWatch instance in that region. Since CloudWatch is an AWS-managed service, logs are scoped to the AWS account where the endpoint is created. The access keys on the export dictate which CloudWatch account receives the logs.

To export logs from multiple Cloud clusters across different regions to a single CloudWatch instance, configure custom DNS so that each target_service_identifier value resolves to the same target CloudWatch endpoint. In this situation, the logexport endpoint automatically sets the region field to the region of the CloudWatch instance.

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east-1",
  "target_service_identifier": "com.amazonaws.us-east-1.log",
  "target_service_type": "PRIVATE_SERVICE"
}'

For more information about log export to Amazon CloudWatch, read the log export documentation.

MSK cluster endpoint

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east-2",
  "target_service_identifier": "arn:aws:kafka:us-east-2:example:cluster/msk-example/99bcd320-3af1-42cc-b8cc-example-7",
  "target_service_type": "MSK_SASL_SCRAM"
}'

GCP private service endpoint

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "us-east1",
  "target_service_identifier": "projects/cc-example/regions/us-east1/serviceAttachments/s-example-service-attachment-us-east1-d",
  "target_service_type": "PRIVATE_SERVICE"
}'

Azure private service endpoint

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-X POST \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "region": "eastus",
  "target_service_identifier": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Network/privateLinkServices/{pls-name}",
  "target_service_type": "PRIVATE_SERVICE"
}'

Example response

{
    "id": "{endpoint_id}",
    "endpoint_connection_id": "",
    "region": "us-east-2",
    "target_service_identifier": "com.amazonaws.vpce.us-east-2.vpce-svc-example",
    "target_service_type": "PRIVATE_SERVICE",
    "endpoint_address": ""
}

Configure custom DNS

If the cloud service has a TLS certificate that requires traffic to be sent from a specific domain, such as Confluent Cloud, you can use the Cloud API to create custom DNS records for a CockroachDB Cloud cluster that points to the endpoint_address of an egress private endpoint.

Before creating a custom DNS record, check that the endpoint status is in the AVAILABLE state. Save the egress_private_endpoints.id value for later use. This ID is distinct from the endpoint_connection_id, which is an external identifier.

Send a PATCH request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id}/domain-names endpoint with the following payload information:

• cluster_id: The CockroachDB Cloud cluster ID.
• endpoint_id: The id value of the egress private endpoint.
• domain_names: A list of domain names to resolve to the private endpoint, as required by the cloud service provider.

For example:

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id}/domain-names \
-X PATCH \
-H "Authorization: Bearer {secret_key}" \
-H 'Content-Type: application/json' \
-d '{
  "cluster_id": "{cluster_id}",
  "endpoint_id": "{endpoint_id}",
  "domain_names": ["*.us-east-2.aws.confluent.cloud"]
}'

The cluster enters maintenance mode once more until the DNS setup is complete, which may take a minute or longer. Traffic from the CockroachDB Cloud cluster should now be routed appropriately to the private endpoint.

Check the endpoint status

Send a GET request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints endpoint to review the status of all egress private endpoints on the cluster:

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints \
-H "Authorization: Bearer {secret_key}"

The response lists all egress private endpoints on the cluster. The state field indicates the status of each endpoint:

{
    "egress_private_endpoints": [
        {
            "id": "{endpoint_id}",
            "endpoint_connection_id": "vpce-example",
            "region": "us-east-2",
            "target_service_identifier": "com.amazonaws.vpce.us-east-2.vpce-svc-example",
            "target_service_type": "PRIVATE_SERVICE",
            "endpoint_address": "vpce-example-onq0bw5q.vpce-svc-example.us-east-2.vpce.amazonaws.com",
            "state": "AVAILABLE",
            "domain_names": []
        }
    ],
    "pagination": null
}

The following list describes all of the possible state values and their meanings:

• PENDING: The endpoint is in the process of being created.
• PENDING_ACCEPTANCE: The endpoint needs to be manually accepted on the cloud provider service.
• AVAILABLE: The endpoint has been created and is available for traffic.
• DELETING: The endpoint is in the process of being deleted from the cluster.
• REJECTED: The endpoint connection was rejected on the cloud provider side.
• FAILED: Something went wrong with the creation of the endpoint in the cloud provider.
• EXPIRED: The connection request expired before it was accepted on the cloud provider service.
• UNSPECIFIED: Could not determine the current state of the endpoint.

Delete a private endpoint

To delete a private endpoint, send a DELETE request to the /api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id} endpoint specifying the following information:

• cluster_id: The CockroachDB Cloud cluster ID.
• endpoint_id: The id value of the egress private endpoint.

curl https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/networking/egress-private-endpoints/{endpoint_id} \
-X DELETE \
-H "Authorization: Bearer {secret_key}"

The endpoint briefly enters the DELETING state then is removed from the list of endpoints on the cluster.