AWS PrivateLink & VPC Endpoints: Interface vs Gateway, Private Service Access Hands-On

Quick take: a gateway endpoint is a route, a interface endpoint is a network card. The gateway type adds one line to your route table so S3 and DynamoDB traffic never leaves the VPC — free, but only those two services. The interface type drops an ENI with a private IP into your subnets, powered by PrivateLink, so any supported service (SSM, KMS, ECR, your own app) answers on a private address — powerful, but you pay per hour and per gigabyte, and a security group on that ENI is the real gate.

Every serious AWS network eventually faces the same question: how does a machine with no public IP, no Internet Gateway, and no NAT still reach S3, pull an image from ECR, be managed by Systems Manager, or call a partner API — without a single packet touching the public internet? The naive answer is “give it a route to a NAT gateway.” That works, but it is the expensive answer: NAT bills you per hour and per gigabyte processed, and it puts your private workloads one misconfiguration away from talking to the internet at large. The right answer, for AWS service traffic and for private service-to-service calls, is a VPC endpoint — and knowing which of the two endpoint types to use is one of the sharpest lines between a junior and a senior AWS network engineer.

There are two endpoint families and they are almost nothing alike under the hood. A gateway endpoint is a target you add to a route table; it works for exactly S3 and DynamoDB, costs nothing, has no ENI, no IP address, and no security group. An interface endpoint is an elastic network interface (ENI) with a private IP that AWS places in your subnets; it is powered by AWS PrivateLink, works for a long list of AWS services and for services you or a partner publish, resolves the service’s normal DNS name privately, is guarded by a security group, and is billed per AZ-hour plus per GB. Confuse the two — reach for an interface endpoint for DynamoDB (it does not exist), or wonder why your “available” SSM endpoint times out (the ENI’s security group blocks 443) — and you will lose an afternoon.

This guide is the senior-engineer treatment. You will learn why keeping traffic off the internet is both a security and a cost decision, the exact mechanics of gateway endpoints (route tables, managed prefix lists, endpoint policies) and interface endpoints (ENIs, private DNS, security groups, PrivateLink), a full side-by-side comparison, how endpoint policies restrict which principals and resources an endpoint may reach, and — the crown jewel — how to expose your own service with an NLB-backed VPC endpoint service that a consumer in another VPC or account reaches through an interface endpoint, with no VPC peering and no CIDR-overlap pain. Then you will build all of it in a hands-on lab: an S3 gateway endpoint, the three interface endpoints that let a private instance use SSM with no NAT, and a tiny PrivateLink service end to end — with aws CLI and Terraform — followed by a troubleshooting playbook for every failure these things throw.

What problem this solves

A private subnet is a subnet whose route table has no 0.0.0.0/0 route to an Internet Gateway. Instances there cannot be reached from the internet, and by default cannot reach out either. But they still need AWS services: pushing logs to CloudWatch, reading objects from S3, being managed by SSM, pulling container images from ECR. You have four ways to bridge that gap, and they differ enormously in cost, security, and blast radius.

What breaks when you get this wrong is concrete and expensive. A data-heavy job in a private subnet that reads terabytes from S3 through a NAT gateway pays the NAT data-processing charge on every gigabyte — a bill that a free S3 gateway endpoint erases entirely. A fleet of private instances you “manage with SSM” silently falls off the managed-instances list because the SSM agent cannot reach ssm.<region>.amazonaws.com without a NAT or the right interface endpoints. A compliance auditor asks you to prove that your S3 buckets are only reachable from inside the VPC, and you cannot, because your traffic goes out through NAT to the public S3 endpoint like everyone else’s. And a partner integration that “must not traverse the internet” cannot be satisfied by NAT at all — it needs PrivateLink.

Who hits this: every network or platform engineer building a landing zone; anyone running workloads in private subnets (which should be most of them); teams under PCI/HIPAA/RBI mandates that forbid internet paths for regulated data; SaaS builders who need to publish a service to customers privately; and every candidate sitting SAA-C03, ANS-C01 (Advanced Networking), or SCS-C02 (Security), where interface-vs-gateway and PrivateLink are guaranteed territory.

Learning objectives

By the end of this guide you can:

• Explain the two reasons to keep traffic off the internet — security isolation and skipping NAT data-processing cost — and quantify the second.
• Build and reason about a gateway endpoint: the route-table entry, the managed prefix list, the endpoint policy, and why it is S3/DynamoDB-only and free.
• Build and reason about an interface endpoint: the ENI and private IP, PrivateLink, private DNS, the security group on the ENI, and the per-hour + per-GB cost.
• Choose interface vs gateway for any given service with a decision table, and know the traps (DynamoDB has no interface endpoint; S3 has both).
• Write endpoint policies that restrict which principals and resources an endpoint may reach, and pair them with aws:sourceVpce conditions on resource policies.
• Stand up a VPC endpoint service: an NLB fronting your app, allowed principals, acceptance, custom private DNS, consumed by an interface endpoint in another VPC/account with no peering and overlapping CIDRs allowed.
• Handle cross-region and hybrid (on-prem) access correctly — including why a gateway endpoint is unreachable from on-prem.
• Diagnose every endpoint failure: unreachable S3 via gateway, DNS still resolving public, SG blocking the ENI, pendingAcceptance, cross-account allow-list, and region mismatch.

Prerequisites & where this fits

This is an advanced networking topic. You should already be fluent with VPC fundamentals; if any row below is shaky, read the linked foundation first.

Where this fits: VPC endpoints are the connective tissue that lets a well-segmented network stay well-segmented. In a landing zone you typically run a set of centralized interface endpoints in a shared-services VPC reachable over Transit Gateway, plus a free gateway endpoint in every VPC that touches S3 or DynamoDB. PrivateLink is also how you consume third-party SaaS privately and how you publish your own internal platform services across account boundaries. Master this and you rarely need NAT for anything except genuine internet calls.

Core concepts

Three ideas carry the topic: a VPC endpoint is a private on-ramp to a service; there are two structurally different types; and PrivateLink is the technology that makes the interface type (and endpoint services) work.

The mental model: a private door into a service

A VPC endpoint lets resources in your VPC connect to a service using private IP addressing, so the traffic stays on the AWS network and never traverses the public internet or needs an Internet Gateway, NAT device, or public IP. That is the whole idea. The two implementations of that idea could not be more different.

The two types at a glance

Keep this teaser table in your head; the rest of the article expands every row.

What PrivateLink actually is

AWS PrivateLink is the service that connects a consumer to a provider over the AWS backbone using an interface endpoint (an ENI) on the consumer side and a Network Load Balancer on the provider side, without exposing anything to the public internet, without VPC peering, and — crucially — regardless of whether the two VPCs have overlapping CIDR blocks. Traffic is unidirectional (consumer initiates to provider) and is effectively NATed through the ENI, which is why overlapping addresses do not matter. Every interface endpoint you create for an AWS service is PrivateLink under the hood; publishing your own endpoint service is the same machinery pointed at your NLB.

Why keep traffic off the internet

There are two independent reasons, and both matter on their own.

Security: shrink the reachable surface

The exfiltration point is the one seniors care about most. With a NAT path, a private host can reach any S3 bucket on the internet, including an attacker-controlled one. With a gateway endpoint carrying a tight endpoint policy (Resource limited to your buckets), that same host can only talk to your S3 — the endpoint becomes a data-egress firewall for S3/DynamoDB.

Cost: the NAT bill you stop paying

For S3/DynamoDB traffic the money argument is decisive because the gateway endpoint is free and removes NAT data-processing charges.

For a chatty S3 workload, the free gateway endpoint can save hundreds of dollars a month versus NAT — the single easiest network cost win in AWS. For interface endpoints the calculus is subtler: you add a per-hour + per-GB cost, but you remove NAT for that service’s traffic and gain private, SG-gated access. The rule of thumb: use the free gateway endpoint for S3/DynamoDB always; use interface endpoints for the AWS services your private workloads actually call, and consolidate them.

Gateway endpoints in depth

A gateway endpoint is the simpler, older mechanism. It is not an appliance — it is a route. When you create one, AWS adds a route to the route tables you associate with it: destination = the service’s managed prefix list (e.g. pl-78a54011 for S3 in ap-south-1), target = the endpoint id. Any instance whose subnet uses that route table now reaches S3 over the AWS network instead of via IGW/NAT.

The defining facts

Settings you actually set

The endpoint policy

Every gateway endpoint carries a resource policy. The default is full access — Principal:*, Action:*, Resource:* — which most teams never tighten and should. Restricting Resource to your own buckets turns the endpoint into an S3 egress control.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "OnlyKloudVinBuckets",
    "Effect": "Allow",
    "Principal": "*",
    "Action": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
    "Resource": [
      "arn:aws:s3:::kloudvin-data",
      "arn:aws:s3:::kloudvin-data/*"
    ]
  }]
}

Create it — CLI and Terraform

# Gateway endpoint for S3, wired into a route table, with a scoped policy
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123 \
  --service-name com.amazonaws.ap-south-1.s3 \
  --vpc-endpoint-type Gateway \
  --route-table-ids rtb-0priv1 rtb-0priv2 \
  --policy-document file://s3-endpoint-policy.json \
  --query "VpcEndpoint.{Id:VpcEndpointId,State:State}"

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.ap-south-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_route_table.private_a.id, aws_route_table.private_b.id]
  policy            = data.aws_iam_policy_document.s3_endpoint.json
  tags              = { Name = "s3-gw" }
}

resource "aws_vpc_endpoint" "dynamodb" {
  vpc_id            = aws_vpc.main.id
  service_name      = "com.amazonaws.ap-south-1.dynamodb"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_route_table.private_a.id, aws_route_table.private_b.id]
}

Verify the route landed: aws ec2 describe-route-tables --route-table-ids rtb-0priv1 --query "RouteTables[].Routes[?GatewayId!=null]" should show a route whose DestinationPrefixListId is the S3 prefix list and whose GatewayId is your vpce-….

Interface endpoints & PrivateLink in depth

An interface endpoint is a different animal: AWS provisions an ENI with a private IP in each subnet (AZ) you select, and PrivateLink carries traffic from that ENI to the target service over the backbone. Because it is a real network interface, it has a security group, it can be reached from on-premises over Direct Connect or VPN, and — with private DNS — the service’s normal public hostname resolves to the ENI so your existing SDK code “just works.”

The defining facts

Common interface-endpoint service names

You will reach for these constantly. Every name is com.amazonaws.<region>.<suffix>; the suffix is what changes.

Interface-endpoint limits & quotas

Settings you actually set

Private DNS — the piece that makes it seamless

Without private DNS, an interface endpoint only answers on endpoint-specific DNS names. With private DNS enabled, AWS creates a Route 53 private hosted zone so the default public name resolves to the ENI inside the VPC.

The practical rule: enable private DNS so existing code that calls ssm.<region>.amazonaws.com transparently uses the endpoint — but know that this is exactly where the nastiest conflicts live (two endpoints for one service; an S3 interface endpoint hijacking the S3 gateway path — covered in troubleshooting).

Create it — CLI and Terraform (SSM example)

aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0abc123 \
  --service-name com.amazonaws.ap-south-1.ssm \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-0priv-a subnet-0priv-b \
  --security-group-ids sg-0endpoint \
  --private-dns-enabled \
  --query "VpcEndpoint.{Id:VpcEndpointId,State:State}"

resource "aws_security_group" "endpoints" {
  name   = "vpce-ingress"
  vpc_id = aws_vpc.main.id
  ingress {                      # the gate for every interface endpoint
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main.cidr_block]
  }
  egress { from_port = 0, to_port = 0, protocol = "-1", cidr_blocks = ["0.0.0.0/0"] }
}

locals { ssm_services = ["ssm", "ssmmessages", "ec2messages"] }

resource "aws_vpc_endpoint" "ssm" {
  for_each            = toset(local.ssm_services)
  vpc_id              = aws_vpc.main.id
  service_name        = "com.amazonaws.ap-south-1.${each.key}"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = [aws_subnet.private_a.id, aws_subnet.private_b.id]
  security_group_ids  = [aws_security_group.endpoints.id]
  private_dns_enabled = true
  tags                = { Name = "vpce-${each.key}" }
}

Interface vs gateway — the full comparison

This is the table the exam and the design review both come back to. Read it once, top to bottom.

Which endpoint for which service

Decision table

S3 — the one service with both endpoint types

S3 is the only service where you genuinely have to choose, and the choice has cost and DNS consequences. Read this before you ever create an S3 interface endpoint.

Endpoint policies

An endpoint policy is a resource-based policy attached to the endpoint (gateway or interface) that acts as an additional filter on what can pass through it. It does not grant permissions — the caller’s IAM policy and the resource’s own policy still apply — it only restricts. Effective access is the intersection of the identity policy, the resource policy, and the endpoint policy.

The powerful pattern is the two-way lock: an endpoint policy limits what the endpoint can reach, and a bucket policy on the resource requires that access arrive through the endpoint. Together they make a bucket reachable only from your VPC, only for your principals.

// Bucket policy: deny any access that did NOT come through our VPC endpoint
{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "DenyUnlessViaEndpoint",
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:*",
    "Resource": ["arn:aws:s3:::kloudvin-data", "arn:aws:s3:::kloudvin-data/*"],
    "Condition": { "StringNotEquals": { "aws:sourceVpce": "vpce-0a1b2c3d4e5f" } }
  }]
}

Expose your own service with PrivateLink

This is the capability that turns “VPC endpoints” from a cost/security tweak into an architecture pattern. You can publish your own service so that consumers — in another VPC, another account, even a customer’s account — reach it through an interface endpoint, privately, with no VPC peering, and with overlapping CIDRs allowed because traffic is one-way and NATed through the ENI.

The components

Provider vs consumer responsibilities

NLB & endpoint-service settings

Endpoint-service limits & quotas

Provider side — CLI and Terraform

# 1) Create the endpoint service in front of an existing internal NLB
aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns arn:aws:elasticloadbalancing:ap-south-1:111111111111:loadbalancer/net/kv-svc/abc \
  --acceptance-required \
  --query "ServiceConfiguration.{Name:ServiceName,State:ServiceState}"

# 2) Allow a specific consumer account/role to discover & connect
aws ec2 modify-vpc-endpoint-service-permissions \
  --service-id vpce-svc-0abc123 \
  --add-allowed-principals arn:aws:iam::222222222222:root

resource "aws_vpc_endpoint_service" "kv" {
  acceptance_required        = true
  network_load_balancer_arns = [aws_lb.kv_nlb.arn]
  allowed_principals         = ["arn:aws:iam::222222222222:root"]
  private_dns_name           = "api.kloudvin.internal"   # optional
  tags                       = { Name = "kv-endpoint-service" }
}

Consumer side — CLI and Terraform

# Consumer (account 222…) creates an interface endpoint to the provider's service
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-0consumer \
  --service-name com.amazonaws.vpce.ap-south-1.vpce-svc-0abc123 \
  --vpc-endpoint-type Interface \
  --subnet-ids subnet-0c-a subnet-0c-b \
  --security-group-ids sg-0consumer-egress \
  --query "VpcEndpoint.{Id:VpcEndpointId,State:State}"
# State returns 'pendingAcceptance' until the provider accepts

resource "aws_vpc_endpoint" "consume_kv" {
  vpc_id              = aws_vpc.consumer.id
  service_name        = "com.amazonaws.vpce.ap-south-1.vpce-svc-0abc123"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = [aws_subnet.c_a.id, aws_subnet.c_b.id]
  security_group_ids  = [aws_security_group.consumer.id]
  private_dns_enabled = false   # only true if the provider verified a custom private DNS name
}

Then the provider accepts: aws ec2 accept-vpc-endpoint-connections --service-id vpce-svc-0abc123 --vpc-endpoint-ids vpce-0consumer-ep. The connection flips from pendingAcceptance to available, and the consumer’s app can now reach the provider’s service on the endpoint’s private IP/DNS — no peering, no route to the provider VPC, overlapping CIDRs irrelevant.

Cross-region & hybrid considerations

Endpoints are regional, and the two types behave differently across Region and network boundaries. Get these straight before you design a multi-Region or hybrid estate.

The centralization pattern is the money-saver at scale: instead of paying for ssm/ecr/kms/logs interface endpoints in every VPC, you run one set in a shared-services VPC behind Transit Gateway and share the Route 53 private hosted zone across the org — turning N×(per-VPC endpoints) into one set. See Transit Gateway Hub-and-Spoke Hands-On for the routing.

Architecture at a glance

The diagram traces both endpoint paths from a single private subnet, left to right. A private EC2 in the consumer VPC has no public IP and no NAT. For AWS-service traffic it uses VPC endpoints: an interface endpoint (an ENI with a private IP, powered by PrivateLink) whose private DNS makes ssm.<region>.amazonaws.com resolve to that ENI — but only if the security group on the ENI allows 443. That interface endpoint reaches two kinds of target: AWS services like Systems Manager (so the box is managed with no NAT), and your own service published as a VPC endpoint service behind an internal NLB, gated by an allow-list and per-connection acceptance. Separately, a gateway endpoint — just a route to a managed prefix list, free — carries S3 and DynamoDB traffic. Each numbered badge marks where a specific failure bites; the legend narrates every one as symptom, confirm, and fix.

Real-world scenario

KloudVin Health, a fictional but realistic health-tech company in Mumbai (ap-south-1), runs a claims-processing platform under an RBI/HIPAA-style mandate: no regulated data may traverse the public internet. Their VPC has three private application subnets (one per AZ) and, at first, a single NAT gateway for “AWS API calls.” The platform reads and writes ~14 TB/month of claim documents to S3, is managed with Systems Manager, pulls container images from ECR, and calls an internal fraud-scoring service that lives in a separate account run by their data-science team — whose VPC, embarrassingly, uses the same 10.0.0.0/16 CIDR as the app VPC.

The first architecture failed on three fronts. Cost: the 14 TB of S3 traffic through NAT cost roughly $630/month in NAT data processing alone, on top of NAT-hours. Security: an auditor pointed out that any compromised app host could reach any S3 bucket on the internet — the NAT path was a wide-open egress. And connectivity: they could not VPC-peer to the fraud-scoring account because the CIDRs overlapped, so they had bolted on a public HTTPS endpoint for the scoring API — exactly the internet path the mandate forbade.

The rebuild followed this article. They created a free S3 gateway endpoint and a DynamoDB gateway endpoint, associating both with all three private route tables, and attached an endpoint policy limiting Resource to KloudVin’s own buckets — killing both the NAT data-processing bill for S3 and the exfiltration path in one move. They added interface endpoints for ssm, ssmmessages, ec2messages (SSM with no NAT), ecr.api, ecr.dkr (plus the S3 gateway already carried image layers), kms, and logs, all sharing one security group that allows 443 from the VPC CIDR, with private DNS enabled so no application code changed. For the cross-account fraud-scoring call — the overlapping-CIDR problem — the data-science team published their service as a VPC endpoint service behind an internal NLB, added KloudVin Health’s account to AllowedPrincipals, and KloudVin created a consumer interface endpoint. PrivateLink’s one-way, NATed-through-the-ENI design made the overlapping 10.0.0.0/16 irrelevant — the very thing that made peering impossible.

The outcomes matched the design. The S3 NAT data-processing charge went to zero; the interface endpoints added about $45/month (seven services × ~3 AZ-hours + modest data) — a large net saving. The auditor got their proof: a bucket policy with aws:sourceVpce now denies any S3 access that did not arrive through the endpoint. And the fraud-scoring integration runs entirely on the AWS backbone with no internet hop and no peering. The one incident during rollout: the SSM interface endpoints were created against the default security group, which did not allow 443 from the app subnets, so instances showed “Connection lost” in Session Manager for an hour until Flow Logs revealed the ENI REJECTs — the single most common interface-endpoint mistake, and the reason badge #2 exists on the diagram.

Advantages and disadvantages

When each matters: for S3/DynamoDB the gateway endpoint is a no-brainer — free, more secure, less NAT. For other AWS services the interface endpoint’s cost is usually dwarfed by the NAT data charges it removes and the security it adds, provided you consolidate (centralize in a shared-services VPC at scale). For your own cross-account services, PrivateLink’s overlapping-CIDR tolerance and one-way isolation are often the only clean option. The disadvantages are almost entirely operational-knowledge items — the SG gate, the DNS conflicts, the DynamoDB asymmetry — which is exactly what the troubleshooting section drills.

Hands-on lab

You will build all three patterns in one VPC: (A) an S3 gateway endpoint, (B) the three interface endpoints that let a private instance use SSM with no NAT, and © a tiny PrivateLink service (NLB → endpoint service → consumer endpoint). Both aws CLI and Terraform paths are shown; a full teardown follows.

⚠️ Cost warning. Gateway endpoints and IAM are free. Interface endpoints and the NLB cost money while they exist (~$0.01/AZ-hour each + the NLB’s hourly + data). This lab’s interface endpoints and NLB run for well under an hour of testing — pennies — but you must do the teardown in Step 8 or they bill continuously. Use a sandbox account in ap-south-1 (adjust the region as needed).

Step 0 — Set variables and find your VPC/subnets.

REGION=ap-south-1
VPC=vpc-0abc123                       # your VPC id
SUB_A=subnet-0priv-a                  # a private subnet in AZ a
SUB_B=subnet-0priv-b                  # a private subnet in AZ b
RTB=rtb-0priv                         # the private subnets' route table
CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC --query "Vpcs[0].CidrBlock" --output text)
echo "VPC $VPC ($CIDR) in $REGION"
# Ensure DNS flags are on (required for private DNS):
aws ec2 modify-vpc-attribute --vpc-id $VPC --enable-dns-support
aws ec2 modify-vpc-attribute --vpc-id $VPC --enable-dns-hostnames

Step 1 — (A) Create the S3 gateway endpoint and wire it to the route table.

aws ec2 create-vpc-endpoint \
  --vpc-id $VPC \
  --service-name com.amazonaws.$REGION.s3 \
  --vpc-endpoint-type Gateway \
  --route-table-ids $RTB \
  --query "VpcEndpoint.VpcEndpointId" --output text

Expected: a vpce-… id. Verify the route: aws ec2 describe-route-tables --route-table-ids $RTB --query "RouteTables[].Routes[?DestinationPrefixListId!=null].[DestinationPrefixListId,GatewayId]" should show pl-… vpce-….

Step 2 — Create the security group the interface endpoints will share. This is the gate — it must allow 443 from the VPC CIDR.

SG=$(aws ec2 create-security-group --group-name vpce-ingress \
  --description "Allow 443 to interface endpoints" --vpc-id $VPC \
  --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $SG \
  --protocol tcp --port 443 --cidr $CIDR
echo "Endpoint SG: $SG"

Step 3 — (B) Create the three SSM interface endpoints with private DNS.

for SVC in ssm ssmmessages ec2messages; do
  aws ec2 create-vpc-endpoint --vpc-id $VPC \
    --service-name com.amazonaws.$REGION.$SVC \
    --vpc-endpoint-type Interface \
    --subnet-ids $SUB_A $SUB_B \
    --security-group-ids $SG \
    --private-dns-enabled \
    --query "VpcEndpoint.VpcEndpointId" --output text
done

Expected: three vpce-… ids, each moving from pending to available within a minute or two (aws ec2 describe-vpc-endpoints --query "VpcEndpoints[].[ServiceName,State]" --output table).

Step 4 — Verify SSM works with no NAT. Launch a private instance (no public IP, in $SUB_A) with an instance profile carrying AmazonSSMManagedInstanceCore, then:

aws ssm describe-instance-information \
  --query "InstanceInformationList[].[InstanceId,PingStatus]" --output table
aws ssm start-session --target i-0yourinstance     # opens a shell — no NAT, no bastion

Expected: your instance shows Online, and Session Manager opens a shell. If it does not, jump to troubleshooting row 3 (the SG) — the #1 cause.

Step 5 — © Provider side: NLB + endpoint service. Create an internal NLB with a listener on 443 and a target group with at least one healthy target (a tiny instance or IP running a TLS listener), then publish the service:

NLB_ARN=arn:aws:elasticloadbalancing:$REGION:111111111111:loadbalancer/net/kv-demo/abc
SVC_ID=$(aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns $NLB_ARN --acceptance-required \
  --query "ServiceConfiguration.ServiceId" --output text)
# Allow this same account (single-account lab) to consume it:
ACCT=$(aws sts get-caller-identity --query Account --output text)
aws ec2 modify-vpc-endpoint-service-permissions --service-id $SVC_ID \
  --add-allowed-principals arn:aws:iam::$ACCT:root
SVC_NAME=$(aws ec2 describe-vpc-endpoint-service-configurations \
  --service-ids $SVC_ID --query "ServiceConfigurations[0].ServiceName" --output text)
echo "Service name: $SVC_NAME"

Step 6 — Consumer side: interface endpoint to your own service, then accept.

EP=$(aws ec2 create-vpc-endpoint --vpc-id $VPC \
  --service-name $SVC_NAME --vpc-endpoint-type Interface \
  --subnet-ids $SUB_A $SUB_B --security-group-ids $SG \
  --query "VpcEndpoint.VpcEndpointId" --output text)
# It starts pendingAcceptance; the provider accepts:
aws ec2 accept-vpc-endpoint-connections --service-id $SVC_ID --vpc-endpoint-ids $EP
aws ec2 describe-vpc-endpoints --vpc-endpoint-ids $EP \
  --query "VpcEndpoints[0].State" --output text     # -> available

Expected: the endpoint transitions to available; the consumer can now hit the service on the endpoint’s private DNS name (aws ec2 describe-vpc-endpoints --vpc-endpoint-ids $EP --query "VpcEndpoints[0].DnsEntries[0].DnsName").

Step 7 — (Optional) Terraform equivalent. The snippets in the gateway, interface, and endpoint-service sections above compose into one main.tf: an aws_vpc_endpoint (Gateway) for S3, a for_each set of interface endpoints for SSM, and an aws_vpc_endpoint_service + consumer aws_vpc_endpoint. terraform init && terraform apply, then terraform destroy for teardown.

Step 8 — Teardown (do this — interface endpoints and the NLB bill hourly).

# Delete endpoints (interface + gateway + consumer)
aws ec2 describe-vpc-endpoints --filters Name=vpc-id,Values=$VPC \
  --query "VpcEndpoints[].VpcEndpointId" --output text | tr '\t' '\n' | \
  xargs -I{} aws ec2 delete-vpc-endpoints --vpc-endpoint-ids {}
# Delete the endpoint service, then the NLB + target group (console or CLI)
aws ec2 delete-vpc-endpoint-service-configurations --service-ids $SVC_ID
# Delete the SG (after its ENIs are gone) and terminate the test instance
aws ec2 delete-security-group --group-id $SG

⚠️ Confirm no vpce-… interface endpoints and no test NLB remain — those are the only line items that keep billing. The S3/DynamoDB gateway endpoints are free but delete them too for a clean slate.

Common mistakes & troubleshooting

This is the section you will return to. Endpoint failures are almost always one of the rows below — match the symptom, run the confirm step, apply the fix. The golden split: gateway problems are route/prefix-list/policy; interface problems are SG/DNS/acceptance.

Error / status reference

The three nastiest failures, in prose

The security group that isn’t there. You create an SSM interface endpoint, it reports available, and every start-session hangs. Everyone stares at IAM. The problem is that an interface endpoint has a security group on its ENI, and if you did not specify one it inherited the VPC default SG — which typically does not allow inbound 443 from your app subnets. The endpoint is “up” but the door is bolted. Confirm with VPC Flow Logs on the endpoint’s ENI (you will see REJECT on 443) or by reading the endpoint SG; fix by allowing TCP 443 inbound from the VPC CIDR. This one failure accounts for the majority of “my interface endpoint doesn’t work” tickets.

The S3 gateway-vs-interface DNS collision. You have a happy, free S3 gateway endpoint. Someone adds an S3 interface endpoint (for on-prem access) and enables private DNS on it. Now s3.<region>.amazonaws.com resolves, VPC-wide, to the interface ENI — so all your in-VPC S3 traffic silently moves off the free gateway path onto the billed interface endpoint, and your S3 data-processing charges appear from nowhere. Confirm with dig s3.<region>.amazonaws.com from an instance (it returns the ENI’s private IP, not a public one). Fix by disabling private DNS on the S3 interface endpoint and using its endpoint-specific/regional DNS name only for the on-prem clients that need it — keep the gateway endpoint as the default in-VPC path. This is the single most expensive endpoint mistake, and why S3 is the only service where you must think hard about running both types.

The connection stuck in pendingAcceptance. A consumer creates an interface endpoint to your published service and it never becomes available. Two independent gates both have to open: the consumer’s principal must be on the provider’s AllowedPrincipals list, and the provider must accept the specific connection (unless AcceptanceRequired=false). Confirm on the provider side with describe-vpc-endpoint-connections (you will see pendingAcceptance) and describe-vpc-endpoint-service-permissions (to check the allow-list). Fix by adding the consumer ARN to the allow-list and accepting the connection — or wire an SNS topic on connection events to a Lambda that auto-accepts trusted principals so onboarding is hands-off.

Best practices

• Always use a free gateway endpoint for S3 and DynamoDB in any VPC that touches them — it is cheaper and more secure than NAT, with no downside.
• Tighten every endpoint policy. The default is wide-open; restrict Resource to your buckets/tables to turn the endpoint into an exfiltration control.
• Pair endpoint policies with aws:sourceVpce deny statements on bucket/key policies so a resource is provably reachable only through your endpoint.
• Give interface endpoints a dedicated security group that allows 443 from the VPC CIDR — never rely on the default SG (the #1 silent failure).
• Put an interface-endpoint ENI in every AZ your workload runs in; a single-AZ endpoint is a single point of failure.
• Enable private DNS for AWS-service interface endpoints so application code is unchanged — but keep one private-DNS endpoint per service to avoid conflicts.
• Never enable private DNS on an S3 interface endpoint if a free S3 gateway endpoint exists in the same VPC, or you will silently reroute (and pay for) all in-VPC S3 traffic.
• Consolidate interface endpoints in a shared-services VPC behind Transit Gateway at scale, sharing the private hosted zone — one set instead of N×per-VPC.
• Front your own PrivateLink service with an internal NLB, enable cross-zone load balancing, and mind the fixed 350 s idle timeout for long-lived connections.
• Keep AcceptanceRequired=true for endpoint services and auto-accept trusted principals via SNS→Lambda; scope AllowedPrincipals to exact account/role ARNs, not *.
• Verify custom private DNS names via the TXT record so consumers can use a friendly hostname.
• Remember the asymmetries: DynamoDB is gateway-only; gateway endpoints are unreachable from on-prem/peered/TGW; endpoints are regional.

Security notes

VPC endpoints are a security control, so configure them like one. Least privilege on the endpoint policy: the default full-access policy should almost always be narrowed — restrict Resource to your own bucket/table ARNs and Principal/Condition to your Organization (aws:PrincipalOrgID), so the endpoint cannot be used to reach an attacker’s S3. Resource-side enforcement: the strongest pattern is a two-way lock — an endpoint policy limiting what the endpoint reaches, plus a bucket/key policy with a Deny when aws:sourceVpce (or aws:sourceVpc) does not match, which makes the resource reachable only through the endpoint and gives auditors provable network isolation. Network isolation: the security group on an interface endpoint’s ENI is your L4 control — allow only 443 from the specific subnets/SGs that need the service, not 0.0.0.0/0; and because the endpoint keeps traffic on the AWS backbone, you can operate genuinely private subnets with no Internet Gateway at all. Identity for endpoint services: scope AllowedPrincipals to exact ARNs (a security account, a specific role — see IAM Cross-Account Roles & AssumeRole), keep acceptance required, and treat the allow-list plus acceptance as the two locks that gate who may even connect. Encryption in transit still matters — PrivateLink keeps traffic off the internet but you should terminate TLS on your NLB/targets for your own services. Auditing: endpoint creation, policy changes, and connection accept/reject are all CloudTrail events; centralize them (see CloudTrail, Config & Audit for Compliance) and alert on new endpoint services or allow-list changes, which are high-signal for data-movement risk.

Cost & sizing

The whole topic has a split personality on cost: gateway endpoints are free and save money; interface endpoints and endpoint services cost money that you weigh against the NAT and security they replace.

Rough INR figures (Mumbai ap-south-1, approximate): one interface endpoint across 3 AZs is roughly ₹1,800/month in AZ-hours before data; seven such services (a typical SSM+ECR+KMS+logs set) run ~₹8,000–12,000/month if run per-VPC — which is precisely why centralizing them behind Transit Gateway is the standard cost optimization. Against that, a free S3/DynamoDB gateway endpoint can erase a NAT data-processing bill of tens of thousands of rupees a month for data-heavy workloads. Sizing guidance: create interface endpoints only for the services your private workloads actually call (check VPC Flow Logs / CloudTrail source IPs), use one shared SG, put an ENI in each AZ you use (no more), and consolidate across VPCs at scale. For endpoint services, right-size the NLB and turn on cross-zone LB so you are not paying for idle AZs.

Interview & exam questions

1. What are the two types of VPC endpoint and how do they differ structurally? A gateway endpoint is a route-table entry pointing a service’s managed prefix list at the endpoint; it supports S3 and DynamoDB only, is free, and has no ENI or security group. An interface endpoint is an ENI with a private IP (powered by PrivateLink) in each chosen AZ; it supports most AWS services plus partner and your own services, is billed per AZ-hour and per GB, and is guarded by a security group. (SAA-C03, ANS-C01)

2. Which services support gateway endpoints, and what does an interface endpoint cost? Gateway endpoints support only S3 and DynamoDB and are free. Interface endpoints cost roughly $0.01 per AZ-hour per endpoint plus ~$0.01/GB processed, region-dependent. (SAA-C03)

3. A private instance can’t be managed by SSM and has no NAT. What do you add? Three interface endpoints — ssm, ssmmessages, and ec2messages — with private DNS enabled and a security group allowing 443 from the VPC, plus an instance profile with AmazonSSMManagedInstanceCore. (SOA-C02, SAA-C03)

4. Your S3 gateway endpoint exists but traffic still goes via NAT. Why? The endpoint is not associated with the subnet’s route table, so no pl-… → vpce-… route exists. Add the route table with modify-vpc-endpoint --add-route-table-ids. (SAA-C03)

5. What is AWS PrivateLink and what problem does it uniquely solve? PrivateLink connects a consumer to a provider service over the AWS backbone via an interface-endpoint ENI and a provider NLB, with no VPC peering and overlapping CIDRs allowed (traffic is one-way and NATed through the ENI). It uniquely enables private, cross-account service access when CIDRs overlap or when you must not peer. (ANS-C01, SAA-C03)

6. How do you expose your own service via PrivateLink? Front it with an internal NLB, create a VPC endpoint service pointing at the NLB, add consumer ARNs to AllowedPrincipals, and (by default) accept each consumer connection; the consumer creates an interface endpoint to your vpce-svc-… name. (ANS-C01)

7. What is pendingAcceptance and how do you resolve it? A consumer’s endpoint connection to your endpoint service is waiting because AcceptanceRequired=true. Add the consumer to AllowedPrincipals and run accept-vpc-endpoint-connections (or auto-accept via SNS→Lambda). (ANS-C01)

8. Why might an interface endpoint be available yet every request time out? The security group on the endpoint ENI does not allow inbound 443 from the client. Confirm with VPC Flow Logs (REJECT) and fix the SG. (SAA-C03, ANS-C01)

9. You add an S3 interface endpoint and S3 costs rise. What happened? Enabling private DNS on the S3 interface endpoint made s3.<region>.amazonaws.com resolve to the ENI VPC-wide, rerouting in-VPC S3 traffic off the free gateway endpoint onto the billed interface endpoint. Disable private DNS on the interface EP. (ANS-C01)

10. Can a gateway endpoint be reached from on-premises over Direct Connect? No. A gateway endpoint is a route in the VPC route table and is not reachable from on-prem, peered VPCs, TGW, or VPN. For private S3 from on-prem, use an interface endpoint. (ANS-C01)

11. How do you make a bucket reachable only through a VPC endpoint? Add a bucket policy Deny when aws:sourceVpce does not equal your endpoint id (optionally aws:sourceVpc), paired with a scoped endpoint policy. This gives provable network isolation. (SCS-C02)

12. Does DynamoDB have an interface endpoint? No — DynamoDB supports gateway endpoints only. If you need DynamoDB-style access from on-prem you must front it yourself (e.g. a proxy/service), because the gateway type can’t serve on-prem. (SAA-C03, ANS-C01)

Quick check

1. Which endpoint type is free, and which two services does it support?
2. Your interface endpoint says available but calls hang. What is the single most likely cause?
3. You need private S3 access from on-premises. Gateway or interface endpoint — and why?
4. Two VPCs in different accounts have the same CIDR and one must call the other’s service privately. What do you use, and why not peering?
5. A consumer’s endpoint to your published service is stuck in pendingAcceptance. Name the two gates you must open.

Answers

1. The gateway endpoint is free; it supports S3 and DynamoDB only.
2. The security group on the endpoint ENI doesn’t allow inbound 443 from the client — the #1 interface-endpoint failure.
3. An interface endpoint — a gateway endpoint is a VPC route-table construct and is not reachable from on-premises; the interface ENI’s private IP is.
4. PrivateLink (an endpoint service consumed by an interface endpoint). Peering fails because it requires non-overlapping CIDRs; PrivateLink is one-way and NATs through the ENI, so overlap is irrelevant.
5. The consumer principal must be on the provider’s AllowedPrincipals list, and the provider must accept the specific connection (unless acceptance is disabled).

Glossary

Next steps

• Master the network it lives in: AWS VPC, Subnets & Security Groups Explained — the subnets, route tables, and SGs every endpoint depends on.
• See what endpoints replace: NAT Gateway & Private-Subnet Egress Hands-On — the cost and egress path a gateway endpoint erases for S3/DynamoDB.
• Know when peering is the wrong tool: VPC Peering: Setup, Routing & Limits — the overlapping-CIDR limit that PrivateLink sidesteps.
• Scale endpoints across many VPCs: Transit Gateway Hub-and-Spoke Hands-On — where centralized interface endpoints live behind a hub.
• Gate who may connect: IAM Cross-Account Roles & AssumeRole — the ARNs you put in AllowedPrincipals and endpoint policies.