External proxy Network Load Balancer overview

This document introduces the concepts that you need to understand to configure a Google Cloud external proxy Network Load Balancer.

The external proxy Network Load Balancer is a reverse proxy load balancer that distributes TCP traffic coming from the internet to virtual machine (VM) instances in your Google Cloud Virtual Private Cloud (VPC) network. When using an external proxy Network Load Balancer, incoming TCP or SSL traffic is terminated at the load balancer. A new connection then forwards traffic to the closest available backend by using either TCP or SSL (recommended). For more use cases, see Proxy Network Load Balancer overview.

External proxy Network Load Balancers let you use a single IP address for all users worldwide. The load balancer automatically routes traffic to the backends that are closest to the user.

In this example, SSL traffic from users in City A and City B is terminated at the load balancing layer, and a separate connection is established to the selected backend.

Modes of operation

You can configure an external proxy Network Load Balancer in the following modes:

• A classic proxy Network Load Balancer is implemented on globally distributed Google Front Ends (GFEs). This load balancer can be configured to handle either TCP or SSL traffic by using either a target TCP proxy or a target SSL proxy respectively. With the Premium Tier, this load balancer can be configured as a global load balancing service. With Standard Tier, this load balancer is configured as a regional load balancing service. Classic proxy Network Load Balancers can also be used for other protocols that use SSL, such as WebSockets and IMAP over SSL.
• A global external proxy Network Load Balancer is implemented on globally distributed GFEs and supports advanced traffic management capabilities. This load balancer can be configured to handle either TCP or SSL traffic by using either a target TCP proxy or a target SSL proxy respectively. This load balancer is configured as a global load balancing service with the Premium Tier. Global external proxy Network Load Balancers can also be used for other protocols that use SSL, such as WebSockets and IMAP over SSL.
• A regional external proxy Network Load Balancer is implemented on the open source Envoy proxy software stack. It can handle only TCP traffic. This load balancer is configured as a regional load balancing service that can use either Premium or Standard Tier.

Identify the mode

To determine the mode of a load balancer, complete the following steps.

Architecture

The following diagrams show the components of external proxy Network Load Balancers.

The following are components of external proxy Network Load Balancers.

Proxy-only subnet

The proxy-only subnet provides a set of IP addresses that Google uses to run Envoy proxies on your behalf. You must create one proxy-only subnet in each region of a VPC network where you use load balancers. The --purpose flag for this proxy-only subnet is set to REGIONAL_MANAGED_PROXY. All regional Envoy-based load balancers in the same region and VPC network share a pool of Envoy proxies from the same proxy-only subnet.

Backend VMs or endpoints of all load balancers in a region and a VPC network receive connections from the proxy-only subnet.

Points to remember:

• Proxy-only subnets are only used for Envoy proxies, not your backends.
• The IP address of the load balancer is not located in the proxy-only subnet. The load balancer's IP address is defined by its external managed forwarding rule.
• Proxy-only subnets must be configured with a stack type of IPV4_IPV6 to terminate incoming IPv6 traffic and IPV4_ONLY or IPV4_IPV6 to terminate incoming IPv4 traffic.

Forwarding rules and IP addresses

Forwarding rules route traffic by IP address, port, and protocol to a load balancing configuration that consists of a target proxy and a backend service.

IP address specification. Each forwarding rule references a single IP address that you can use in DNS records for your application. You can either reserve a static IP address that you can use or let Cloud Load Balancing assign one for you. We recommend that you reserve a static IP address. Otherwise, you must update your DNS record with the newly assigned ephemeral IP address whenever you delete a forwarding rule and create a new one.

Port specification. External forwarding rules used in the definition of this load balancer can reference exactly one port from 1-65535. If you want to support multiple consecutive ports, you need to configure multiple forwarding rules. Multiple forwarding rules can be configured with the same virtual IP address and different ports; therefore, you can proxy multiple applications with separate custom ports to the same TCP proxy virtual IP address. For more details, see Port specifications for forwarding rules.

To support multiple consecutive ports, you have to configure multiple forwarding rules. Multiple forwarding rules can be configured with the same virtual IP address and different ports. Therefore, you can proxy multiple applications with separate custom ports to the same TCP proxy virtual IP address.

The following table shows the forwarding rule requirements for external proxy Network Load Balancers.

Load balancer mode	Network Service Tier	Forwarding rule, IP address, and load balancing scheme	Routing from the internet to the load balancer frontend
Classic proxy Network Load Balancer	Premium Tier	Global external forwarding rule Global external IP address Load balancing scheme: EXTERNAL	Requests routed to the GFEs that are closest to the client on the internet.
	Standard Tier	Regional external forwarding rule Regional external IP address Load balancing scheme: EXTERNAL	Requests routed to a GFE in the load balancer's region.
Global external proxy Network Load Balancer	Premium Tier	Global external forwarding rule Global external IP address Load balancing scheme: EXTERNAL_MANAGED	Requests routed to the GFEs that are closest to the client on the internet.
Regional external proxy Network Load Balancer	Premiumand Standard Tier	Regional external forwarding rule Regional external IP address Load balancing scheme: EXTERNAL_MANAGED	Requests routed to the Envoy proxies in the same region as the load balancer.

Forwarding rules and VPC networks

This section describes how forwarding rules used by external Application Load Balancers are associated with VPC networks.

Load balancer mode	VPC network association
Global external proxy Network Load Balancer Classic proxy Network Load Balancer	No associated VPC network. The forwarding rule always uses an IP address that is outside the VPC network. Therefore, the forwarding rule has no associated VPC network.
Regional external proxy Network Load Balancer	The forwarding rule's VPC network is the network where the proxy-only subnet has been created. You specify the network when you create the forwarding rule. Depending on whether you use an IPv4 address or an IPv6 address range, there is always an explicit or implicit VPC network associated with the forwarding rule. Regional external IPv4 addresses always exist outside of VPC networks. However, when you create the forwarding rule, you're required to specify the VPC network where the proxy-only subnet has been created. Hence, the forwarding rule has an explicit network association. Regional external IPv6 address ranges always exist inside a VPC network. When you create the forwarding rule, you're required to specify the subnet from which the IP address range is taken. This subnet must be in the same region and VPC network where a proxy-only subnet has been created. Thus, there is an implied network association. Network and subnet requirements. For IPv6 traffic, your network and subnets must meet the following configuration requirements: VPC network: you must use a custom mode VPC network configured with the --enable-ula-internal-ipv6 flag. Forwarding rule subnet: this subnet must be a dual-stack (IPv4_IPv6) or IPV6_ONLY subnet with the ipv6-access-type set to EXTERNAL. IPv6 address allocation options. The forwarding rule must reference a /96 range of IPv6 addresses from the subnet's /64 external IPv6 address range. You can assign this /96 IPv6 address range using one of the following methods: Specifying a reserved external IPv6 address. Specifying a custom ephemeral IPv6 address. Letting Google Cloud automatically assign an ephemeral IPv6 address. Limitations. To specify a custom ephemeral IPv6 address, you must use the Google Cloud CLI or the API. The Google Cloud console doesn't support specifying custom ephemeral IPv6 addresses for forwarding rules.

Target proxies

External proxy Network Load Balancers terminate connections from the client and create new connections to the backends. The target proxy routes these new connections to the backend service.

Depending on the type of traffic your application needs to handle, you can configure an external proxy Network Load Balancer with either a target TCP proxy or a target SSL proxy.

• Target TCP proxy: Configure the load balancer with a target TCP proxy if you're expecting TCP traffic.
• Target SSL proxy: Configure the load balancer with a target SSL proxy if you're expecting encrypted client traffic. This type of load balancer is intended for non-HTTP(S) traffic only. For HTTP(S) traffic, we recommend that you use an external Application Load Balancer.

By default, the target proxy does not preserve the original client IP address and port information. You can preserve this information by enabling the PROXY protocol on the target proxy.

The following table shows the target proxy requirements for external proxy Network Load Balancers.

Load balancer mode	Network Service Tier	Target proxy	Reference
Classic proxy Network Load Balancer	Premium Tier	targetTcpProxies or targetSslProxies	Target proxy references a single backend service.
	Standard Tier	targetTcpProxies or targetSslProxies
Global external proxy Network Load Balancer	Premium Tier	targetTcpProxies or targetSslProxies	Target proxy references a single backend service.
Regional external proxy Network Load Balancer	Premium and Standard Tier	regionTargetTcpProxies	Target proxy references either a single backend service or one or more TLS routes.

SSL certificates

SSL certificates are only required if you're deploying a global external proxy Network Load Balancer and classic proxy Network Load Balancer with a target SSL proxy.

External proxy Network Load Balancers using target SSL proxies require private keys and SSL certificates as part of the load balancer configuration.

• Google Cloud provides two configuration methods for assigning private keys and SSL certificates to target SSL proxies: Compute Engine SSL certificates and Certificate Manager. For a description of each configuration, see Certificate configuration methods in the SSL certificates overview.

• Google Cloud provides two certificate types: Self-managed and Google-managed. For a description of each type, see Certificate types in the SSL certificates overview.

TLS routes

A TLS route resource lets you define how traffic is routed to backend services based on SNI hostnames.

If the load balancer's target TCP proxy is not referenced by a TLS route, then only the single default backend service is used, and the value of SNI hostname sent by the client or its absence isn't relevant.

TLS routes are only available for regional external proxy Network Load Balancers. They aren't supported for classic proxy Network Load Balancers and global external proxy Network Load Balancers.

You can attach a TLS route configuration to a load balancer's target proxy by using the gcloud network-services tls-routes commands.

The following table shows the TLS routes APIs required by external proxy Network Load Balancers:

Load balancer mode	TLS route	Reference
Regional external proxy Network Load Balancer	Regional tlsRoutes	Each TLS route can reference one or more backend services.

Backend services

Backend services direct incoming traffic to one or more attached backends. Each backend is composed of an instance group or network endpoint group and information about the backend's serving capacity. Backend serving capacity can be based on CPU or requests per second (RPS).

Each load balancer has at least one backend service resource. If you use TLS routes (Preview) to configure routing, you can configure multiple backend services for your load balancer. Without TLS routes, you're limited to only a single backend service per load balancer.

The backend service specifies the health check to be performed for the available backends.

Changes made to the backend service are not instantaneous. It can take several minutes for changes to propagate to GFEs. To ensure minimal interruptions to your users, you can enable connection draining on backend services. Such interruptions might happen when a backend is terminated, removed manually, or removed by an autoscaler. To learn more about using connection draining to minimize service interruptions, see Enabling connection draining.

For more information about the backend service resource, see Backend services overview.

The following table specifies the different backends supported on the backend service of external proxy Network Load Balancers.

Load balancer mode	Supported backends on a backend service
	Instance groups	Zonal NEGs	Internet NEGs	Serverless NEGs	Hybrid NEGs	Private Service Connect NEGs	GKE
Classic proxy Network Load Balancer		Use standalone zonal NEGs
Global external proxy Network Load Balancer	*	GCE_VM_IP_PORT type endpoints *
Regional external proxy Network Load Balancer		GCE_VM_IP_PORT type endpoints	Regional NEGs only			Add a Private Service Connect NEG

Backends and VPC networks

For global external proxy Network Load Balancer and classic proxy Network Load Balancer backends, all backend instances from instance group backends and all backend endpoints from NEG backends must be located in the same project. However, an instance group backend or a NEG can use a different VPC network in that project. The different VPC networks don't need to be connected using VPC Network Peering because GFEs communicate directly with backends in their respective VPC networks.

For regional external proxy Network Load Balancer backends, the following applies:

• For instance groups, zonal NEGs, and hybrid connectivity NEGs, all backends must be located in the same project and region as the backend service. However, a load balancer can reference a backend that uses a different VPC network in the same project as the backend service. Connectivity between the load balancer's VPC network and the backend VPC network can be configured using either VPC Network Peering, Cloud VPN tunnels, Cloud Interconnect VLAN attachments, or a Network Connectivity Center framework.

  Backend network definition

  • For zonal NEGs and hybrid NEGs, you explicitly specify the VPC network when you create the NEG.
  • For managed instance groups, the VPC network is defined in the instance template.
  • For unmanaged instance groups, the instance group's VPC network is set to match the VPC network of the nic0 interface for the first VM added to the instance group.

  Backend network requirements

  Your backend's network must satisfy one of the following network requirements:

  • The backend's VPC network must exactly match the forwarding rule's VPC network.

  • The backend's VPC network must be connected to the forwarding rule's VPC network using VPC Network Peering. You must configure subnet route exchanges to allow communication between the proxy-only subnet in the forwarding rule's VPC network and the subnets used by the backend instances or endpoints.

  Backend IPv6 subnet requirements

  The IP version used for the frontend connection is independent of the backend connection. Since the proxy-only subnet is dual-stack (IPV4_IPV6), the proxy-only subnet can communicate with backends using either IPv4 or IPv6.

  If your backend instances are handling IPv6 traffic, the backend subnet can be configured with a stack type of IPV4_ONLY or IPV4_IPV6 (dual-stack). If the backend subnet's stack type includes IPv6, you must explicitly set the subnet's ipv6-access-type to EXTERNAL.

• Both the backend's VPC network and the forwarding rule's VPC network must be VPC spokes attached to the same NCC hub. Import and export filters must allow communication between the proxy-only subnet in the forwarding rule's VPC network and the subnets used by backend instances or endpoints.

• For all other backend types, all backends must be located in the same VPC network and region.

Backends and network interfaces

If you use instance group backends, packets are always delivered to nic0. If you want to send packets to non-nic0 interfaces (either vNICs or Dynamic Network Interfaces), use NEG backends instead.

If you use zonal NEG backends, packets are sent to whatever network interface is represented by the endpoint in the NEG. The NEG endpoints must be in the same VPC network as the NEG's explicitly defined VPC network.

Protocol for communicating with the backends

When you configure a backend service for an external proxy Network Load Balancer, you set the protocol that the backend service uses to communicate with the backends.

• For classic proxy Network Load Balancers, you can choose either TCP or SSL.
• For global external proxy Network Load Balancers, you can choose either TCP or SSL.
• For regional external proxy Network Load Balancers, you can use TCP.

The load balancer uses only the protocol that you specify, and does not attempt to negotiate a connection with the other protocol.

Firewall rules

The following firewall rules are required:

• For classic proxy Network Load Balancers, an ingress allow firewall rule to permit traffic from GFEs to reach your backends.