Extending GCP Cloud DNS to On-prem with Multiple VPCs

Cloud DNS is a managed DNS service from Google Cloud and we will be working with Private Hosted Zone through this blog. In one of our recent engagements, we were migrating a mobile payments platform from on-prem to GCP.

This is a hybrid environment where back-office applications will remain on-prem for time being and all the core applications will get migrated to GKE. We are working with 3 environments i.e Dev, stage, and Production created as 3 projects on GCP with each one having its own VPC. The task at hand is to connect these 3 environments with on-prem in a secure way where service level communication should happen privately and with service names and not IP addresses.

We have used Cloud VPN (HA) for network-level communication from each project back to on-prem. Discovery of services by their names will be completed by use of Cloud DNS forwarders.

Credit: https://excalidraw.com/

Let’s get into it

What is already in place? You should check this list and have these details ready

  • Network communication between On-prem and GCP via Cloud VPN from all 3 projects
  • A local DNS server on-prem (which hosts the on-prem services). In our case, the DNS server used is DNS hosted on Windows server 2016.
  • List of on-prem hosted internal domains (For ex: we have umesh.local as the main domain on on-prem)
  • The IP address of DNS server on-prem
  • List of domains that you will host on GCP (For ex: we will create private hosted zone dev.umesh.gcp , stage.umesh.gcp and prod.umesh.gcp on Cloud DNS in respective projects)

What do we need to create on GCP?

  • Cloud DNS private hosted zone in each project (dev.umesh.gcp in dev, stage.umesh.gcp in stage and prod.umesh.gcp in production. Choose the appropriate VPC while configuring them). Note: I am going to use gcloud commands to create them, you can an option to do this from the console

gcloud beta dns — project=your-project-id managed-zones create dev-umesh-gcp — description=”private hosted zone for dev project” — dns-name=”dev.umesh.gcp.” — visibility=”private” — networks=”your-vpc”

Created private hosted zone in Dev project.

Repeat this step for the other 2 projects by changing the values for project id, name, DNS name, and network name.

  • Cloud DNS forwarding zone (Under Option: Forward queries to another server) for on-prem umesh.local domain. This has to be created in only one project(in our case, prod) and not all of the projects. We will be using DNS peeing from this project(Prod) to all other projects (Dev and Stage)

gcloud beta dns — project=your-project managed-zones create umesh-local-forward-zone — description=”forwarding zone for umesh.local dns” — dns-name=”umesh.local.” — visibility=”private” — networks=”your-network” — private-forwarding-targets=”10.100.0.1"

Created forwarding zone in Prod project

You can’t add records to this forwarding zone as the data comes from your on-prem DNS. Keep in mind that you can add multiple DNS server ips if the on-prem server(s) are hosted that way.

We would normally create this in all the projects but this is where we have some limitations (or designed this way) from GCP. Offical from Google: “queries from any VPC network have the same IP range 35.199.192.0/19 as the source. Therefore, responses can’t be routed correctly unless you have separate environments on-premises”. Read more

  • Cloud DNS peering zones. Since DNS peering unidirectionally forwards DNS requests, for our case, we have created a forwarding zone in the prod project, so we will have to create DNS peering connections from Dev and Stage projects VPC (called DNS consumer network) to Prod project VPC (DNS producer network). If you want all networks to discover anything and everything in the whole network setup, create a DNS Peering zone in all 3 projects so they all become consumer and producer.

In this example, we will go ahead and create 2 peering zone in dev and stage projects

gcloud beta dns — project=your-dev-project managed-zones create peering-with-prod-network — description=”DNS peering for umesh.local DNS with prod project” — dns-name=”umesh.local.” — visibility=”private” — networks=”your-dev-vpc” — target-project=your-prod-project — target-network=”your-prod-vpc”

Created peering zone in Dev project

Repeat this for stage project.

  • The next step is to create DNS policies in all 3 projects for your respective VPC. You can configure one DNS server policy for each VPC network. you have control over both inbound server policy and outbound server policy. In our use case, we are required to create an inbound server policy.

gcloud dns policies create prod-dns-policy — description=inbound dns server policy — networks=your-prod-project — enable-inbound-forwarding

you can read more about DNS policies here

At this stage, by creating private zones in each project, forwarding zone in one of the projects (prod), creating peering zone in dev and stage VPC, and then adding inbound DNS server policy, we have made sure that the GCP side of systems is well connected to route requests to appropriate on-prem DNS servers.

What do we require to configure on on-prem?

  • There are a couple of things we need to perform on the on-prem side. The first one is to add DNS forwarders IPs for all 3 GCP networks on 3 GCP projects. How do we get DNS server IPs for hosted Cloud DNS zones?

gcloud compute addresses list — filter=’purpose = “DNS_RESOLVER”’ — format=’csv(address, region, subnetwork)’

you run this command in each of the projects and it will list you one IP for each subnet in your VPC. Take note of all these IPs as you have to add this to the On-prem DNS server as DNS forwarder IP.

You can use this article to see how to add forwarders in windows server

  • The second one is to configure your on-premises routers and firewalls to permit traffic from Google’s 35.199.192.0/19 IP address range. Cloud DNS uses the 35.199.192.0/19 source range for all customers. This range is only accessible from a Google Cloud VPC network or from an on-premises network connected to a VPC network.

With this in place, you will be able to discover domains hosted on-prem (umesh.local) from GCPs in any private hosted zone (dev.umesh.gcp, stage.umesh.gcp and prod.umesh.gcp) and vice versa.

Helpful Resources

  1. hybrid-architecture-using-multiple-vpc-networks
  2. Troubleshooting
  3. Issues with DNS Peering

Thanks for reading this, hope it helped you in some way. Happy Networking on Google Cloud

Lead Cloud Architect at Searce | Cloud Engineering in all its forms

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store