From Chaos to Control: How to Import Existing Route53 Records into Terraform and Overcome various Challenges.
Before we begin to go deeper into this, I am assuming that you already have an understanding of the below pointers.
- Terraform
- AWS
- Python and Shell Script
Scenario/Problem:- So we were on a mission to make our AWS infra fully terraformed. As we were importing all the manually created resources to terraform files, I had the responsibility to make Route53 fully managed via terraform and also to delete non-used/stale route53 records and hosted zones. Till now we used to create Route53 Records manually.
So now the First step was to how can we identify if the route53 record is in use or not. To achieve this, We have DNS Query Logging. In short, This will create a log file with all the records that are being used or getting hit.
Once you configure query logging, Route 53 will send logs to CloudWatch Logs. Now if you have 2–3 hosted zones then it's fine to enable it via console but In my case, as I have 2 AWS environments with 59 and 20 hosted zones respectively, It would have been a tedious task to just enable them. So I wrote python automation for this, below is the code
After we have enabled query logging for all of our hosted zones, Now we should see logs in CloudWatch. But we have another challenge here because Route 53 creates one CloudWatch Logs log stream for each Route 53 edge location that responds to DNS queries for the specified hosted zone and sends query logs to the applicable log stream. The format for the name of each log stream is hosted-zone-id/edge-location-ID, for example, Z1D633PJN98FT9/DFW3. In reality, these log stream for a single hosted zone looks like the below screenshot.
The log file looks like
1.0 2017-12-13T08:16:02.130Z Z123412341234 example.com A NOERROR UDP Region 192.168.1.1 -
1.0 2017-12-13T08:15:50.235Z Z123412341234 example.com AAAA NOERROR TCP Region 192.168.3.1 192.168.222.0/24
1.0 2017-12-13T08:16:03.983Z Z123412341234 example.com ANY NOERROR UDP Region 2001:db8::1234 2001:db8:abcd::/48
1.0 2017-12-13T08:15:50.342Z Z123412341234 bad.example.com A NXDOMAIN UDP Region 192.168.3.1 192.168.111.0/24
1.0 2017-12-13T08:16:05.744Z Z123412341234 txt.example.com TXT NOERROR UDP Region 192.168.1.2 -The log files are present inside every log group that we have. Which makes it harder to consolidate all the log files into one. We can use CloudWatch log insights to query these log groups, but this has a limitation to query only 10k log events, so if my query has more than 10000 log events, there seems to be no way to get the complete data.
Now to solve this problem, I thought to write a python utility to fetch all log files and consolidate them, but this would have taken a lot of my time. So I started looking for other solutions and after a lot of effort, I found https://github.com/jorgebastida/awslogs
awslogs is a simple command line tool for querying groups, streams, and events from Amazon CloudWatch logs. Now using this I can get the logs output in a file. This saved me so much time and effort. This tool is a lifesaver.
Now I had a log file for every hosted zone, Next step was to compare the record present in the file with the record present in the hosted zone. For this, I wrote a python script for this
Now we can have a file with all the records that are not in use. We can add automation on top of the result to delete such records that are not present in the log file.
One more thing that we can do to clean up route53 records that we may have attached to ec2 instances IP or ELB’s endpoint that does not exist anymore. For this you can use the below code, this python code will check for all the route53 records which are attached to a private IP or elb’s endpoint if these IP or endpoint does not exist in our AWS account. it will delete them
Now we can say that we have cleaned our Route53 records by deleting unused records.
Our next step is to import all the records present in the hosted zone to terraform to make it easy we can use the below script that will create tf file automatically for all the Route53 records and its hosted zone and this will also run terraform import command to import each and every record. You just have to run terraform init command to initialize the directory where you want to create a tf file and run the below script. We don't have to do any manual work to import these.
Once you are done with all of these, You would also want to disable the DNS query logging that we enabled in the first step. I don't want you to disable it manually, Which is going to be hectic and time-consuming, So you can use my code given below to disable the logging from all the hosted zone.
In conclusion, importing existing Route53 records into Terraform can be a powerful way to gain more control and automation over your DNS management. However, the process can come with its fair share of challenges. From discrepancies in record values to issues with stale records, there are several obstacles that you may encounter along the way.
By sharing my experience and tips for success, I hope that this guide has provided you with a helpful roadmap for navigating the importation process. With the right approach, you can successfully import and manage your Route53 records with Terraform, streamlining your infrastructure and maximizing efficiency.
PS:- If you are trying to perform the above steps and you encounter any issues, feel free to comment here or connect with me over LinkedIn.
