Dude, Where's my Code? - Using terraform-docs to Automate Cloud Documentation

All of us have fallen victim to a lack of documentation. Whether it’s a mishap at work, or a 3AM pet project, we all get mad at ourselves when the inevitable happens. With infrastructure, we have nmap and various open source projects. For IaC though, it can seem tough to figure out. We’re not fully there yet, but terraform-docs is brilliant for creating a quick rundown of what may be very complex code.

Installation #

Technically, you don’t even need Terraform installed, as it’s a separate package. Here’s install commands for Mac, Windows and Ubuntu respectively:

#If you don't already have brew installed, run this first
/bin/bash -c "$(curl xs-fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
#If you do, just run this:
brew install terraform-docs


# If you don't already have Scoop installed, run this first:
> Set-ExecutionPolicy RemoteSigned -Scope CurrentUser # Optional: Needed to run a remote script the first time
> irm get.scoop.sh | iex


scoop bucket add terraform-docs https://github.com/terraform-docs/scoop-bucket
scoop install terraform-docs


curl -sSLo ./terraform-docs.tar.gz https://terraform-docs.io/dl/v0.16.0/terraform-docs-v0.16.0-$(uname)-amd64.tar.gz
tar -xzf terraform-docs.tar.gz
chmod +x terraform-docs
mv terraform-docs /some-dir-in-your-PATH/terraform-docs

source <(terraform-docs completion bash)

Running terraform-docs #

If you’ve got your environment setup correctly, you should be able to just go to your terraform code directory and run:

terraform-docs yaml --output-file terraform-docs.yml

and voila! You’ll have an output looking like this:

<!-- BEGIN_TF_DOCS -->
header: ""
footer: ""
inputs:
  - name: architecture
    type: string
    description: Select the Architecture to Use. Valid options are x86_64 and arm64
    default: null
    required: true
  - name: config_location
    type: string
    description: Enter the directory where you store your AWS config and credentials file with no trailling slash
    default: null
    required: true
    ...
<!-- END_TF_DOCS -->
    ...

Now… when you first run this, unless you’ve been following full DRY principals, you’ll end up seeing a few lack of descriptions and start to question things. This is good! Implementing good documentation makes you ask, and hopefully answer, these kind of questions. Take part of the output for an old network module I made for example:

outputs:
  - name: security_group
    description: null
  - name: subnet_id
    description: null
  - name: vpc_id
    description: null

Luckily, these are obvious enough, but what about the times where I have to create more than one security group, VPC or subnet? This is a relatively trivial example, but you can quickly run into these kind of naming issues for large modules.

You can go one step further and add –recursive to the end of the above command if you’re running lots of modules, and you’ll get a separate doc for each module you run.

But One Command is Too Much Effort! #

I may or may not have said this to myself, thinking that doing this before every commit was going to be time consuming. I would be wrong, but it definitely can lead to human error. That’s why I put this into my terraform pipeline!

I also combined this with an infracost report to give me a cost breakdown of my deployed infra so my docs end up looking quite fancy. Check out the github action I setup (Partially redacted for brevity):

name: 'Terraform'
on:
  push:
    branches:
      - prod
jobs:
    - name: Setup Infracost
      uses: infracost/actions/setup@v2
      with:
        api-key: ${{ secrets.INFRACOST_API_KEY}}
    - name: Generate Infracost cost estimate baseline
      run: |
        infracost breakdown --path=./ \
                            --format=table \
                             --out-file=./infracost-report.yml
        infracost breakdown --path=./ \
                            --format=json \
                             --out-file=./infracost-report.json        
    
    - name: Generates infracost diff
      run: |
        infracost diff --path=./ \
                        --format=diff \
                        --compare-to=./infracost-report.json \
                        --out-file=./diff.yml        
    - name: git config user & git push infracost
      run: |
       git config --global user.name "Liam Hardman"
       git config --global user.email "[email protected]"
       git add ./infracost-report.yml
       git add ./infracost-report.json
       git add ./diff.yml
       git commit -m "adding infracost report"
       git push origin ${{ steps.extract_branch.outputs.branch }}       
    - name: Generate new documentation
      uses: terraform-docs/[email protected]
      with:
        working-dir: .
        output-file: README.md
        output-method: inject
        git-push: "true"

Quite cool, huh? Indeed, having this combo of documentation is pretty cool. It’s only possible by also using a template file. Here’s how mine’s configured:

formatter: markdown

output:
  file: README.md
  mode: inject


output-values:
  enabled: false
  from: ""

sort:
  enabled: true
  by: name

settings:
  anchor: true
  color: true
  default: true
  description: false
  escape: true
  hide-empty: false
  html: true
  indent: 2
  lockfile: true
  read-comments: true
  required: true
  sensitive: true
  type: true

content: |-
  {{ .Header }}
    Welcome to my private repo! This has been setup to provide version control & documentation for the solution powering https://liamhardman.cloud :
    The terraform folder will contain the deployment config.
    ## Useful commands:
    terrascan scan
    rover
    
   ## Cost Breakdown
   
    {{ include "./infracost-report.yml" }}
   
   ## Cost Diff for Last Run
   
     {{ include "./diff.yml" }}
   
  {{ .Requirements }}    
  {{ .Providers }}
  {{ .Resources }}
  {{ .Inputs }}
  {{ .Outputs }}
  {{ .Footer }}

So… what have I learned from this? Well… I like to document, but as much as I do it, we all need to do it a lot more. I’ve already identified a decent few places I can improve. Combining that with the knowledge from my [diagrams as code](https://liamhardman.cloud/automation/diagramsascode/ article, I should be on a good road to improving how I document even more.