Support Terraform 0.12 state format

[AndiDog]

Solves #113 by implementing basic support for the output of Terraform 0.12's terraform show -json. I introduced new JSON-to-struct mappings for 0.12, but kept all old code to remain compatible with old Terraform versions (99% of people probably still use 0.11 right now). Internally, new-style resources are mapped to the existing struct's so almost no code apart from the parser needed any change.

[adammck]

Wow, thanks for delivering this so quickly! I will try to review it this weekend. Please fix the build/tests? Positive/negative feedback from anyone else trying this out is very welcome.

[AndiDog]

[...] Please fix the build/tests? [...]

The build is missing the sort.Slice function which was introduced with Go 1.8 (release date 2017-02-16). I've added a commit to update the used Go versions.

[ctrlok]

Right now there is a problem with latest version

$ terraform-inventory --list ./
overwriting already existing individual key main.0, old: [18.234.233.11], new: 3.91.252.244"all":{
"hosts":["18.234.233.11","3.91.252.244"] ... "main":["18.234.233.11","3.91.252.244"
],"main.0":["3.91.252.244"]..

I have ec2_instance with count=2

Also I have a problem with parsing existing terraform state file

$ terraform-inventory --list ./terraform.tfstate
Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

Problem with parsing local tfstate is probably because of different formats of tfstate and terraform show --json

[ctrlok]

I'm not sure, but looks like we need to rewrite func NewResource(keyName string, state resourceState) (*Resource, error) function. Because in terraform show --json command resource name is the same for resources with the same name, but those resources have additional parameter index

          "address": "aws_instance.main",
          "mode": "managed",
          "type": "aws_instance",
          "name": "main",
          "index": 0,

[AndiDog]

Saw that coming since I tested simple count-less cases. I can check next week. Any sample snippets and more testing helps, thank you!!

[AndiDog]

Please try again now. The pushed patch works in my Terraform project for count=2.

[ctrlok]

Works fine, but I got this error message:

tags overwriting already existing output key main, old: [54.164.36.15 3.89.164.2], new: [3.89.164.2 54.164.36.15]

main is a name of resource. I.E. aws_ec2_instance.main

[AndiDog]

Works fine, but I got this error message:

tags overwriting already existing output key main, old: [54.164.36.15 3.89.164.2], new: [3.89.164.2 54.164.36.15]

main is a name of resource. I.E. aws_ec2_instance.main

Cannot reproduce that. Do you have a minimal example?

[wegerhoff]

The patch is working fine when working with terraform modules to group hosts. Unfortunately we are using vpshere-tags to group hosts. This was supported by terraform-inventory prior to 0.12.*. Is there a chance to get this feature back without me having to learn go first? :)

Current output of terraform-inventory -inventory:

[module_consul_vms_vm]
10.27.101.38
10.27.101.36
10.27.101.40

...

Expected additional output:

[vsphere_consul_tag]
10.27.101.38
10.27.101.36
10.27.101.40

[vsphere_solr_tag]
10.27.101.4
10.27.101.2
10.27.101.3

[AndiDog]

As said, please provide a full working example (input + expected output + if possible Terraform 0.12 state file) and it can be tested and fixed hopefully by covering 0.12 with more of the previously existing unit tests.

[mkjmdski]

Hey, what is the status of this PR? I'm really looking forward to use new dynamic inventory for terraform 0.12 ;)

[adammck]

It seems like some people are still having trouble with this PR, but it seems useful enough that we ought to merge and release it anyway, perhaps as a pre-release. You're done with this, @AndiDog?

[AndiDog]

I have been using the version from this PR for weeks without issue (on AWS). Once the above commenters can provide a reproducible case, they can create separate PRs, since the changes are simple to develop thanks to the unit tests. Please go ahead and merge. I agree that it's already useful now.

[ctrlok]

It works for me, but I still have this message even with inventory with a single host. I get this message every time, so I didn't understand how any of my state files can show anything.

[jury89]

Hi guys, i'm having the same problem with the new Terraform version. Any idea when this PR will be merged? 🙏

[astubbs]

PR works for me - thanks @AndiDog !

[dpisano]

I'm still getting the 0.12 format error: <nil>

$ terraform-inventory --list ./terraform.tfstate
Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

If you like I can share my .tfstate file.

[ctrlok]

@dpisano please, read the first message. That version of the terraform works with terraform show --json output

[dpisano]

@ctrlok sorry it looks like I was misunderstanding how how to run it. Running just terraform-inventory --list ./ works for me.

[dpisano]

As a node this only works when I use it with Ansible when I set TF_STATE=./ when I run the playbook. The full command is something like this: TF_STATE=./ ansible-playbook --inventory-file=/usr/local/bin/terraform-inventory ../site.yml

[AndiDog]

Can we get this merged and released? Is the PR waiting for anything?

[danihodovic]

https://github.com/nbering/terraform-inventory is similar to this project, but is compatible with both Terraform 0.12 and 0.11.

https://github.com/nbering/terraform-inventory/blob/master/CHANGELOG.md#200---2019-05-05

[adammck]

Nope, this is good to go, I just haven't had time to test it properly. Merging anyway, since it seems to work for plenty of people on this thread. Thanks, @AndiDog!

[adammck]

FYI this was just released as v0.9. Thanks for your patience!