Hi everybody! 🙂 Thank you for creating pants, it looks very promising! I am trying to get the experimental terraform backend up and running, but it keeps complaining that init has not been run to download the required providers. Running init manually did not fix it for me, and I did not find anything helpful in docs or issues except for an old issue that claims to have solved that (or something similar). How should I proceed? Thanks a lot and cheers for all your efforts! 🙂

It looks like the plugin never calls "init" as you suggest:

jsirois@Gill-Windows:~/dev/pantsbuild/jsirois-pants (main) $ git grep -A5 "TerraformProcess("
src/python/pants/backend/terraform/goals/check.py:            TerraformProcess(
src/python/pants/backend/terraform/goals/check.py-                args=("validate", directory),
src/python/pants/backend/terraform/goals/check.py-                input_digest=source_files.snapshot.digest,
src/python/pants/backend/terraform/goals/check.py-                output_files=tuple(files),
src/python/pants/backend/terraform/goals/check.py-                description=f"Run `terraform fmt` on {pluralize(len(files), 'file')}.",
src/python/pants/backend/terraform/goals/check.py-            ),
--
src/python/pants/backend/terraform/lint/tffmt/tffmt.py:        TerraformProcess(
src/python/pants/backend/terraform/lint/tffmt/tffmt.py-            args=("fmt", directory),
src/python/pants/backend/terraform/lint/tffmt/tffmt.py-            input_digest=request.snapshot.digest,
src/python/pants/backend/terraform/lint/tffmt/tffmt.py-            output_files=request.files,
src/python/pants/backend/terraform/lint/tffmt/tffmt.py-            description=f"Run `terraform fmt` on {pluralize(len(request.files), 'file')}.",
src/python/pants/backend/terraform/lint/tffmt/tffmt.py-        ),

So you'd want to start by filing an issue with your Pants version, full command line and output. If you're interested in contributing a fix we can use that ticket as the rally point and you can ask questions in #development / read up starting here: https://www.pantsbuild.org/docs/plugins-overview

@fast-nail-55400 thoughts here?

Thank you for your quick response! 😄 I may be embarrassingly wrong, but https://github.com/pantsbuild/pants/pull/12948 sounds as if there had been a problem with missing inits as well?

That fixed a test of all this to use .tf files that don't require init! So it worked around the problem you see.

Ah, alright then! I've had to do quick fixes like that. Not embarrassing - "Real Life" (TM). 😉 So I see mainly two, maybe three tasks I would like to be able to perform: terraform validate (which I agree fits perfectly under "check") terraform plan / apply (would that fit into run, package or something else entirely?) terraform fmt works just fine and demonstrates nicely why I would want to use pants over poetry and terraform separately. 🙂 How would the "init" - especially "init -upgrade" - fit into pants?

I think the intent was just to plant a seed for terraform support as is the case with early experimental contribs. This one just never had anyone's time to circle back and continue the work.

Indeed, that is the case.

Hmmm, that would be a pity, as uniting the actual code and the infrastructure-as-code to provision and run it on seem like a major use case for a monorepo to me. Especially with the rise of Microservices, coordinating both is one of the main arguments in favour of them, and leaving that whole feature field to others sounds... sad. 😕 OTOH, this sounds like a fundamental question on the use and scope of pants, which will likely need some more thinking about.

That would be an interesting direction to take this plugin in, but it would require some work.

Okay, just read up on that old PR. Maybe Terraform really isn't in scope for pants. I will try to gather some more experience with pants & python then and see what opinions will form from that. 🙂

The interesting aspect here would be if it's possible to chain the infra to the binary/image. Today you'd use Pants to build a deployable binary and embed it in a Docker image. Then your infra would pick up that image from a tag.

I think to me, the situation looks much like between a program/app and a library it uses, with the code being in the role of the program and the infra in the role of the library. You may change the lib underneath the app or the app on top of the lib, but the true added value happens whenever you need coordinated changes. So I would ask, how much of that already handled relationship could be harnessed to the code/infra scenario? And what in that has not been covered yet?

Taking a step back; so it's clear - Pants does support interacting with the outside world (somewhat obviously) - but, at its core, it models execution as side-effect free functions. It is challenging to deal with external systems sometimes as a result. We have a built-in filesystem watcher that , in combination with process sandboxing, allows us to get control over your file edits. If the external system is truly external, like a remote service - this takes alot more care. That service may or may not be idempotent, etc. So this sort of consideration is what is in play that generally makes integrating these sorts of external system take longer and require more careful deliberation thatn adding a new linter or language vertical to Pants support.

Thank you John, I will ruminate on that. 🙂

And there currently is no "proper" backend to deploy to MS Azure, as there is for AWS and Google Cloud, correct?

I am not aware of any backend that deploys to anywhere. We have support for packaging AWSLambda and Google Cloud Function zip files, but you have to deploy those yourself.

There is a

./pants experimental-deploy

goal that (AFAIK) currently publishes dists to PyPI (by running Twine), and pushes Docker images and Helm charts to registries. But these are relatively straightforward since they require very little extra metadata to know where to deploy to, and they're just pushing files to, essentially, storage. I think doing AWS and GCP might also be fairly straightforward. Terraform seems like a much bigger kettle of fish...

Great, thank you both for clarifying! 😄