Why is the terraform subsystem config scope called...
# development
f
Why is the terraform subsystem config scope called
download-terraform
? If it affects terraform generally, shouldn't it just be
terraform
? https://github.com/pantsbuild/pants/blob/f2246fdfa8089a819ba1764d52e96d063de7111e/src/python/pants/backend/terraform/tool.py#L47
w
Are there multiple terraform subsystems/scopes (either now or future)? I know approximately nothing about terraform, but I would have assumed similar to you on the naming
f
Possibly, but this one seem to be acting as the main one, given the options it has, like
Additional environment variables that would be made available to all Terraform processes.
So the name is slightly confusing today I think
There's no actual
terraform
scope currently
w
Yeah, I saw that a while back - and just assumed there was a reason for that 🤷 Maybe not? Is it mentioned in the OG PR?
f
Not that I could find, maybe @fast-nail-55400 would know if it was on purpose
f
The issue is that tools are their own subsystems and names cannot be the same.
So the tool subsystem is
download-terraform
and the general options for the backend is
terraform
✅ 1
w
https://pantsbuild.slack.com/archives/C0D7TNJHL/p1719781841075449?thread_ts=1719780913.456259&cid=C0D7TNJHL Relevant - Daniel was mentioning that this migration would be nice to allow leaving experimental
👍 1
f
• change options scope
download-terraform
->
terraform
Ah! 😅 it is already noted then
Could we just do that change, or would it conflict with what Tom said?
w
I'm myself confused now, because I thought what Tom said was how it would work - as I thought the options qualified the subsystems the same as the backends - but could just try changing it and see if pants complains
Ive never really given much thought to how the backend option system vs the subsystem option system namespacing works to be completely honest. @happy-kitchen-89482 ^^ - I think you're the local options expert
h
There is no “backend option system”. All options are on subsystems.
f
Then if there's no current subsystem option scope for "terraform", it should just be fine to change that scope to it?
Maybe more interesting is, is there a way to do that migration without breaking configs using the current scope? Like supporting both, deprecating one or similar?
h
There is a built-in capability for deprecating an entire scope: https://github.com/pantsbuild/pants/blob/main/src/python/pants/option/subsystem.py#L71
👍 1
w
> There is no “backend option system”. All options are on subsystems. In that case, I think we have a naming convention problem. I enable the
pants.backend.python
in pants.toml, and I can configure it as:
Copy code
[python]
… whatever
And then, the subsystem representing that is in a file named
setup.py
with the class
PythonSetup
, that has an
options_scope=“python”
with the help description of “Options for Pants’ Python backend”
So, should the Terraform backend follow something similar for the “top level” subsystem representing configuration for the whole backend?
f
Yes. Is it just missing having a subsystems submodule carrying the options? It would be just moving it to another module, unless extracting something from it? The scope-rename would be the same, no?
So, a (maybe) minor refactor and a scope rename with deprecation is possibly okay to do here?
h
Often, the naming issue is that we have a convenient affordance for subsystems that represent tools to be downloaded (or resolved from PyPI) and very often those tools have the same name as the overall backend functionality (e.g.,
terraform
or
mypy
or whatever). So we end up with two subsystems, one for configuring downloading the tool, and the other for configuring how the tool is used.
Possibly those could be the same subsystem though. I don’t offhand see why you couldn’t add extra options to the subclass of
TemplatedExternalTool
(in this case
TerraformTool
) or of
PythonToolBase
(e.g.,
MyPy
).
You would effectively be saying “this backend is so closely tied to a single tool that the two subsystems can be one” but I think that is very true in most cases
👍 1
f
You would effectively be saying “this backend is so closely tied to a single tool that the two subsystems can be one” but I think that is very true in most cases
Yeah maybe that is quite fair. Should there be a way to model that case more explicitly, more than just naming convention? If it would simplify the "common case" somehow?
Maybe that's what you meant by just making it one subsystem
👆 1
c
I mean if we're good renaming things, I'm good renaming things. Ideally, we'd have an update-build-files that could operate on the pants.toml itself. Then, we could just have people run that and not have any breaking.
The reason for why it's
download-terraform
was because it started out as just the info to download terraform (version number and urls and stuff) and then I pushed more things onto it and didn't realise I was bloating it until it was too late
there's probably a rationale for keeping both a
terraform
and
download-terraform
(or
terraform-download
) for adding OpenTofu, as I think there would be some logic that we could contain there.
f
there's probably a rationale for keeping both a
terraform
and
download-terraform
(or
terraform-download
) for adding OpenTofu, as I think there would be some logic that we could contain there.
Yeah maybe it's more moving options to a
terraform
scope rather than renaming it then? I think it's the only scope that has a
download
-classifier, confusion comes from options like
platforms
that affect the actual terraform operations for example