I'm trying to come up with a good way to manage local environment variables. I asked a question a couple days ago about using a

.env

file, but hardcoding the file into a

BUILD

file doesn't work well since

.env

isn't tracked in version control, so CI tests throw warnings. I'm wondering whether a better approach is a plugin that explicitly loads the

.env

variables as environment variables for the test, then run that plugin as a command line argument to test? That way I could run it locally, but not include it in CI. Two things I'm wondering about: 1. would this approach work? I know tests are "hermetic", so would building a plugin that loads the variables work? 2. Can you "stack" targets; e.g. call the

.env

plugin prior to test so that they work together?

So

.pants.bootstrap

is a hidden thing. That can be arbitrary bash, but should really just stick to setting env vars.

Is there an intention for it to be undocumented, or is that just an accidental omission?

ok so to be clear, I would create a file named

pants.bootstrap

and in it I put, for example

HOSTNAME=localhost

, then in pants.toml I have

[test]
extra_env_vars = ["HOSTNAME"]

Now my tests should have

$HOSTNAME

available as an environment variable?

Yes, except

.pants.bootstrap

- leading dot.

hmm still missing something. I can tell it's working because I can cause errors in

.pants.bootstrap

and I have the variable defined in the pants.toml. But when I run the test in python

os.environ["HOSTNAME"]

throws a key error because it can't find the environment variable

@loud-laptop-89838 please try poking the hole using BUILD instead of `pants.toml`https://www.pantsbuild.org/docs/reference-python_tests#codeextra_env_varscode I just had to use this yesterday for my own project and it worked.

yeah target isn't working either. Really weird. Like I said, I know bootstrap is loading because I can make it throw errors. Then under the directory BUILD file in the python_tests target I added

extra_env_vars

as just an array of strings, which match the values set in the bootstrap file

Hrm, OK. I'm on 2.14.0. I'll have a review out with my repo later today so I can point at that as an example, but makes no sense to me how this isn't working for you.

same. And just to confirm, if I set

extra_env_vars=["HOSTNAME=123"]

that should add the env var to the test right?

Yeah, that's a way to narrow the debugging here. The test should see 123.

Ok so that worked. I got my test to print out

os.environ

when it fails and it shows HOSTNAME=123

Ok, so its just the pass-through somehow not working.

Seems to be.

Do you have a pants script with the lines I linked?

Yup.

No clue. You'll need to dig a bit.

yeah I guess so.

So, another unlocker but still a bug needs to be fixed, go back to your pants.toml approach, but edit like so:

[test]
extra_env_vars = ["HOSTNAME=%(env.HOSTNAME)s"]

ahh stupid mistake. I didn't have

export

at the start of the lines in bootstrap. I just copied from .env.

Aha. Ok great. Works then?

I'm getting some other weirdness, but the key error is gone now, so the variables seem to be set

Cool.

appreciate the help!

The

./pants

replacement thing? That'll be a new pantsbuild/scie-pants repo later today. I'd just like to finish the new update capability ~

SCIE_BOOT=update-scie-pants pants

to self-update the new "pants" script, which will just be a

scie-pants

binary anywhere on your

PATH

. I'm afraid without the update capability folks will start using it and then not get fixes for this very new thing easily if there are issues.

you're probably right. I look forward to seeing it

CI setup, a few PRs to add self-update and pants-from-sources support later today along with releases: https://github.com/pantsbuild/scie-pants Once there are releases those will be safe to download and use, static binaries so just download, name to your desired name and put on PATH.

I've been using it today. So far, so dope!

I think the big quality of life improvement is you can write all this slow-path work in Python now.

The readme mentions that python is self contained in

scie-pants

. Does this mean for all python projects as well? Or just the pants internals? Ie how does this interact with, eg

pyenv

?

One of the ideas here is to distribute an embedded Python interpreter with Pants, thereby removing the requirement of a specific system Python just to act as a Pants runner. So, pants becomes more of a "standalone binary", rather than "a collection of python files that are run by the user" So, this requirement goes away: https://www.pantsbuild.org/docs/prerequisites

Python 3.7, 3.8, or 3.9 discoverable on your

PATH

Here's a long-running tracking issue: https://github.com/pantsbuild/pants/issues/7369

Totally makes sense since not all projects are going to be on python backends. But just so I understand, this bundled python interpreter would not be used if I have a python project. Eg I couldn't just install pants (no python on the system) and then turn on the python backend and then run

test

for example on a python project? I would still need a python interpreter for my project. It's just that pants has one for it's own internals.

That's a good question. Today, you can specify a subset of the Python interpreters available on your machine to be used by your application code (e.g. I use Python 3.10 for my code, even though it can't be used to run Pants - so I have a separate Python 3.9 for Pants). In a blank system though, I wonder if you could (or even would want to) expose that embedded interpreter to act as a system python to run your application 🤔 For some more context, these are embedded: https://github.com/indygreg/python-build-standalone and they're placed in your system's cache directory, so they could be theoretically wiped out from beneath you (but

scie-pants

doesn't care, as it'll just grab it again).

Packaging pr? Are you creating a build system in pants?

Oh, sorry...

goal

, like

./pants package xyz

Well what about bundling

pyenv

as a backend. Then you could just get whatever the user specifies as a constraint and easily manage any number of different version constraints in the monorepo. Not saying I want to do it 😆 but maybe sometime in the future. In the meantime I'm just looking forward to testing this!

You can play around with

scie-pants

today - I've been using it since yesterday

Right now the Python distribution used to bootstrap and run Pants is added to the PATH last here: https://github.com/pantsbuild/scie-pants/blob/main/package/scie-pants.lift.json#L93 That's probably a bad idea though. These interpreters are not full featured. Any attempt to use them to build a wheel with native extensions will fail (that is the ~only known limitation though). The question of helping you use pyenv or not is a Pants question and not a scie-pants question. The role of scie-pants should stay tightly scoped to launching Pants (and installing it as needed). It should be very fast in the former role and constrained in the latter. Pants itself and Pants goals and rules are the right tools for almost all other features.

That's helpful to understand the scope actually.

Right, so you could still have

scie-pants init

and the

scie-pants

part of that would see you don't have a

pants.toml

and it would set that up, which would launch pants, which would interpret the

init

goal and run that to further initialize the setup. Etc. So scie-pants got out of the way quick there and delegated everything fancy to the Pants

init

goal - a full fledged goal with rules and all the fancy complicated logic.