Hey folks, qq here: I'm working on

pipenv_requirements

build alias to source python requirements from pipenv using the

PythonRequirements

class as a model. its working like a charm. However, we've traditionally kept our

Pipfile.lock

file at the project root but, for ease of migration, i'd still like the build addresses for the requirements to go to

3rdparty/python:<req>

is there a way i can use the

_parse_context

to get this to happen?

Hey good morning / good afternoon.

However, we’ve traditionally kept our Pipfile.lock file at the project root but, for ease of migration, i’d still like the build addresses for the requirements to go to 3rdparty/python:<req> is there a way i can use the

_parse_context

to get this to happen?

I would recommend that you perhaps ignore the

rel_path

from the parse context and that you always look at the build root for the file. You would call your new macro in

3rdparty/python/BUILD

, which would result in the new

python_requirement_library

targets that get generated from your macro being created there, in

3rdparty/python

. Rather than

os.path.join(self._parse_context.rel_path, requirements_relpath)

, use

get_buildroot()

from

pants.base.environment

, so something like

os.path.join(get_buildroot(), "Pipfile.lock")

You could also then make it more generic by having your macro take some parameter like

path_from_buildroot

.

relatedly, im not sure i understand the purpose the target defined here:

Indeed, that target is being used so that invalidation works properly. Without adding a dependency on all the generated

python_requirement_library

targets, Pants would not know that they’re invalidated upon changes to the Pipfile. Previously, this dependency was expressed by creating a

files()

target. But, that resulted in the requirements.txt file showing up more than we wanted to, as it was included wherever else

files()

targets would be pulled in. So, Benjy created

_python_requirement_library

to better model the situation. Afaict, you can reuse that same target type, no need to add a new

_pipfile_requirement_library

or anything.

ok im down with always look at the root ("im actually defining the

pipenv_requirements

in

3rdparty/python

and setting the rel to

../../Pipfile.lock

with no issues. Where the problem comes in is in defining the

_python_requirement_library

target since the sources fielld being passed is

['../../Pipfile.lock']

and PEX is not happy with that

Hm where is that showing up with Pex? If you use

os.path.join(get_buildroot(), "Pipfile.lock")

, I think you’d had have an absolute path and Pex would be fine

yeah let me revert one sec

so i think that may be a path forward

Which part? The

get_buildroot()

suggestion?

yes! now let me ask you this, if i wanted to design this in a way that might get pulled into the main project, would something like a

resolve_from_buildroot

option on the build_alias be acceptable to toggle this behavior on and off?

To get pulled into the main project (which would be great!), we’d want it to be generic. I think the default behavior would be like

python_requirements

that the

Pipfile.lock

is a sibling file to where your BUILD file is. For any new users, if they have a top-level

Pipfile.lock

, we would encourage them to use a top-level

BUILD

and specify reqs via

//:my-req

, rather than

3rdparty/python:req

. From there, though, the macro could have options like

resolve_from_buildroot

and/or

requirements_relpath

to do what you’re describing.

yup that makes sense

Ahh, hm. @witty-crayon-22786 is it possible for a CAFO to create an object in an ancestor folder? JP is trying to create a

_python_requirements_file

defined in the build root from a CAFO that gets called in

3rdparty/python/BUILD

. He wants all the generated

python_requirement_library

targets to still be in

3rdparty/python

, though.

it is not.

the

python_requirement_library

?

the “CAFO” API

Bummer. This is less automatic, but I think this will solve your specific problem, JP. Manually create a

_python_requirements_file

target in your top level

BUILD

for

Pipfile.lock

. Check that in. Then, in your macro, have it declare an explicit dependency on

//:pipfile

for every generated

python_requirement_library

.

ah

Where CAFO = context-aware object factory

yeah i think that'll work

If you still want it to be general, you could maybe have an option on the macro called

pipfile_target: Optional[str]

. If unset, then generate the

_python_requirement_library

in the same directory as the macro. Otherwise, use the provided explicit dep. This does leak far more than the original design, but is somewhat general.

at any rate, its working functionally