Heyo! I'm working on embedding an open source repo...
# general
r
Heyo! I'm working on embedding an open source repo as a subproject in one of our closed source repos. Initial thread here. The open source repo has a WIP Pants branch, but when trying to run any goal with that added as a subproject I get errors like
Copy code
MappingError: Failed to parse <subproject dir>/docs/BUILD:
ParseError: <subproject dir>/docs/BUILD:41: Name 'adhoc_tool' is not defined.
I guess
subproject_roots
doesn't infer nested backend packages? I haven't yet figured out all that's needed to be added to the closed/parent project for us to work around it but so far it doesn't pick up • backend packages:
adhoc
,
docformatter
• nested resolves:
Copy code
UnrecognizedResolveNamesError: Unrecognized resolve name from the field `resolve` in the target <subproject dir>:docformatter: docformatter
So if anyone knows the easiest way to get around this please let me know! Trying to build the docs (
pants package <subproject dir>/docs
) in the WIP Pants branch with a somewhat minimal parent-pants.toml should at least be a decent subproject example.
c
from what I can tell, the subproject feature does nothing more than adjust target addresses.. so you'd need to run compatible version of pants in the parent project along with any backends used enabled and configured in the parent pants.toml as well.
r
I manually added the missing backends & resolves, managed to generate the lockfiles but on
pants run //:binary-using-subproject
it seems to not have inferred the dependencies correctly either way
Copy code
InvalidFieldException: src/py/emote: Failed to get dependencies for <subproject dir>:root#gymnasium: The file or directory <subproject dir>/<subproject dir>/pyproject.toml' does not exist on disk in the workspace, so the address '<subproject dir>/<subproject dir>/pyproject.toml:root' from the `dependencies` field from the target <subproject dir>:root#gymnasium cannot be resolved.
(notice the path to the subproject dir/root getting duplicated/nested in it self). Maybe I can get around that by not referencing dependencies by
//root#dep
in the subproject but rather in some explicit way like
//emote#dep
?
c
huh 🤷
g
Relative paths maybe?
(If everything would be relative I'm wondering if subproject_dirs is even necessary since configs aren't merged either way...)
r
It seems I have to add (the same?) resolves for the sub package even though the sub package doesn't actually enable resolves 😞
So even if I would manage to reference the packages correctly, it seems like the (missing) resolves will get in the way either way
g
Hmm. Not sure I follow. Does the pants.toml from the subpackage actually get used for some things? Or do you mean when using it "standalone"?
r
From what I can tell, the subpackage pants.toml is completely ignored, not absolutely sure but seems like that
Standalone, the subpackage works fine!
g
So which resolve is it complaining about and where is it defined, and where do you have to add it? Is it e.g. subpackage has resolve
x
and when used from the parent package that isn't defined?
r
The resolve(s) are defined in the parent project but are expected to be denoted in the subproject
Copy code
11:30:41.03 [ERROR] 1 Exception encountered:

Engine traceback:
  in `dependencies` goal

ResolveError: The address `<subproject dir>/docs:docs` from <infallible> was not generated by the target `<subproject dir>/docs:docs`. Did you mean one of these addresses?

  * <subproject dir>/docs@resolve=cpu
  * <subproject dir>/docs/conf.py@resolve=cpu
  * <subproject dir>/docs@resolve=gpu
  * <subproject dir>/docs/conf.py@resolve=gpu
  * <subproject dir>/docs@resolve=base
  * <subproject dir>/docs/conf.py@resolve=base
In the parent project we have
base
,
cpu
and
gpu
as resolves
g
Ah, this is likely an actual config error caused by a catch-all parametrization. Try adding
__defaults__()
in
docs/BUILD
.
r
No difference. Unless I need to rebuild lockfiles?