Hi, I need some advice/best practices on how to wo...
# general
h
Hi, I need some advice/best practices on how to work with resolves for shared code. In my repo, i have 40-50 files which form shared library, which till yet were using default resolve. Now i have 2 main files(2 pex_binary) which use 2 different resolve(say
ai-cpu
and
ai-gpu
). my resolves looks like:
Copy code
python_requirements(
    name="reqs",
    resolve=parametrize("python-default", "ai-gpu", "ai-cpu"),
)
python_requirements(
    name="ai-gpu",
    source="ai-gpu-requirements.txt",
    resolve="ai-gpu",
)

python_requirements(
    name="ai-cpu",
    source="ai-cpu-requirements.txt",
    resolve="ai-cpu",
)
Based on docs,
If a first-party target is compatible with multiple resolves, e.g., shared utility code, you can use the parametrize mechanism with the resolve= field.
i should use `python_sources(resolve=parametrize('ai-gpu', 'ai-cpu', 'python-default'). Given my shared code is spread across multiple directories, it will get unmaintainable to always set these resolves for all the targets. So i am thinking to set the defaults at root of python source code, like:
Copy code
__defaults__(
    {(python_sources, python_source): dict(resolve=parametrize(python-default", "ai-gpu", "ai-cpu) )}
)
and for the 2 main files(pex_binary targets) that i am interested it, i will set the resolve
ai-cpu
,
ai-gpu
. Do you think this is a good idea? Should
__defaults__
be used/exploited like this, does it form a good practice, or is there a better way of doing this?
c
If you have that as default at the root, it will apply to everything, in which case I wonder why you have different resolves at all?
Other than that, to me it makes sense to use defaults in this way, yes.
h
i was thinking, all my code will have 3 resolves, specify resolve in
pex_binary
as below
Copy code
# ai/inference/BUILD
pex_binary(name="app-cpu", entry_point="main_inference.py", resolve="ai-cpu")
pex_binary(name="app-gpu", entry_point="main_inference.py", resolve="ai-gpu")

# webserver/BUILD
# non-ai-related code which doesn't care about cpu/gpu
pex_binary(name="app", entry_point="main.py")  -> this can take default resovle
if i don't provide all 3 resolves to all shared python_sources, i will not be able to provide different resolves to each pex_binaries
the downside is this. say i have a script
webserver/script.py
, because of 3 resolves in default, it generated 3 target for this script file. so now i cannot do
pants run webserver/script.py
, i'll have to do
pants run webserver/script.py@python-default
, ie explicitly provide the resolve
c
ah, ok. yea that makes sense I guess.
you can provide the resolve explicitly to any targets that you only want/need the default resolve for:
Copy code
python_sources(overrides={"script.py": dict(resolve=None)})
h
thanks