https://pantsbuild.org/ logo
#general
Title
# general
q

quiet-evening-25363

04/22/2022, 8:27 PM
hi all - I'm having trouble with multiple versions of a dependency that I could use some help with. I would like to generate 2 lockfiles will all of the same dependencies except for a single package (one to include psycopg2, one to include psycopg2-binary). Following the docs I'm able to generate 2 lockfiles, but the problem is that the non-default generated 3rdparty/python/beam.lock only includes pyscopg2-binary, but I would like it to include all requirements in requirements.txt and add psycopg2-binary. The default 3rdparty/python/default.lock file is correct. Is that possible or do I need to duplicate my requirements.txt file with the 2 versions of pyscopg2? I'll include my BUILD files in the thread. Thanks!
1
I have a single 3rdparty/python/requirements.txt file that does not include psycopg2/psycopg2-binary. pants.toml
Copy code
[python]
interpreter_constraints = ["CPython==3.9.*"]
enable_resolves = true
default_resolve = "default"

[python.resolves]
default = "3rdparty/python/default.lock"
beam = "3rdparty/python/beam.lock"
3rdparty/python/BUILD:
Copy code
python_requirement(
    name="psycopg2",
    requirements=["psycopg2==2.9.3"],
    resolve="default",
)

python_requirement(
    name="psycopg2-binary",
    requirements=["psycopg2-binary==2.9.3"],
    resolve="beam",
)
src/where/I'm/using/the/beam/lockfile/BUILD
Copy code
python_sources(
    dependencies=[
        "3rdparty/python#google-cloud-storage",
    ],
    resolve="beam",
)

pex_binary(
    name="search_index_main",
    entry_point="main.py",
    dependencies=[
        "3rdparty/python#google-cloud-storage",
    ],
    resolve="beam",
)
h

hundreds-father-404

04/22/2022, 9:36 PM
Hello! Which pants version are you using? The answer changes a little bit
q

quiet-evening-25363

04/22/2022, 9:49 PM
just upgraded to 2.10.0 to try this out
👍 1
h

hundreds-father-404

04/22/2022, 10:11 PM
In 2.10, you have to create a distinct target for each resolve
Copy code
# normal one you had
python_requirements()

python_requirements(
  name="beam",
  resolve="beam"
)
In 2.11, you can still do that, or you can use the new parametrize mechanism https://www.pantsbuild.org/v2.11/docs/targets#parametrizing-targets
Copy code
python_requirements(resolve=parametrize("beam", "default-resolve"))
Warning that you will need to not only update the Python requirement targets, but also any transitive dependency of
src/where/I'm/using/the/beam/lockfile:search_index_main
, e.g.
python_sources
targets. If that code should work with both of your resolves, then you either need to use
parametrize
or manually create 2 targets
2.11
Latest release is 2.11.0rc3. We are hoping to stabilize next week If you are upgrading, you will see a deprecation message about the lock file generator. I recommend sticking w/ poetry in the short term while you play w/ multiple lockfiles to minimize changes
q

quiet-evening-25363

04/22/2022, 10:22 PM
ok great! adding the 2nd python_requirements target was the missing piece. I'll take a look at upgrading all the way to 2.11 and the lock file generator deprecation. thanks for the help and info!
❤️ 1
h

hundreds-father-404

04/22/2022, 10:25 PM
Feedback is definitely appreciated on this new feature! For example, we are aware that the error messages need to be better when a dependency is not compatible: https://github.com/pantsbuild/pants/issues/14864. I suspect that there are other things we need to make more clear too Maybe we need more quality examples? This is a fairly unique approach to multiple lockfiles
👍 1
5 Views