ambitious-student-81104
10/25/2021, 7:58 PMpyparsing==2.4.7
(it is a transitive dependency of a package I import) in constraints.txt
. When I pip install -r constraints.txt
, the correct version gets installed; however, when I use a package that depends on it in the test, the test fails because pyparsing==3.0.1
is used despite what is specified in constraints.txt
. What are the general causes of this issue and what is the recommended mitigation?./pants repl path_to_test.py
I get the right version:
❯ ./pants repl path_to/test_xxx.py
Python 3.7.11 (default, Jul 27 2021, 07:03:16)
[Clang 10.0.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import pyparsing
>>> pyparsing.__init__
<method-wrapper '__init__' of module object at 0x7fc370142350>
>>> pyparsing.__version__
'2.4.7'
however when I add this in the test:
import pyparsing
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
it becomes:
_ ERROR collecting xxx/test_xxx.py _
xxx/test_xxx.py:3: in <module>
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
E AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
E assert False
Why would their be a discrepancy between the actual runtime version and the repl
version of a package? 🤔happy-kitchen-89482
10/25/2021, 8:18 PMrequirement_constraints
in your config file?ambitious-student-81104
10/25/2021, 8:21 PMpython_requirement_library(
name="pyparsing",
requirements=["pyparsing==2.4.7"],
)
and depend on it by adding :pyparsing
in my test BUILD file, it still gets the latest, not the pinned versionhundreds-father-404
10/25/2021, 8:22 PMambitious-student-81104
10/25/2021, 8:23 PM./pants dependencies --transitive test:target
it also gets the right pyparsing
which is the python_requirement_library
I just added in the BUILD file to explicitly pin it, but in ./pants test
it's still getting the latest versionpyparsing
, so I added a target that does nothing but import pyparsing and reveal its version:
python_requirement_library(
name="pyparsing",
requirements=["pyparsing==2.4.7"],
)
python_tests(
name="test_pyparsing",
sources=["test_google_sheets.py"],
skip_mypy=True,
dependencies=[
":pyparsing",
]
)
the content of the test file is:
import pyparsing
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
And it still prints out
E AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
so I think this is probably a pants issue rather than conflicthappy-kitchen-89482
10/25/2021, 8:57 PM[INFO] Completed: Extracting 1 requirement to build requirements.pex from repository.pex: pyparsing==2.4.7
when you run?ambitious-student-81104
10/25/2021, 9:41 PM13.81s Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, p
⠦
happy-kitchen-89482
10/25/2021, 9:45 PMExtracting...
in a separate log line?ambitious-student-81104
10/25/2021, 9:46 PM[INFO] Completed: Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, pyyaml==5.4.1, setuptools==54.2.0, snowflak... (86 characters truncated)
17:44:06.93 [INFO] Completed: Building pytest_runner.pex
17:44:11.39 [WARN] Completed: test - airflow2/test/python/airflow_common_tests/utils/test_google_sheets.py:tests failed (exit code 2).
============================= test session starts ==============================
platform darwin -- Python 3.7.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /private/var/folders/5c/tp4p_z3d2bnc6fb68lth0d4w0000gp/T/process-execution0XzyvT
plugins: cov-2.11.1, anyio-3.3.4
collected 0 items / 1 error
==================================== ERRORS ====================================
_ ERROR collecting xxpy:3: in <module>
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
E AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
E assert False
But nonot now that it's completed; i didn't pay attention while it was doing that. Is it possible to clear cache for just that one target?in a separate log line?Extracting...
happy-kitchen-89482
10/25/2021, 9:49 PM--no-process-execution-local-cache
--no-pantsd
to really force it to re-work everythingambitious-student-81104
10/25/2021, 9:50 PM17:50:48.42 [INFO] Completed: Building pytest.pex with 2 requirements: pytest-cov>=2.10.1,<2.12, pytest>=5.4
⠖ 14.23s Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, p
⠦
happy-kitchen-89482
10/25/2021, 9:51 PMResolving 3rdparty/python/constraints.txt
or wherever your constraints file isambitious-student-81104
10/25/2021, 9:51 PMpython_requirement_library(
name="pyparsing",
requirements=["pyparsing==2.4.7"],
)
happy-kitchen-89482
10/25/2021, 9:54 PMrequirement_constraints
set then it always doesambitious-student-81104
10/25/2021, 9:55 PMhappy-kitchen-89482
10/25/2021, 9:57 PMambitious-student-81104
10/25/2021, 9:57 PM⠁ 27.70s Resolving constraints.txt
this takes forever. we are waiting to use pants' extra index + lock file when it becomes available to speed it uppython_requirement_library
and depending on that target is not workinghappy-kitchen-89482
10/25/2021, 9:58 PMExtracting...
operationsrequirement_constraints
is a big performance winambitious-student-81104
10/25/2021, 9:59 PMhappy-kitchen-89482
10/25/2021, 9:59 PMambitious-student-81104
10/25/2021, 9:59 PMhappy-kitchen-89482
10/25/2021, 10:00 PMpackaging
, which has an unconstrained dep on pyparser
pytest.pex
which contains pytest and its deps, and then requirements.pex
which contains your own requirements.pytest.pex
is earlier on the sys.pathambitious-student-81104
10/25/2021, 10:15 PMhundreds-father-404
10/25/2021, 10:15 PMhappy-kitchen-89482
10/25/2021, 10:15 PMrequirement_constraints
back on?hundreds-father-404
10/25/2021, 10:20 PM[python-setup]
requirement_constraints = "path/to/constraints.txt"
resolve_all_constraints = false
happy-kitchen-89482
10/25/2021, 10:20 PMambitious-student-81104
10/25/2021, 10:51 PMWhat happens if you turnit finds the pinned versionback on?requirement_constraints
happy-kitchen-89482
10/25/2021, 11:50 PMrequirements_constraints
an option until you can upgrade to 2.7? And in general, we recommend having that option on, so separately from all this would like to find out why it was turned offambitious-student-81104
10/26/2021, 3:47 PM