https://pantsbuild.org/ logo
Powered by
a

ambitious-student-81104

10/25/2021, 7:58 PM
Hi! I noticed that sometimes the constraints.txt file is not enforced. For example, I have a package 
pyparsing==2.4.7
(it is a transitive dependency of a package I import) in 
constraints.txt
. When I 
pip install -r constraints.txt
, the correct version gets installed; however, when I use a package that depends on it in the test, the test fails because 
pyparsing==3.0.1
is used despite what is specified in 
constraints.txt
. What are the general causes of this issue and what is the recommended mitigation?
More bizarre is: when I 
./pants repl path_to_test.py
I get the right version:
Copy code
❯ ./pants repl path_to/test_xxx.py
Python 3.7.11 (default, Jul 27 2021, 07:03:16)
[Clang 10.0.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import pyparsing
>>> pyparsing.__init__
<method-wrapper '__init__' of module object at 0x7fc370142350>
>>> pyparsing.__version__
'2.4.7'
however when I add this in the test:
Copy code
import pyparsing
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
it becomes:
Copy code
_ ERROR collecting xxx/test_xxx.py _
xxx/test_xxx.py:3: in <module>
    assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
E   AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
E   assert False
Why would their be a discrepancy between the actual runtime version and the 
repl
version of a package? 🤔
👀 1
h

happy-kitchen-89482

10/25/2021, 8:18 PM
Just to check the obvious thing - you do have the constraints.txt set as 
requirement_constraints
in your config file?
✅ 1
@hundreds-father-404 is this ringing any bells?
a

ambitious-student-81104

10/25/2021, 8:21 PM
Also -- even if I explicitly add
Copy code
python_requirement_library(
    name="pyparsing",
    requirements=["pyparsing==2.4.7"],
)
and depend on it by adding 
:pyparsing
in my test BUILD file, it still gets the latest, not the pinned version
h

hundreds-father-404

10/25/2021, 8:22 PM
No, not at all. Unless we made a mistake somehow in implementation @ambitious-student-81104 what Pants version is this? We did make changes to these code paths in 2.7 to add tool lockfiles, so possible we have a regression
a

ambitious-student-81104

10/25/2021, 8:23 PM
pants_version = "2.6.0" this hasn't changed for us in 2 months, not a new thing. the package recently released a major version which is why it just started failing; but my guess would be Pants has been fetching the latest rather than pinned version this whole time
👍 2
when i 
./pants dependencies --transitive test:target
it also gets the right 
pyparsing
which is the 
python_requirement_library
I just added in the BUILD file to explicitly pin it, but in 
./pants test
it's still getting the latest version
I was wondering if this could be caused by some version conflict with another dependency that wants the latest 
pyparsing
, so I added a target that does nothing but import pyparsing and reveal its version:
Copy code
python_requirement_library(
    name="pyparsing",
    requirements=["pyparsing==2.4.7"],
)


python_tests(
    name="test_pyparsing",
    sources=["test_google_sheets.py"],
    skip_mypy=True,
    dependencies=[
        ":pyparsing",
    ]
)
the content of the test file is:
Copy code
import pyparsing
assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
And it still prints out
Copy code
E   AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
so I think this is probably a pants issue rather than conflict
h

happy-kitchen-89482

10/25/2021, 8:57 PM
Let me see if I can reproduce
Thanks for the very clear steps!
✅ 2
Hmm, I can't reproduce in 2.6 or at HEAD
Do you see a log line like 
[INFO] Completed: Extracting 1 requirement to build requirements.pex from repository.pex: pyparsing==2.4.7
when you run?
a

ambitious-student-81104

10/25/2021, 9:41 PM
yes
Copy code
13.81s Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, p
⠦
h

happy-kitchen-89482

10/25/2021, 9:45 PM
But no 
Extracting...
in a separate log line?
a

ambitious-student-81104

10/25/2021, 9:46 PM
Copy code
[INFO] Completed: Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, pyyaml==5.4.1, setuptools==54.2.0, snowflak... (86 characters truncated)
17:44:06.93 [INFO] Completed: Building pytest_runner.pex
17:44:11.39 [WARN] Completed: test - airflow2/test/python/airflow_common_tests/utils/test_google_sheets.py:tests failed (exit code 2).
============================= test session starts ==============================
platform darwin -- Python 3.7.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /private/var/folders/5c/tp4p_z3d2bnc6fb68lth0d4w0000gp/T/process-execution0XzyvT
plugins: cov-2.11.1, anyio-3.3.4
collected 0 items / 1 error

==================================== ERRORS ====================================
_ ERROR collecting xxpy:3: in <module>
    assert False, f"HORUS_DEBUG: pyparsing version is {pyparsing.__version__}"
E   AssertionError: HORUS_DEBUG: pyparsing version is 3.0.1
E   assert False
But no 
Extracting...
 in a separate log line?
not now that it's completed; i didn't pay attention while it was doing that. Is it possible to clear cache for just that one target?
h

happy-kitchen-89482

10/25/2021, 9:49 PM
Maybe try running with 
--no-process-execution-local-cache
And 
--no-pantsd
to really force it to re-work everything
a

ambitious-student-81104

10/25/2021, 9:50 PM
Copy code
17:50:48.42 [INFO] Completed: Building pytest.pex with 2 requirements: pytest-cov>=2.10.1,<2.12, pytest>=5.4
⠖ 14.23s Building requirements.pex with 12 requirements: apache-airflow==2.1.2, future==0.18.2, gspread==4.0.0, oauth2client==4.1.3, pandas==1.2.5, pyparsing==2.4.7, p
⠦
no new line saying extracting...
h

happy-kitchen-89482

10/25/2021, 9:51 PM
And no 
Resolving 3rdparty/python/constraints.txt
or wherever your constraints file is
a

ambitious-student-81104

10/25/2021, 9:51 PM
yes
well this one is expected to only depend on this in the build file:
Copy code
python_requirement_library(
    name="pyparsing",
    requirements=["pyparsing==2.4.7"],
)
so it doesn't need to look into constraints.txt i think
h

happy-kitchen-89482

10/25/2021, 9:54 PM
if you have 
requirement_constraints
set then it always does
that is the way you say to Pants "any requirement I use is available, already resolved, in the constraints file, so use that"
So the fact that it's not using that is strange
a

ambitious-student-81104

10/25/2021, 9:55 PM
we commented it out.. lemme uncomment it
sorry bout that
@dazzling-diamond-4749 any reason we commented it out a month ago? To speed up builds?
h

happy-kitchen-89482

10/25/2021, 9:57 PM
it is what should speed up builds - but you might not be able to use it if you can't have a single global resolve (we are working on multiple user lockfile support), so maybe that is why it was commented out
✅ 1
a

ambitious-student-81104

10/25/2021, 9:57 PM
⠁ 27.70s Resolving constraints.txt
this takes forever. we are waiting to use pants' extra index + lock file when it becomes available to speed it up
I still find it surprising though, that pinning it in 
python_requirement_library
and depending on that target is not working
h

happy-kitchen-89482

10/25/2021, 9:58 PM
It may take a while to resolve constraints.txt but then that should be cached and you only need to re-resolve it if it changes
and meanwhile every operation that uses it needs no resolve at all, just those fast 
Extracting...
operations
so usually having 
requirement_constraints
is a big performance win
even though resolving constraints.txt one time can be a one-time pain
a

ambitious-student-81104

10/25/2021, 9:59 PM
we don't have remote cache on buildkite set up
h

happy-kitchen-89482

10/25/2021, 9:59 PM
Ah
But yes, you are correct that even without all that, the setup you propose should use the version in the requirement_library
➕ 1
a

ambitious-student-81104

10/25/2021, 9:59 PM
is there any way we don't have to resolve constraints file each time but still pin version?
and you said you're not able to repro? that's bizarre
h

happy-kitchen-89482

10/25/2021, 10:00 PM
Well, I had requirement_constraints on
(we happen to have that version of pyparser in the constraints file in the pants repo, so I was testing there)
let me try turning that off
Bingo
Now I can reproduce
✅ 2
Uuuuuuuuugh
👀 2
this is because pytest itself has an unconstrained dep on 
packaging
, which has an unconstrained dep on 
pyparser
🙌 1
This would be handled OK in 2.7 because we have tool lockfiles, so there is a lockfile for pytest that locks all its 3rdparty deps
In the sandbox where pytest runs there are two pexes, 
pytest.pex
which contains pytest and its deps, and then 
requirements.pex
which contains your own requirements.
But 
pytest.pex
is earlier on the sys.path
I assume, let me check
a

ambitious-student-81104

10/25/2021, 10:15 PM
wow, thanks. Would there be any backwards-incompatible changes from 2.6 to 2.7?
h

hundreds-father-404

10/25/2021, 10:15 PM
in 2.6, I thought that constraints.txt was being applied to tool PEXes too
h

happy-kitchen-89482

10/25/2021, 10:15 PM
They have constraints turned off
There should always be a deprecation cycle between backwards-incompatible changes, so 2.7 will show you deprecation messages for whatever you'd need to change for 2.8, but it'll still work without those changes.
What happens if you turn 
requirement_constraints
back on?
Ignoring performance for a sec, does that solve this issue?
h

hundreds-father-404

10/25/2021, 10:20 PM
Hm could y'all turn on constraints, but turn off that performance optimization to resolve the constraints file once then extract subsets from it? Concretely: https://www.pantsbuild.org/docs/reference-python-setup#section-resolve-all-constraints
Copy code
[python-setup]
requirement_constraints = "path/to/constraints.txt"
resolve_all_constraints = false
h

happy-kitchen-89482

10/25/2021, 10:20 PM
The "real" solution should be to "shade" tools (that is, rewrite their symbols and those of their deps, so they don't conflict with user symbols). But that is a major undertaking, especially with something like pytest that relies extremely heavily on runtime name-based dynamic loading
a

ambitious-student-81104

10/25/2021, 10:51 PM
What happens if you turn 
requirement_constraints
 back on?
it finds the pinned version
h

happy-kitchen-89482

10/25/2021, 11:50 PM
Aha
OK, so is using 
requirements_constraints
an option until you can upgrade to 2.7? And in general, we recommend having that option on, so separately from all this would like to find out why it was turned off
a

ambitious-student-81104

10/26/2021, 3:47 PM
gotcha, will do. thanks!
I'll check the release notes to see if we can upgrade to 2.7
2 Views
Powered by