https://pantsbuild.org/ logo
Join Slack
Powered by
I set `[python].interpreter_constraints` to CPytho...
# general
d
I set 
[python].interpreter_constraints
to CPython>=2.7,<3 and pants instructed me to generate a new lockfile for pytest. Running 
/v2pants generate-lockfiles --resolve=pytest
fails on what looks like a py2/3 compatibility error in parsing python imports.
Copy code
ProcessExecutionFailure: Process 'Determine Python imports for src/python/fsluigi/main.py' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "./__parse_python_imports.py", line 114, in <module>
    main(sys.argv[1])
  File "./__parse_python_imports.py", line 105, in main
    visitor.visit(tree)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 251, in generic_visit
    self.visit(value)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "./__parse_python_imports.py", line 57, in visit_Call
    elif isinstance(node.args[0], ast.Constant):
AttributeError: 'module' object has no attribute 'Constant'
Pants itself still appears to be running python3.9, but maybe part of the import parsing is being affected by my new interpreter constraint?
h
What Pants version? Can you please send 
flsuigi/main.py
to us, even if over DM? I'm curious if we can narrow down what's causing the parse error and fix it. That's a critical bug
d
pantsd command is: PANTS_DAEMON_ENTRYPOINT=pants.pantsd.pants_daemon:launch_new_pantsd_instance PYTHONPATH=/Users/tansy/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.KynvEQ/install/bin:/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python39.zip:/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9:/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload:/Users/tansy/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.KynvEQ/install/lib/python3.9/site-packages /Users/tansy/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.KynvEQ/install/bin/python /Users/tansy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.9.0_py39/bin/pants --pants-bin-name=./v2pants --pants-version=2.9.0 --level=debug generate-lockfiles --resolve=pytest
I think the file it's failing on just happens to be the first one it runs, but I'll DM it to you. Do you need the whole thing or only the imports?
h
Wait do you expect this file to work with Py2.7?
Pants itself still appears to be running python3.9, but maybe part of the import parsing is being affected by my new interpreter constraint?
Yep, each file is parsed using the interpreter constraints you specify for that file
d
Yes, I expect it to run in 2.7
👍 1
h
Do you need the whole thing or only the imports?
If you have the time, it would be super helpful to try to find the minimal reproduction. Meaning, delete some things and run again to see if it still fails. We want to find the specific line(s) that cause the parse error and remove all the other noise
d
I'll see what I can do. Is there a way to limit 
generate-lockfiles
to only run on this file?
h
You should be able to run 
./pants dependencies path/to/fsluigi/main.py
-- it's failing at dependency inference time
d
👍
Well, it's definitely happening in the code itself, not in the imports. If I delete all the code the imports are parsed fine. So it's weirdness on our end.
h
Ohhhhh it's a bug in our code, not yours. Dang
d
Yep. If I comment out the three places this file is using
Copy code
__import__()
h
Ah and we didn't catch it because it's specifically when using 
__import__
, okay
d
everything works fine
Oh, I really thought it was our code behaving badly, but if you wanna take credit for it, I'm fine with that 😂
git blame:
Copy code
a61fb7681862 (Eric Arellano      2018-08-21 16:22:48 -0700  32)       __import__("{}.{}".format(module_prefix, modname), fromlist="dummy")
😂
❗ 1
h
No not at all. Thanks for reporting it! I'll get out a bug fix soon and cherry-pick to 2.9
d
👍 Thanks Eric!
h
omg what commit was that? yikes
d
Copy code
commit a61fb76818627a80c1865d0ab00af0b820c9904f
Author: Eric Arellano <ericarellano@me.com>
Date:   Tue Aug 21 16:22:48 2018 -0700

    Modernize/string interpolation luigi (#8312)
👍 1
Not your fault 🙂
That file also does
Copy code
if profile_path:
    import cProfile
which pants appears to handle correctly.
Humm. I added 
src/fsluigi
to pants_ignore so I could keep going. I'm now running into a new error:
Copy code
The current project's Python requirement (>=2.7,<3 || >=3.6,<4) is not compatible with some of the required packages Python requirement:
    - pytest requires Python >=3.6, so it will not be satisfied for Python >=2.7,<3
My python requirement is only 
>=2,<3
, I'm not sure where the other version constraint is coming from, or why it claims pytest requires >=3.6.
h
see this tip to debug: https://www.pantsbuild.org/docs/python-interpreter-compatibility#tip-activate-pantsbackendpythonmixed_interpreter_constraints Written with what I wish I had when working on Foursquare's Py3 migration 🙂
d
👍
h
Thanks again, I'll cherry-pick to 2.9: https://github.com/pantsbuild/pants/pull/14232
d
Thank you!
I think the pytest version issue is the result of the PANTS_PYTEST_VERSION default.
h
wait what pytest vesrion issue? but yeah, we use a modern version of pytest by default that doesn't work with Py27
d
Fixed it, i think. I generated lockfiles, but I had to upgrade our version of pytest from 
==3.6
to 
<=4.6
and I have yet to see if that will break anything.
We just had a meeting to discuss updating our requirements more frequently. Some of these are really old.
➕ 1
h
You'll need a python 2.7-compatible version of pytest, but the resolver should do the right thing if you don't constrain
d
👍
Things seem to be working now, in that I have a new error instead (Pants doesn't know about where we host our custom wheels)
h
d
Yeah, I found that. Any idea how I get credentials working? It's a private artifactory repo. I'm extremely unclear on how our V1 pants is accessing it right now.
h
You can set up a custom CA bundle and/or pass auth-related env vars through to the process, as described here: https://www.pantsbuild.org/docs/proxies#setting-up-a-certificate-authority
But you'll have to figure out how that artifactory's auth is supposed to work first
d
Thanks Benjy!
7 Views
Open in Slack
PreviousNext
Powered by