https://pantsbuild.org/ logo
Join Slack
Powered by
I’ve been trying to upgrade from 1.16 -> 1.27, ...
# general
p
I’ve been trying to upgrade from 1.16 -> 1.27, made a lot of progress, but im having issues with python lint. I see this step fail 
pythonstyle
. I’ll attach the actual stack trace as a response to this message. I added some debugging to the pants python code, and it looks like it is resolving dependencies to the root site-packages directory rather than the individual one per each dependency 
[pantsbuild.pants.contrib.python.checks.checker 1.27.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), pyflakes 2.1.1 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), pycodestyle 2.4.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), six 1.15.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages)]
. Not sure what to do here, maybe i have some dumb flag?
Copy code
Exception caught: (builtins.AttributeError)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/bin/pants", line 8, in <module>
    sys.exit(main())
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/pants_loader.py", line 94, in main
    PantsLoader.run()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/pants_loader.py", line 90, in run
    cls.load_and_execute(entrypoint)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/pants_loader.py", line 83, in load_and_execute
    entrypoint_main()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/pants_exe.py", line 33, in main
    runner.run(start_time)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/pants_runner.py", line 102, in run
    runner.run()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 346, in run
    goal_runner_result = self._maybe_run_v1(v1)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/local_pants_runner.py", line 274, in _maybe_run_v1
    self._exiter,
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/goal_runner.py", line 230, in run
    return self._run_goals()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/goal_runner.py", line 201, in _run_goals
    result = self._execute_engine()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/bin/goal_runner.py", line 189, in _execute_engine
    result = engine.execute(self._context, self._goals)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/engine/legacy_engine.py", line 21, in execute
    self.attempt(context, goals)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/engine/round_engine.py", line 252, in attempt
    goal_executor.attempt(explain)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/engine/round_engine.py", line 50, in attempt
    task.execute()
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/contrib/python/checks/tasks/checkstyle/checkstyle.py", line 287, in execute
    failure_count += self.checkstyle(interpreter, sources)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/contrib/python/checks/tasks/checkstyle/checkstyle.py", line 220, in checkstyle
    checker = self.checker_pex(interpreter)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/contrib/python/checks/tasks/checkstyle/checkstyle.py", line 198, in checker_pex
    pex_builder.add_dist_location(dist.location)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pants/python/pex_build_util.py", line 576, in add_dist_location
    self._builder.add_dist_location(location)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pex/pex_builder.py", line 410, in add_dist_location
    self.add_distribution(dist, dist_name=name)
  File "/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/1.27.0_py36/lib/python3.6/site-packages/pex/pex_builder.py", line 371, in add_distribution
    dist_name = dist_name or os.path.basename(dist.location)

Exception message: 'NoneType' object has no attribute 'location'
w
sorry for the trouble… is there a particular reason you chose to target 1.27? it’s possible that releasing a slightly older version, and then jumping to 1.30 might be feasible?
p
im not sure if i remember why 27, but is it buggy or just that no one remembers it very well? 😛
w
the latter =/
i’m not able to find any references to that error, although i seem to remember that it was fixed at some point. so my bias was toward suggesting getting on a more recent version. but given the distance you’re upgrading, at least two hops might be reasonable.
p
i dont think ill be able to carve out the time to do that, a lot of time went into the 1.27 upgrade already and it is mostly working except for py lint 😞
w
ok. then you might consider targeting the very latest
(latest 1.x)
1.30
p
hmmm, py check isn’t in 1.30?
w
so in 1.30.x, the expectation is that linters will be run with v2 (and they are not distributed as a separate module).
could go to 1.29.x if you want to defer doing that.
p
should we be using a different linter?
even in 27?
w
what do you mean?
p
i mean, is it meant to work in 27? or should i switch to a newer linter?
im getting the same error in 29 too
w
it is supposed to work before 1.30.x… i just don’t recall where that issue would have been fixed. if it’s broken in 1.29.x though, it might be an issue that we did not fix in v1.
@enough-analyst-54434: does the stacktrace at the top look familiar to you? i could have sworn that that was related to a fixed bug, but cannot find it.
h
By 1.27 you can run all python functionality in v2, no? Is that the ultimate goal here?
p
how do you mix run things in v1 and v2?
i added some debugging info, it looks like it gets stuck on trying to resolve a dependency of six, does that sound familiar at all @witty-crayon-22786?
w
https://pantsbuild.slack.com/archives/C046T6T9U/p1607972963187600?thread_ts=1607962998.182900&amp;cid=C046T6T9U to do it on a goal-by-goal basis, you can do roughly: 
./pants --v2 lint
, for example.
the 1.30.x “upgrading” docs talk about running in mixed mode: https://www.pantsbuild.org/v1.30/docs/how-to-upgrade-to-the-v2-engine
h
@polite-vase-75369 add 
pants.backend.python.lint.flake8
under 
backend_packages2
. Also set 
v2 = true
. Both in the 
[GLOBAL]
section Then, lint will use both v1 and v2 linters. You'll want to remove the problematic v1 plugin
Flake8 should be a much better experience than the checkstyle task, which was a kind of weird fork of flake8. You'll get more checks, can use a standard flake8 config file, and can use flake8 plugins
p
this works in 1.27 too?
h
Yes, it should. No need to target 1.30
You can also hook up Black, v2 isort, docformatter, and Pylint. All without using v2 for the rest of things like running tests. https://www.pantsbuild.org/v1.27/docs/python-linters-and-formatters We wanted to allow for incremental adoption of v2
p
let me see if i can get this to work. our company is scala + python, and sadly im not very pythonic
hmmm i have a custom build type that isn’t being recognized in the new v2
Copy code
Target type 'aiq_python_awslambda'
but i added it to backend_packages2
h
You'll need to write a light weight binding with the Target API. See the info box on https://www.pantsbuild.org/v1.28/docs/target-api-concepts for instructions and an example Even though those docs are 1.28, we added the target api in 1.27 so it should work
p
ok hmm
im also getting
Copy code
Exception: Unmatched glob from the option `--flake8-config`: "build-support/.flake8"
some reason it isn’t happy in that dir
h
Hm, what's the full error? The value should be relative to the build root, eg 
build-support/.flake8
It's possible also that it's being unintentionally included in 
pants_ignore
. You could try removing the leading . to check - in older pants version, we were over eager with the default value for pants_ignore
p
yeah i think that was it
im trying to ignore those weird lambda packages because i dont really care to lint them right now, but the flake8 config for ignoring doesn’t seem to handle it?
i guess i can make tags
okay i got flake8 to work, hacked up the jenkinsfile to run new lint commands like --v2 or whatever, so i think it is working fine, thanks all
h
W00t! We're encouraging people to upgrade to v2 where possible, and once we have Scala support in v2 you should be able to completely switch over
But we don't know when that will be exactly yet
p
yeah, im eager to keep upgrading. any idea on the general timing around that? i know that’s hard to come up with lol
5 Views
Open in Slack
PreviousNext
Powered by