How does Pants decide how many jobs to run for parallel invocations? I'm running

./pants lint ::

with the

flake8

and

pylint

backends enabled, and it seemed to have run flake8 massively parallel, and pylint in just one job. Is this an intentional decision due to bugs in pylint?

which version of Pants?

2.9.0

2.9.0 doesn’t do much to control parallelism in those tools, but 2.10.x will: https://github.com/pantsbuild/pants/pull/14186 and https://github.com/pantsbuild/pants/pull/14184 are in there

that's fine, I can skip the pylint backend and just pass the results of a

--changed-dependees=transitive

call to pylint

in the meantime, you can explicitly pass concurrency settings to linters: see https://github.com/pantsbuild/pants/issues/13462#issuecomment-1007504648 …

pylint

in particular is insanely slow: like 10x of

mypy

even

pylint

looks at source code contents to infer types, whereas

mypy

has strict rules and uses annotations

Yeah... I'm not wild about pylint in general, but we use it and I'm not ready to fight that battle yet

Same 😐

should be faster anyway when I can run it on changes rather than on the whole repo

(It's still slow as hell 🙂 )

I'm a bit more broad, I'm a static analysis skeptic in general 😅

FWIW I plan on finding the

pylint

rules that

mypy

has basically made impotent and turning them off. I can share findings here. I would hope tossing those out would make it much speedier 🤞

I think it's all a big excuse for devs to avoid having to write more testable code

That's just sucky devs. Don't hate the game, player.

It's good devs too. A lot of very talented devs don't consider testing as part of their design because their skill makes them overconfident. And I think they've looked at enough overly mocked tests and assumed that you can't test anything important with unit tests. So you end up with a bunch of code that's impossible to test, and everyone's afraid to refactor, and assume that tacking on linters and typecheckers will fix the problem. And they do a little bit, but in many cases I'm not convinced the cost is worth it

Those aren't "good" devs in my book 😉

I'll give that a try when 2.10 comes out. For now I think I can do something like

./pants --changed-since=$CMP_BRANCH --changed-dependees=transitive list \
  | xargs ./pants filter --target-type=python_source \
  | sed -r 's,(^//|:[^:]+$),,g' \
  | xargs pylint $PYLINT_OPTIONS

We're hoping for first release candidate to be this week

OK These are seemingly all handled by mypy:

# E1700 (yield-inside-async-function)
# E0102 (function-redefined) [no-redef]
# E0103 (not-in-loop)
# E0104 (return-outside-function)
# E0105 (yield-outside-function)
# E0107 (nonexistent-operator)
# E0108 (duplicate-argument-name)
# E0110 (abstract-class-instantiated)
# E0112 (too-many-star-expressions)
# E0113 (invalid-star-assignment-target)
# E0114 (star-needs-assignment-target)
# E0119 (misplaced-format-function) [attr-defined]
# W0111 (assign-to-new-keyword)
# W0124 (confusing-with-statement)
# W0143 (comparison-with-callable)
# W0199 (assert-on-tuple)
# E0202 (method-hidden)
# E0203 (access-member-before-definition)
# E0211 (no-method-argument)
# E0213 (no-self-argument)
# E0237 (assigning-non-slot)
# E0238 (invalid-slots)
# E0239 (inherit-non-class)
# E0240 (inconsistent-mro)
# E0241 (duplicate-bases)
# R0203 (no-staticmethod-decorator)
# R0206 (property-with-parameters)
# W0221 (arguments-differ)
# W0222 (signature-differs)
# E0702 (raising-bad-type)
# E0703 (bad-exception-context)
# E0710 (raising-non-exception)
# E0712 (catching-non-exception)
# W0711 (binary-op-exception)
# W0716 (wrong-exception-operation)
# E1507 (invalid-envvar-value)
# W1506 (bad-thread-instantiation)
# E1101 (no-member)

# E1102 (not-callable)
# E1111 (assignment-from-no-return)
# E1120 (no-value-for-parameter)
# E1121 (too-many-function-args)
# E1123 (unexpected-keyword-arg)
# E1124 (redundant-keyword-arg)
# E1125 (missing-kwoa)
# E1126 (invalid-sequence-index)
# E1127 (invalid-slice-index)
# E1128 (assignment-from-none)
# E1129 (not-context-manager)
# E1130 (invalid-unary-operand-type)
# E1131 (unsupported-binary-operation)
# E1132 (repeated-keyword)
# E1133 (not-an-iterable)
# E1134 (not-a-mapping)
# E1135 (unsupported-membership-test)
# E1136 (unsubscriptable-object)
# E1137 (unsupported-assignment-operation)
# E1138 (unsupported-delete-operation)
# E1139 (invalid-metaclass)
# E1140 (unhashable-dict-key)
# E1141 (dict-iter-missing-items)
# I1101 (c-extension-no-mem)
# W1113 (keyword-arg-before-vararg)
# W1114 (arguments-out-of-order)
# E0604 (invalid-all-object)
# E0611 (no-name-in-module)
# E0633 (unpacking-non-sequence)
# W0601 (global-variable-undefined)
# W0632 (unbalanced-tuple-unpacking)

cool! how did you figure that out?

Just copied the example code from https://vald-phoenix.github.io/pylint-errors/ into a file and ran

mypy

on it. Rinse Repeat.

neat, that would be super cool to write a blog showing this. not necessarily Pantsbuild blog, just in genera

I guess I need a blog