Edit: Solved again by using my eyes How can I add...
# general
b
Edit: Solved again by using my eyes How can I add VCS
extra_requirements
to a Pants thirdparty lockfile? I'm waiting on upstream PRs to merge, so in the meantime I'd like to use my branch so I'm not stuck. But setting
git+<https://github.com/>....
results in an error.
h
Are you trying to generate a lockfile with it in particular? If so, we're trying to figure this out. VCS requirements do not work with lockfiles atm https://github.com/pantsbuild/pex/issues/1556 The workaround for now is to set that tool's
lockfile = "<none>"
:/
b
Oh yeah, I see now the lockfile gets generated but not with what I expect
Alternatively I'm OK with being a bad person and monkeypatching the code, but:
If you want to write first-party plugins for other linters like Flake8, let us know on Slack.
h
To clarify, that plugin note is about writing plugins like
flake8-2020
for example, but consuming via first-party sources rather than third-party distribution. Is that what you mean?
b
Yeah, I could "cheat" here and make a "plugin" for
flake8
which when loaded monkeypatches the relevant code
👍 1
(I actually do the monkeypatching today, since our current build tooling also disallows VCS reqs. But I'm invoking
flake8
manually via python)
h
Got it. Are you able to work around this all today by disabling the lockfile and using a VCS requirement? Ack that it's suboptimal to not be able to use a lockfile. (Feedback welcomed on https://github.com/pantsbuild/pex/issues/1556 if you have any)
b
What's the side-effect of not using a lockfile?
h
More risk your build will break overnight if a transitive dep changes, and more risk for supply chain attack. You can very hackily recreate a lockfile by pinning every transitive dep in
[flake8].extra_requirements
...that will remove the risk of things breaking overnight. But it does not help with supply chain because of
--hash
not working.
b
🤔 works for me I think, will try out first thing tomorrow
👍 1
(I was already pinning transitive deps, to make migrating have the fewest hiccups)
Running
lint
with
flake8
after this change still shows unexpected errors... I'll try with
--no=...-cleanup
stuff
👀 1
Yeah I'm still seeing the PyPI version in the PEX
Copy code
[flake8]
version = "flake8==4.0.1"
lockfile = "<none>"
extra_requirements.add = [
    "darglint@ git+<https://github.com/thejcannon/darglint@master#egg=darglint>",
    ...
I might still want to make a fake monkeypatch plugin, honestly. We lint test files with less error codes, and I'm not sure flake8 can handle that natively or if there's a plugin for it 😭
h
Hmmm I'm surprised it's not using the Git version...if you use
--no-process-cleanup
and then
unzip
to look at
PEX-INFO
, it's showing the PyPI version? That would be a bug if so
b
That's what I'm seeing 🤔 Let me double-check when I'm back on that branch
Must have been a red herring 🤔
👀 1
But I'm still probably going to need a first-party plugin for monkeypatching tests error code filtering
(red herring was I was specifying the wrong branch 🤦‍♂️ )
👍 1
Looking at https://flake8.pycqa.org/en/latest/user/configuration.html#using-local-pluginslake8 looks like flake8 local plugins might not be too different from pylint
🚀 1
h
awesome! contribution definitely welcomed, should be able to copy pasta a bit from Pylint. Read the help string in
pylint/subsystem.py
, the weird restrictions on how to set up PYTHONPATH/source roots was definitely the most confusing part when I added this all last year
b
Hehe I'll add it to my list of things I need to contribute 😈
h
s/need/want to 🙂 never an expectation you contribute a particular thing
b
The need isn't fro y'all. It's from my dayjob 🤓
1
e
I think the Pex issue is a bit off base since we aren't using Pex for this right now. I was lazy and did not file an issue, but this comment explains what happens when you use a vcs direct reference requirement today: https://github.com/pantsbuild/pants/issues/13965#issuecomment-1000483186 You get a silent failure to lock what you asked for and get a lock on the latest public version instead - if there is one.
The bug is, specifically, demonstrated by:
Copy code
$ pex setuptools -- -c 'from pkg_resources import Requirement; req = Requirement.parse("darglint @ git+<https://github.com/thejcannon/darglint@XYZ>"); print(f"req: {req} spec: {req.specifier}")'
req: darglint@ git+<https://github.com/thejcannon/darglint@XYZ> spec:
And these lines of our code: + https://github.com/pantsbuild/pants/blob/315dd5c37a3e3394dee363e232d3c516583d4ead/src/python/pants/backend/python/subsystems/poetry.py#L97 + https://github.com/pantsbuild/pants/blob/315dd5c37a3e3394dee363e232d3c516583d4ead/src/python/pants/backend/python/subsystems/poetry.py#L107
IMO we should be failing fast for this case today since we can't do what you asked.