https://pantsbuild.org/ logo
Join Slack
Powered by
FYI y'all I just tried `PY=<PBS path> pants ...
# development
b
FYI y'all I just tried 
PY=<PBS path> pants help
and it works. So I think we should: • Pin our development lockfile to 
--only-binary
(to workaround sdist issues with PBDS) • Have 
scie-pants
in the Pants repo provide Python Could be helpful.
CC @happy-kitchen-89482 @witty-crayon-22786
Well fuck me, 
hdrhistogram
is sdist-only C code: https://pypi.org/project/hdrhistogram/#files
f
I guess we can: • Replace/remove it • Build a wheel for it and keep it in another index • Vendor it
b
That means that our own plugins go against the PBS-provided model for Pants 😬 I'm not 100% sure how we're avoiding disaster other than happenstance (caching or something else)
f
our own plugins are mostly pure python though... so why does it matter?
b
We suggest users add it to their plugins if they use 
--stats-log
f
also how exactly are sdists bad for PBS ?
b
(there used to be a note on the docs site, but now I'm not seeing it)
l
I've documented my findings here: https://github.com/pantsbuild/pants/pull/19585
b
OK I went so far as to open a PR against their CI to now upload wheels for most OS/Python versions: https://github.com/HdrHistogram/HdrHistogram_py/pull/47
l
Cool
b
Unfortunately that project doesn't see much activity it seems 😕
(We could always publish the wheels against a different name from a fork)
l
I don't really understand what the problem with Standalone Python is. If that thing requires clang to be installed, then it's going to need that regardless of which Python interpreter you're using, right?
b
PBS doesn't require clang to be installed. It's plug-and-play. But Python in 
siteconfig
module hardcodes paths/flags for what itself was compiled using. Some sdists (rightfully) use those paths/flags when compiling modules so they are compatible with the Python you're building for. But that means the Python you downloaded has paths specific to the docker container (or whatever) it was built in, not your system
l
Oh, so the hdrhistogram thing might not work?
b
If you got it to compile, it almost certainly will
But you had to install 
clang
which is against the spirit of the whole thing 😛
l
Yeah, but every other Python interpreter would have the same problem, right?
b
No, those are usually compiled specifically to roughly match your distro/OS (as I understand it)
l
But I'd still have to install clang…
Anyway, I've documented what works for me, now you can decide if you want to merge that or not 😉
b
Well, the point of this thread is that 
pants
would provide PBS, and nothing would need to be installed 🙂 (minus that protobuf thing from the other thread)
l
Honestly I think that this clang thing isn't so bad.
recall the other problem that I've had, where it couldn't find libstdc++.so.6
b
No, 
hdrhistogram
is fairly vanilla. Something like 
numpy
goes boom more spectacularly 🙂
l
That suggests to me that some of the packages include pre-compiled C++ code that expects that library to be in a specific location.
And because Nix uses a different directory layout it failed
that's IMO a much nastier problem
I'd rather have it download and compile that C++ source code too
But what do I know 😅
@bitter-ability-32190 @flat-zoo-31952, I actually got this whole thing working in a nix shell now.
Oddly enough, it always fails on the first attempt and I get this error:
Copy code
native_engine.IntrinsicError: Could not identify a process to backtrack to for: Missing digest: Was not present in the local store: Digest { hash: Fingerprint<faad51a6a108fba9d40b2a10e82a2646fccbaf8c3d9be47818f4bffae02d94b8>, size_bytes: 4098329 }
But then I just run 
./pants
again and then on the second attempt it works.
b
(FYI The owner of 
hdrhistogram
replied to my PR, so we might be in business 😛)
👍🏻 1
We have wheels! Huzzah for open source! https://pypi.org/project/hdrhistogram/#files
🎉 1
OK, so the holdouts from our deps list: • pytest ◦ 
pytest-icdiff
(filed issue) ▪︎ Made a PR ◦ 
icdiff
(filed issue) ▪︎ Made a PR ◦ (we could use a different diffing plugin anyways, it's been suggested and I personally don't like 
icdiff
) • python-default ◦ 
pydevd-pycharm
▪︎ can probably make this an optional dependency. Probably should? ▪︎ No issue tracker on GitHub 😠 ▪︎ LOL their PyPI description mentions uploading wheels ▪︎ Found an issue: https://youtrack.jetbrains.com/issue/PY-52338/Consider-providing-wheels-for-pydevd-pycharm ◦ 
python-multipart
▪︎ Has wheels for 
0.0.6
, but 
fastapi
pins this to 
<0.0.6
▪︎ So we likely can re-lock with newer fastapi and see what happens • EDIT: Yup this worked after I tossed `pydevd-pycharm
h
I do find pydevd-pycharm very useful when debugging Pants, FWIW
b
You don't use the 
--debug-adapter
+ VS Code support? 😛
h
Super annoying that it's not a wheel
b
(It's a no-change-required) way of debugging your code
h
switching to VS Code is "no change required"???
b
As in no changing your code. No imports, no 
breakpoints
just plug-and-play
Found an issue for 
pydevd-pycharm
, but its 2 years old 🪦
Open in Slack
PreviousNext
Powered by