When placing a package with a compiled binary into...
# general
When placing a package with a compiled binary into
, Pants only copies the
files when running the test goal. So
import pkg
results in
being invoked and then an import error from
from .bin import *
is not in the sandboxed environment. Confirmed by dropping into pdb and looking at the folder structure in the execution env. Any ideas? Is that expected behavior or?
Where and what is the associated owning target?
I assume you have a BUILD file somewhere in that path?
But to skip to the point, if the target is
, that only globs (small lie)
by default; so you'll manually need to specify
globs: https://www.pantsbuild.org/v2.8/docs/reference-python_sources
Right, added a BUILD file after one of the dependents was not reporting it as a dependency. Then it was, but now this. So everything internal that we want Pants to consider as a source that other targets might depend on, definitely needs a BUILD with at least a python_sources target? And if that target has source files other than .py, we should put explicitly
sources=['*.py', '*.so', etc.]
Yes, exactly. Have you stumbled on the
goal for automating basic BUILD target seeding?
Yah, started playing with that. But still need to do a comprehensive read of all the goals/targets/subsystems; think it will clarify things later, but for now we are just trying to get a proof of concept!
Still not having any luck. This is my BUILD:
python_sources(description="fidlib", sources=["*.py", "*.so"])
; when I drop into the debugger I only see
; the binary
is missing. I tried explicitly listing it as well, no luck.
Found a clue in the warnings: `[WARN] Unmatched glob from 3rdparty/fidlib:fidlib's
field: "3rdparty/fidlib/*.so"` But if I grep the exact directory it's complaining with .so I see the file...
Ahha! Just realized
is a pattern in our
Ok. Does that fix things?
Ok so now it matches and tries, but getting this error:
[ERROR] The 'sources' field in target 3rdparty/fidlib:fidlib can only contain files that end in one of ['', '.py', .pyi'] but it had these files: ['3rdparty/fidlib/fidbin.so']
Ok, I personally find this stuff fiddly and confusing too. I think you need 2 targets. Like so: python_sources( name="fidlib", dependencies=[":so"], ) resources( name="so", sources=["*.so"], )
Ok, let me try that. I see the logic of that given this is more of a resource than a source file, but would have never thought to! So this just creates a new resources target w/ "./*.so" as sources on which other targets can depend (and by transitivity include their sources as their own)?
Alright that seems to work now; fingers crossed our firm let's us upload binaries to source control..
🤞 1
I see the logic of that given this is more of a resource than a source file
(Indeed, Idea being that you cannot do things like run Black or Flake8 on the
file. It's not a Python file, only a resource understood by Python)
Although that's true since a generic so is not python, a so side-by-side with Python code is almost undoubtedly an extension; so this boilerplate is a bit unfortunate. We'd need an ELF / mach-O parser though to divine all this properly.