https://pantsbuild.org/ logo
Join Slack
Powered by
Got another one that used to work, but is now brok...
# general
f
Got another one that used to work, but is now broken. After looking @ https://www.pantsbuild.org/2.18/docs/using-pants/assets-and-archives cant quite figure out whats wrong. We have a BUILD file, which is supposed to copy over some .json files, but they seem to not being copied over to the sandbox folder. In the folder that houses the seed.py file, the BUILD file contains 
python_sources()
c
I think you might need an explicit dependency link between the 
python_sources
and the 
resources
here.
the python backend can do some dependency inference for assets based on things that look like file paths, so it might have been detecting that before? I'm pretty sure we didn't change the behaviour of that though
f
Like this?
Copy code
python_sources(
    dependencies=[":data"],
)

resources(
    name="data",
    sources=["*.json"],
)
c
yep, that should work
f
No luck 🤔
c
just checking, are you expecting to load them with package utils/importlib.resources or as normal files?
also to confirm, when you say that the files aren't making it into the sandbox: Is that the sandbox with 
--keep-sandboxes=always
, or that you're getting FileNotFoundError or something like that? If they show up with 
--keep-sandboxes
but you still get errors saying it's not there, Python package resource loading might be the cause. The files have to exist in a python package for 
importlib
to load it. you might need to add a 
__init__.py
in the dir and a 
python_sources
target to grab it.
b
I wonder if the “use to work” difference is https://github.com/pantsbuild/pants/issues/20324 Do you set 
string_imports = true
in pants.toml? What version of pants?
f
string_imports = true
is not set. im using 2.18.0
no luck on --keep-sandboxes=always
c
resources
won't be able to be opened with 
open
, for that you'd want 
files
(but then also those files aren't included in python packages). I think the issue might be that you're using a filepath instead of a package in the call to 
importlib.resources.open_text
. You might need to try with something like
Copy code
import data.seed.common
f = importlib.resources.open_text(data.seed.common, "capability.json")
The error message you get is a little helpful, it can indicate whether the problem is about resolving the package itself or whether the package resolution suceeded but the resource didn't come with
using the 
--keep-sandboxes
arg will preserve the environment where Pants expanded the PEX to. You can then inspect it for whether the files showed up and it's an 
importlib
loading thing or whether they didn't show up at all.
f
Copy code
12:20:41.23 [WARN] Pants cannot infer owners for the following imports in the target src/data/seed/dev/seed.py:

  * src.data.seed.common (line: 7)

If you do not expect an import to be inferrable, add `# pants: no-infer-dep` to the import line. Otherwise, see <https://www.pantsbuild.org/v2.18/docs/troubleshooting#import-errors-and-missing-dependencies> for common problems.
^CTraceback (most recent call last):
  File "/private/var/folders/rq/g_3vlcwn3z99ssl__23dcwv40000gn/T/pants-sandbox-37YFRx/./.cache/pex_root/venvs/677e24f7e4da96ef93f1f1466b335cc5cb90c257/3ee58c3b3b7f54fee5790e90b78eab18840cf1a3/pex", line 274, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 225, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/folders/rq/g_3vlcwn3z99ssl__23dcwv40000gn/T/pants-sandbox-37YFRx/src/tasks/seed.py", line 11, in <module>
    from crud.profile import _update_profile, create_profile_from_dict
  File "/private/var/folders/rq/g_3vlcwn3z99ssl__23dcwv40000gn/T/pants-sandbox-37YFRx/src/crud/profile.py", line 31, in <module>
    from core.utils.powerpoint.template import ProfileMachinePPT
  File "/private/var/folders/rq/g_3vlcwn3z99ssl__23dcwv40000gn/T/pants-sandbox-37YFRx/src/core/utils/powerpoint/template.py", line 17, in <module>
    for root, dirs, files in os.walk("/"):
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/os.py", line 418, in _walk
    yield from _walk(new_path, topdown, onerror, followlinks)
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/os.py", line 418, in _walk
    yield from _walk(new_path, topdown, onerror, followlinks)
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/os.py", line 418, in _walk
    yield from _walk(new_path, topdown, onerror, followlinks)
  [Previous line repeated 6 more times]
  File "/Users/harsmith/.pyenv/versions/3.9.16/lib/python3.9/os.py", line 357, in _walk
    scandir_it = scandir(top)
KeyboardInterrupt
Interrupted by user.

make: *** [backend/run/seed/local] Error 1
Ive been different packages and i get the same message. When i use 
--keep-sandboxes
the files are missing from the folder
One sec, i'll shoot through what I think got it working
Just doing an extra test!
Copy code
python_sources(
    dependencies=[":data"],
)

files(
    name="data",
    sources=["*.json"],
)
In my BUILD file for common and adding init.py to common, did the trick
c
Oh! I was wondering if in the BUILD file "src/data/seed/dev/BUILD", the 
python_sources
target has the dependency link to "data" target. I was thinking that the dep chain might've not been working. I think the link you added might tie it together transitively
you can try poking around on the deps of each target with 
pants dependencies
for individual targets, and the 
--transitive
for all transitive deps too
4 Views
Open in Slack
PreviousNext
Powered by