https://pantsbuild.org/ logo
Join Slack
Powered by
So I've finally got to the point in my plugin deve...
# general
r
So I've finally got to the point in my plugin development where I'm getting exploding rule graph errors. Is it legal to use a field set as a hydration request? I thought it seemed convenient to have my field set inherit from my hydration request because it requires all of the same fields. My hydration rule also requires the subsystem and an 
Address
. Do I need to define a rule telling pants how to get the subsystem and address out of my field set? I was under the impression it would already know how to do that.
w
All I can say is that ... I feel your pain.
r
Basically, I have my package rule which takes a field set, subsystem, and union_membership
And then my hydration rule which takes my hydration request, subsystem, and address
w
Posting the rule graph error -might- help (not necessarily though)
As well as your 
rules
and 
register
r
I think I'm doing something very wrog because there's like 60 rule graph errors
even though I'm only adding 2 rules
w
I wouldn;t make any correlations between the number of errors and code size.
Each new plugin I build runs between 10-100 errors, and the fix is usually a 1-liner
r
if I separate out the field set and the hydration request, how do I provide other information from the field set to the request? Can I add the 
address
and 
subsystem
to the hydration request dataclass. Or can I get rid of the hydration request and just make a rule that maps directly from the field set (which has everything I need) to the hydrated dataclass?
w
I'd probably need to see code to provide any useful feedback. It's a bit abstract to me otherwise
r
Just got it working! I simplified it and just went straight from the (field set, subsystem) -> hydrated_dataclass
and that worked 😂
👍 1
Is it possible to redirect stdout in a 
Process
request? I'm getting an 
IntrinsicError
when I try with the following error message:
Copy code
Error launching process: Os { code: 2, kind: NotFound, message: "No such file or directory" }
w
processes don’t have a shell by default, and redirection is a shell concept. if you spawn a shell 
bash -c '$original_process > somewhere'
you can redirect (located via 
BashBinary
). but you might also consider capturing the stdout/stderr of your 
ProcessResult
instead, and then creating a digest from it with 
CreateDigest
🙏 1
🐐 1
r
Oh that's great, yeah I can just use 
stdout_digest
! You literally just saved me like an hour of searching hahaha
So 
CreateDigest
won't work in this context because it's part of a custom 
package
goal implementation I'm writing. Is there a way to convert a 
FileDigest
to a regular 
Digest
?
w
CreateDigest([FileEntry($filename, file_digest)])
r
If I use 
CreateDigest
it complains because it says the filesystem has changed and restarts the goal.
w
hm. that’s probably a panic due to misusing the API… those should be fixed in 
2.17.x
something like 
digest = await Get(Digest, CreateDigest([FileEntry($filename, file_digest)]))
should do it… can you share a snippet?
r
What I'm trying to do is laughably simple, just aggregating some configurations into a couple config files and trying to spit it out in the 
dist
directory. So if I do this
Copy code
digest = await Get(Digest, CreateDigest([
        FileContent(final_location_file_path, yaml.dump(location_config)),
        FileContent(final_deploy_file_path, json.dumps(deploy_config))
    ]))
    return BuiltPackage(digest, (BuiltPackageArtifact(final_location_file_path), BuiltPackageArtifact(final_deploy_file_path)))
I get this
Copy code
15:57:21.15 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:21.65 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:22.16 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:22.66 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:23.17 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:23.67 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
⠈ 
Interrupted by user.
w
FileContent
takes 
bytes
, but 
json.dumps
will be a string: you’ll need to encode
😅 1
sorry about that. we’re working on cleaning up error messages… you might see it in 
.pants.d/pants.log
currently, but it should be getting raised as an actual useful error instead.
🙏 1
r
Hahaha no worries, thank you for the assistance because I never would've figured that out otherwise!
is this the right way to create a pex from an entire lockfile?
Copy code
dagster_pex_request = await Get(VenvPexRequest, PexRequest(
            output_filename="dagster.pex",
            internal_only=True,
            requirements=EntireLockfile(await Get(Lockfile, Resolve(dagster_subsystem.resolve))),
            inject_env={"DAGSTER_ENVIRONMENT": "LOCAL_DEV"}
        ))
Getting errors downloading artifacts from this and not sure where to find these logs
Copy code
stderr:
There were 2 errors downloading required artifacts:
1. orjson 3.6.9 from <https://files.pythonhosted.org/packages/b0/ee/5e8bc274e5573cec50704615e3e57af1c12ba9549ffd6fb481402eda8e66/orjson-3.6.9.tar.gz>
    ERROR: Command errored out with exit status 1: /Users/nick.dellosa/.cache/pants/named_caches/pex_root/venvs/0898bf1dbd31a3bb8b1e505138b2c7ad5b8eae63/2d174fd7cbeab53a668fd7075373254c14e15f80/bin/python /Users/nick.dellosa/.cache/pants/named_caches/pex_root/venvs/0898bf1dbd31a3bb8b1e505138b2c7ad5b8eae63/2d174fd7cbeab53a668fd7075373254c14e15f80/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /Users/nick.dellosa/.cache/pants/named_caches/pex_root/pip_cache/.tmp/tmpjaorja4x Check the logs for full command output.
2. psycopg2-binary 2.9.6 from <https://files.pythonhosted.org/packages/98/3e/05ab0922422c91ca0ecb5939a100f8dc2b5d15f5978433beadc87c5329bf/psycopg2-binary-2.9.6.tar.gz>
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
w
you’ll probably need to actually constrain your interpreter somewhere in the PexRequest
not sure which it will choose otherwise.
Open in Slack
PreviousNext
Powered by