https://pantsbuild.org/ logo
#general
Title
# general
r

ripe-gigabyte-88964

04/07/2023, 4:45 PM
So I've finally got to the point in my plugin development where I'm getting exploding rule graph errors. Is it legal to use a field set as a hydration request? I thought it seemed convenient to have my field set inherit from my hydration request because it requires all of the same fields. My hydration rule also requires the subsystem and an
Address
. Do I need to define a rule telling pants how to get the subsystem and address out of my field set? I was under the impression it would already know how to do that.
w

wide-midnight-78598

04/07/2023, 4:45 PM
All I can say is that ... I feel your pain.
r

ripe-gigabyte-88964

04/07/2023, 4:46 PM
Basically, I have my package rule which takes a field set, subsystem, and union_membership
And then my hydration rule which takes my hydration request, subsystem, and address
w

wide-midnight-78598

04/07/2023, 4:46 PM
Posting the rule graph error -might- help (not necessarily though)
As well as your
rules
and
register
r

ripe-gigabyte-88964

04/07/2023, 4:46 PM
I think I'm doing something very wrog because there's like 60 rule graph errors
even though I'm only adding 2 rules
w

wide-midnight-78598

04/07/2023, 4:47 PM
I wouldn;t make any correlations between the number of errors and code size.
Each new plugin I build runs between 10-100 errors, and the fix is usually a 1-liner
r

ripe-gigabyte-88964

04/07/2023, 4:54 PM
if I separate out the field set and the hydration request, how do I provide other information from the field set to the request? Can I add the
address
and
subsystem
to the hydration request dataclass. Or can I get rid of the hydration request and just make a rule that maps directly from the field set (which has everything I need) to the hydrated dataclass?
w

wide-midnight-78598

04/07/2023, 4:55 PM
I'd probably need to see code to provide any useful feedback. It's a bit abstract to me otherwise
r

ripe-gigabyte-88964

04/07/2023, 5:31 PM
Just got it working! I simplified it and just went straight from the (field set, subsystem) -> hydrated_dataclass
and that worked 😂
👍 1
Is it possible to redirect stdout in a
Process
request? I'm getting an
IntrinsicError
when I try with the following error message:
Copy code
Error launching process: Os { code: 2, kind: NotFound, message: "No such file or directory" }
w

witty-crayon-22786

04/07/2023, 7:13 PM
processes don’t have a shell by default, and redirection is a shell concept. if you spawn a shell
bash -c '$original_process > somewhere'
you can redirect (located via
BashBinary
). but you might also consider capturing the stdout/stderr of your
ProcessResult
instead, and then creating a digest from it with
CreateDigest
🙏 1
🐐 1
r

ripe-gigabyte-88964

04/07/2023, 7:15 PM
Oh that's great, yeah I can just use
stdout_digest
! You literally just saved me like an hour of searching hahaha
So
CreateDigest
won't work in this context because it's part of a custom
package
goal implementation I'm writing. Is there a way to convert a
FileDigest
to a regular
Digest
?
w

witty-crayon-22786

04/07/2023, 7:47 PM
CreateDigest([FileEntry($filename, file_digest)])
r

ripe-gigabyte-88964

04/07/2023, 7:49 PM
If I use
CreateDigest
it complains because it says the filesystem has changed and restarts the goal.
w

witty-crayon-22786

04/07/2023, 7:49 PM
hm. that’s probably a panic due to misusing the API… those should be fixed in
2.17.x
something like
digest = await Get(Digest, CreateDigest([FileEntry($filename, file_digest)]))
should do it… can you share a snippet?
r

ripe-gigabyte-88964

04/07/2023, 7:57 PM
What I'm trying to do is laughably simple, just aggregating some configurations into a couple config files and trying to spit it out in the
dist
directory. So if I do this
Copy code
digest = await Get(Digest, CreateDigest([
        FileContent(final_location_file_path, yaml.dump(location_config)),
        FileContent(final_deploy_file_path, json.dumps(deploy_config))
    ]))
    return BuiltPackage(digest, (BuiltPackageArtifact(final_location_file_path), BuiltPackageArtifact(final_deploy_file_path)))
I get this
Copy code
15:57:21.15 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:21.65 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:22.16 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:22.66 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:23.17 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
15:57:23.67 [INFO] Filesystem changed during run: retrying `Package` in 500ms...
⠈ 
Interrupted by user.
w

witty-crayon-22786

04/07/2023, 7:58 PM
FileContent
takes
bytes
, but
json.dumps
will be a string: you’ll need to encode
😅 1
sorry about that. we’re working on cleaning up error messages… you might see it in
.pants.d/pants.log
currently, but it should be getting raised as an actual useful error instead.
🙏 1
r

ripe-gigabyte-88964

04/07/2023, 8:04 PM
Hahaha no worries, thank you for the assistance because I never would've figured that out otherwise!
is this the right way to create a pex from an entire lockfile?
Copy code
dagster_pex_request = await Get(VenvPexRequest, PexRequest(
            output_filename="dagster.pex",
            internal_only=True,
            requirements=EntireLockfile(await Get(Lockfile, Resolve(dagster_subsystem.resolve))),
            inject_env={"DAGSTER_ENVIRONMENT": "LOCAL_DEV"}
        ))
Getting errors downloading artifacts from this and not sure where to find these logs
Copy code
stderr:
There were 2 errors downloading required artifacts:
1. orjson 3.6.9 from <https://files.pythonhosted.org/packages/b0/ee/5e8bc274e5573cec50704615e3e57af1c12ba9549ffd6fb481402eda8e66/orjson-3.6.9.tar.gz>
    ERROR: Command errored out with exit status 1: /Users/nick.dellosa/.cache/pants/named_caches/pex_root/venvs/0898bf1dbd31a3bb8b1e505138b2c7ad5b8eae63/2d174fd7cbeab53a668fd7075373254c14e15f80/bin/python /Users/nick.dellosa/.cache/pants/named_caches/pex_root/venvs/0898bf1dbd31a3bb8b1e505138b2c7ad5b8eae63/2d174fd7cbeab53a668fd7075373254c14e15f80/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /Users/nick.dellosa/.cache/pants/named_caches/pex_root/pip_cache/.tmp/tmpjaorja4x Check the logs for full command output.
2. psycopg2-binary 2.9.6 from <https://files.pythonhosted.org/packages/98/3e/05ab0922422c91ca0ecb5939a100f8dc2b5d15f5978433beadc87c5329bf/psycopg2-binary-2.9.6.tar.gz>
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
w

witty-crayon-22786

04/14/2023, 5:55 PM
you’ll probably need to actually constrain your interpreter somewhere in the PexRequest
not sure which it will choose otherwise.