https://pantsbuild.org/ logo
Join Slack
Powered by
Hey everybody, I’m trying to build a plugin to de...
# plugins
a
Hey everybody, I’m trying to build a plugin to deploy to Google Cloud Run. I want to add a new target 
google_cloud_run
and a 
publish
goal. The target part is pretty easy. I’m now stuck trying to extend the publish goal.
Copy code
./pants list publish
18:00:44.82 [INFO] Initializing scheduler...
18:00:44.90 [INFO] Scheduler initialized.

  * docker_image
  * google_cloud_run
It looks like the goal is recognized. However, if I try to run it, nothing happen.
Copy code
./pants publish ./packages/hello_world:cloud_run
18:08:36.24 [INFO] Initializing scheduler...
18:08:36.31 [INFO] Scheduler initialized.
I’m pretty sure I’m missing something silly and important here -_- I tried to look at the Docker Goal but that’s pretty much where I’m stuck. Simple repo to repro here ps: Thanks @wide-midnight-78598 for the tutorial, very helpful! Especially the “empty required_fields” trick -_-
👍 1
f
In `rules`method, you need 
UnionRule(PublishRequest, PublishGoogleCloudRunServiceRequest)
never mind, that is what 
.rules()
does
it only imports rules from 
rules.py
you would also need to import rules from 
goals.py
as well
the issue being that the UnionRule’s generated by the call to 
PublishGoogleCloudRunServiceFieldSet.rules()
are not being registered because the rules from 
goals.py
are not being added to the list in 
register.py
a
On 
packages/pants-plugins/google_cloud_run/rules.py
Copy code
from google_cloud_run.goals import rules as google_cloud_run_goal_rules

def rules():
    return [*collect_rules(), *gcloud_binary_rules(), *google_cloud_run_goal_rules()]
it I’m correct, it means that the 
rules
from 
goals
are added
and if they were not, I suppose that 
./pants publish ./packages/hello_world:cloud_run
would fail entirely?
f
huh at the link I posted I see:
Copy code
from google_cloud_run.rules import rules as google_cloud_run_rules
from google_cloud_run.target_types import GoogleCloudRunTarget


def target_types():
    return [GoogleCloudRunTarget]


def rules():
    return [*google_cloud_run_rules()]
ah you are right, rules.py imports it
is pantsd running?
can you try running the command with 
--no-pantsd
in front? My thought is maybe a result from a prior iteration of the code got cached.
and do you get any more interesting log output if you run with 
-ldebug
?
and if they were not, I suppose that 
./pants publish ./packages/hello_world:cloud_run
would fail entirely?
yes, I only was reading some of the files 😞
a
can you try running the command with 
--no-pantsd
in front? My thought is maybe a result from a prior iteration of the code got cached.
Same, nothing happend
and do you get any more interesting log output if you run with 
-ldebug
?
Sort of!
Copy code
22:05:00.24 [DEBUG] acquiring lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x1056a98b0>
22:05:00.24 [DEBUG] terminating pantsd
22:05:00.24 [DEBUG] sending signal 15 to pid 58592
22:05:00.35 [DEBUG] successfully terminated pid 58592
22:05:00.35 [DEBUG] purging metadata directory: /Users/quentin/dev/data/pants-flask/.pids/81c302b706a0/pantsd
22:05:00.35 [DEBUG] Launching pantsd
22:05:00.35 [DEBUG] purging metadata directory: /Users/quentin/dev/data/pants-flask/.pids/81c302b706a0/pantsd
22:05:00.35 [DEBUG] pantsd command is: PANTS_DAEMON_ENTRYPOINT=pants.pantsd.pants_daemon:launch_new_pantsd_instance PYTHONPATH=/Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/bin:/opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python39.zip:/opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9:/opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload:/Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/lib/python3.9/site-packages /Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/bin/python /Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/2.11.0_py39/bin/pants --pants-bin-name=./pants --pants-version=2.11.0 publish ./packages/hello_world:cloud_run -ldebug
22:05:01.07 [DEBUG] pantsd is running at pid 60660, pailgun port is 62365
22:05:01.07 [DEBUG] releasing lock: <pants.pantsd.lock.OwnerPrintingInterProcessFileLock object at 0x1056a98b0>
22:05:01.07 [DEBUG] Connecting to pantsd on port 62365
22:05:01.08 [DEBUG] Connecting to pantsd on port 62365 attempt 1/3
22:05:01.08 [DEBUG] Connected to pantsd
22:05:01.09 [DEBUG] Launching 1 roots (poll=false).
22:05:01.09 [DEBUG] computed 1 nodes in 0.006146 seconds. there are 7 total nodes.
22:05:01.15 [INFO] Initializing scheduler...
22:05:01.15 [DEBUG] File handle limit is: 1048575
22:05:01.22 [DEBUG] Changes to /Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/bin, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] Changes to /Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/bin, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] Changes to /opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python39.zip, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] Changes to /opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] Changes to /opt/homebrew/Cellar/python@3.9/3.9.12_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] Changes to /Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/pants.G0ob8P/install/lib/python3.9/site-packages, outside of the buildroot, will not be invalidated.
22:05:01.22 [DEBUG] setting up service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x103930fd0>
22:05:01.22 [DEBUG] setting up service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x105074cd0>
22:05:01.22 [DEBUG] starting service <pants.pantsd.service.scheduler_service.SchedulerService object at 0x103930fd0>
22:05:01.22 [DEBUG] starting service <pants.pantsd.service.store_gc_service.StoreGCService object at 0x105074cd0>
22:05:01.22 [INFO] Scheduler initialized.
22:05:01.22 [DEBUG] Launching 1 roots (poll=false).
22:05:01.22 [DEBUG] computed 1 nodes in 0.000123 seconds. there are 7 total nodes.
22:05:01.26 [DEBUG] specs are: Specs(address_specs=AddressSpecs(literals=(AddressLiteralSpec(path_component='packages/hello_world', target_component='cloud_run', generated_component=None, parameters=FrozenDict({})),), globs=(), filter_by_global_options=True), filesystem_specs=FilesystemSpecs(file_includes=(), dir_includes=(), ignores=()))
22:05:01.26 [DEBUG] changed_options are: ChangedOptions(since=None, diffspec=None, dependees=<DependeesOption.NONE: 'none'>)
22:05:01.26 [DEBUG] Launching 1 roots (poll=false).
22:05:01.26 [DEBUG] computed 1 nodes in 0.000823 seconds. there are 29 total nodes.
22:05:01.26 [DEBUG] requesting <class 'pants.core.goals.publish.Publish'> to satisfy execution of `publish` goal
22:05:01.26 [DEBUG] Launching 1 roots (poll=false).
22:05:01.26 [DEBUG] Completed: Find targets from input specs
22:05:01.26 [DEBUG] Completed: `publish` goal
22:05:01.26 [DEBUG] computed 1 nodes in 0.002980 seconds. there are 62 total nodes.
it just seems to be calling the based 
published
not the one from the plugin…
c
@agreeable-oyster-28981 Hi! You’ve got everything right so far, from what I can see (looks great too!) The final missing piece is, that the publish goal has an assumption that the target being published, is also the target doing the packaging, which is not the case here. This is the missing hint you were looking for: https://github.com/pantsbuild/pants/blob/47b9f4fb7db28474de48e4c6fa3df95354c030dd/src/python/pants/core/goals/publish.py#L207 Sorry for the trouble finding it, and also, perhaps that assumption could be tweaked somewhat..
a
Yup, seems to be working. Thanks @curved-television-6568. @fast-nail-55400 the 
-ldebug
log level has been pretty helpful too. 🙏 What would be the best way to deal with that? (Other than tweaking the existing logic) Should I have a “Dummy” packaging that does nothing? or try to reuse the Docker packaging function?
c
I'd go with a dummy, that simply calls the docker packaging for each dependency.
a
I was actually thinking even more dummy than this 😄 https://github.com/qventura/pants-playground/blob/quentin/gce-plugin/packages/pants-plugins/google_cloud_run/goals/package.py I’ll move forward with this, as it seems that it unlocked me, as I now have an error coming from GCloud binary I’ll move back to it later, as I do see some value about being consistent with other targets. thanks for the help, I’ll ask another question if I’m other issues .
c
Cool, and please do.
I realize the reason for it being ad it is may be to be able to guarantee the thing to publish is up-to-date.
That is, it should be enough to publish a target, and it gets built if needed.
a
yeah I think it makes sense.
2 Views
Open in Slack
PreviousNext
Powered by