Stylistic question: If I'm pulling in all files (regardless of extension) in a target, should I create a new target for that (i.e.

ansible_sources

), or just use a

files

or

resources

field? While ansible execution files are largely yaml, there can be any extension of files which are used as resources or references. There is a semantic difference to be had, but not a practical or functional one I feel. For example, I can do something like the following:

files(
    name="myansiblesources",
    sources=["**/*"],
)

or

ansible_sources(
    name="myansiblesources",
    sources=["**/*"],
)

ansible_deployment(
    name="helloansible",
    dependencies=[":myansiblesources"],
    playbook="playbook.yml",
    #inventory="", # TODO: Could this be setup like the Docker registries? in pants.toml
    #tags="", # TODO: Might need a different name to not overlap
    #timeout="",
)

Or, I could get even more specific by calling out the YAML files and then the supplementary files to help with formatting/linting I guess... In summary, I'm not sure about the API 🙂 I'm going forward to get something functional, but in terms of something maximally usable/pragmatic + pants-like

Maybe start off with just

ansible_sources

? That solution would not preclude you from introducing separate targets later (or having a field that references addresses of `files`/`resources` targets later).

I think a dedicated target makes sense, even if it relies on a regular files sources field behind the scenes.. that way there's more room for future changes with less BUILD file impact..

Okay, I'll start with that route then. Feels a bit, dunno, API creepy. Would the recommendation be for the sources to just be YAML code + another files target? Or anything/everything I need for a standalone ansible target?

There is a wide variety of stuff used with ansible. When working with

ansible-lint

, for example, you need to pass a list of playbooks and/ore roles, not raw tasks or vars files. Some content types to think about: •

role

(a directory with a certain structure that includes yaml files in certain paths (but would not include something like linter config in the root of the role) •

playbook

yaml and some directories of yaml files alongside it (

tasks

,

group_vars

, …) •

modules

/

plugins

are python code, so they’ll probably need to be covered by python targets that are depended on by your ansible_sources targets. •

collection

can include all other kinds of ansible content in a certain structure •

inventory

can be yaml, ini, an executable script, or a directory of inventory files. If you need ideas here’s the list that ansible-lint uses (sadly it doesn’t support collections yet): https://github.com/ansible/ansible-lint/blob/main/src/ansiblelint/config.py#L8-L35

@careful-address-89803 has been working on ansible-lint, and in my case, I just opened up what I'm using internally - because my internal stuff is pretty straightforward. Playbook that calls out some custom roles, and boom - done. Some of these will be target-level, while some will need to be top-level configurations. I don't know which one should be which yet. At the moment, I'm going to get something as reasonably trivial up and running so the pipes are in place, then we can really get down to an API. I anticipate a lot of commands will be passed in via

args

otherwise the target API just balloons. My immediate thought is that, we can pull in `files`/`resources` /`ansible_sources` or whatever as dependencies, and let regular ansible take over as much as possible. The target will mostly be about configuring the

deploy

command

exciting stuff!

If you have any suggestions on any tests that would be appropriate, I'd love to hear. Right now, just deploying to my VMs with some projects, but that's not very open-sourceable or CIable

I had a separate sources type called [AnsiblePlayContext](https://github.com/lilatomic/pants-plugins/blob/99cf85079de36bce00db81e37c8e72a813b0f49f/pants-plugins/experimental/ansible/sources.py#L66) which pulls in all those files. That source type is part of an ansible_deployment, defaulting to what ansible should be able to reach from that directory. Other targets, like Collections and Roles, also just pull in whatever they theoretically can, but don't use the AnsiblePlayContext because they're different. I think there's room for refinement, especially since there are different sources types (files vs templates vs ansibles). Also I have yet to look into target generation, which sounds like it might be useful for this. Unfortunately, there isn't a way with Ansible to infer dependencies without executing (think

copy

with

src: "{{ service_name }}.service"

), and there's nothing stopping

src: "/home/me/.ssh/id_ed25519.pub"

. Hot take, maybe we shouldn't support that, and enforce a more constrained structure.

In order to not get too bogged down, I went in the most trivial direction I could: https://github.com/sureshjoshi/pants-plugins/blob/main/pants-plugins/experimental/ansible/target_types.py Essentially, pull in the directory (or whatever files/directories you'd like to glob) and let ansible run wild

I have a huge repo of playbooks and roles that I'm trying to get under control. Each of the playbooks (or directory of playbooks) would have its own set of requirements. A ton of people as stuff to that repo, app it ends up being a "mono repo" of Ansible content with lots of different and conflicting requirements.

Hmm, interesting. That first case sounds pretty complicated, and not specifically what I had in mind when starting this. In this instance, I was working more on a linear workflow basis (where an ansible deploy is the final step) For me, I have monorepos full of code, and then I keep most of my custom ansible roles/collections elsewhere - and then have a playbook that can deploy on a per-project basis (whatever "deploying" is for that project). So, provisioning and configuring a server might be elsewhere (or I guess part of a monorepo, situationally), but then the part that pulls my docker containers together, or pex files, or whatever together is located in the repo. For one of my typical projects of a certain size or scale, it just feels like overkill to put the devops stuff elsewhere.

Another use case for open source collection repos. I can also imagine using this in ansible collection repos like https://github.com/ansible-collections/community.mongodb and https://github.com/ansible-collections/community.rabbitmq and https://github.com/cognifloyd/st2_installer.ansible where I would use it

pants

as an entry point for running lint/fmt (

ansible-lint

) test (

molecule

) and package (

ansible-galaxy collection build

) and possibly deploy (

ansible-galaxy collection publish

). (I'm one of the maintainers for these repos) For these repos, I can't imagine a time when I would have

pants

actually run a playbook.

This is similar the the position that the experimental Pants terraform backend takes. Pants can do

fmt

/

lint

but leaves applying plans to

terraform

and not Pants.

I noticed that @fast-nail-55400 - I wasn't sure if it was "intentional" or if the terraform backend just took a backseat while some other stuff was being developed.

I had actually implemented it in https://github.com/pantsbuild/pants/pull/13023 but decision was made not to land it.

Interesting... This actually begs the question if there is another mechanism to allow Pants plugins without being mainline? Like a

pip install dopeplugin

kinda thing. I'm re-vamping a bunch of repos, and a lot of plugins I'm writing have absolutely no utility to the population, but are important for me across projects and repos I would assume that functionality should be possible, as it's just registering another plugin

You could just distribute on PyPi. That is what we do for the Toolchain Pants plugin.

Whoa, didn't even know that was a thing! Thanks!

I think it makes sense to pull in collections on a per-playbook basis. My primary usecase would be local collections. EI, the same repo has a collection for deploying an application, and several playbooks/entrypoints for deploying staging/prod or app0/app1. So it's necessary to know to build those when deploying a target.

So, the collections per-playbook is for Galaxy mostly. Local stuff should be pullable regardless, as it's in the hierarchy

oh right, yeah, nevermind then. For python, I think Pants just uses a single universe for all dependencies, so it should be fine to put Galaxy dependencies in the root as well? On the other hand, like Jacob Floyd mentioned, sometimes you have a monorepo with playbooks which depend on conflicting versions of a Galaxy collection, so having [multiple resolves](https://www.pantsbuild.org/docs/python-third-party-dependencies#warning-multiple-versions-of-the-same-dependency) would be necessary for that use case

For python, I think Pants just uses a single universe for all dependencies, so it should be fine to put Galaxy dependencies in the root as well?

Now with multiple resolves, you have a universe per resolve

I feel like I'm not understanding the multiple resolves, probably because it's not something I need to use - but TLDR that you can have conflicting versions of a dep per source root?

Not necessarily per source route, you can define however many resolves make sense for your repository. And one single source route might use multiple resolves. https://blog.pantsbuild.org/multiple-lockfiles-python/ I'm making a tech talk this week about python dependency management in mono repos. I'm planning on sharing it on YouTube as well so that the pants community can see it

Okay, similar to what I had assumed just based on the name - but not necessary per unit of code that I originally thought. Sweet. Maybe I'll use that some day 🙂

@hundreds-father-404 you’ll have to teach Siri to say “root” rather than “route” 😄

If you can get away with a single resolve, that is usually the best way to go. Less boiler plate to deal with and less complexity overall