Hi folks, I'm trying to make a plugin for

ansible-lint

, which requires the

ansible

script to be there too. I tried subclassing PythonToolBase and adding 'ansible==5.3.0' to the extra requirements, but it looks like it's not included in the PEX as I get "No such file or directory: 'ansible'" when running .

ansible-lint

. I'm not sure what question to ask. Is it possible to have a PEX with multiple binaries? Can I have a PythonToolBase depend/include another PythonToolBase? Is there a better way to do this? Any help is appreciated :)

Hello! What is your full class definition? I supsect you need to set

default_main = ConsoleScript("ansible")

well, problem is that I need to invoke

ansible-lint

, which then `subprocess.run`s

ansible

.

@careful-address-89803 are you using this with the tool you've defined?: https://github.com/pantsbuild/pants/blob/12d56a388c1d0ee02523d19c7e7ad6b2b9550c33/src/python/pants/backend/python/util_rules/pex.py#L704-L725

no, I've been using PexRequest. I'll give that a shot, thanks!

If that doesn't work, there is a Pex option for venv-mode PEXes which ensures all venv bin scripts are visible on the PATH but I'm not sure we set that or allow you to set it. Looking...

so the first option seems to have worked, thanks!

Really! Huh. OK

oh hm. So, ansible has code internally to locate things, I wonder if that's part of it?

The bin_names shim scripts are generated in the same dir and that dir is CWD of the process Pants invokes. Maybe CWD is on the PATH propagated through Pants or else ansible treats CWD specially?

digging into ansible-lint, it looks like it just runs

subprocess.run(args=["ansible", "--version"], ...)

. I can't find it monkeying around with the sys.path anywhere. Is there a quick way to test this, maybe with another command?

In general

--no-process-cleanup

is your friend when testing rule sandboxed processes. That will print out the paths to all the sandboxes Pants creates. Grab the path of the relevant rule fro the Pants run output, then cd there and poke around. Everything is exactly as executed by the Rust code except: 1. There is a

__run.sh

script that encodes how Rust will execute this sandbox. 2. That script is buggy in 1 way, it does not mask env vars like the rust code does, it lets your ambient environment through. You should be able to use this knowledge to hack on the

__run.sh

script and run it to see what happens. I'd 1st have it echo the

$PATH

and see if there are CWD (`:rest`or

.:rest

) style entries. If there are, I'd try running

PATH=... ./__run.sh

to exclude CWD and see if things still work.

well I learned a lot about python and pex today. It looks like it can find the

ansible

executable because python is adding the CWD to the path.

ansible-lint

is invoked directly via path to "${venv_dir}/bin/ansible-lint" with a shebang of a pex-venv-based python, it gets the CWD of "${venv_dir}/bin" added to its path, and that's the same place as ansible and other binaries are installed (including bins from other python dependencies!).

@careful-address-89803 Just to check, did you end up doing something like I did here with the vanilla ansible? https://github.com/sureshjoshi/pants-plugins/blob/4-prototype-ansible-plugin/pants-plugins/experimental/ansible/rules.py But, you'd be operating on sources, unlike that, which only needs 1 playbook

ah, actually, CWD is the wrong word there. I meant the directory of the file

🤯 An ansible-lint plugin!? Totally awesome. I've wanted to do that, but couldn't pull my head out of building some big features for ansible-lint 6.0. Not sure if that makes a difference to you, but 6.0 will be able to reformat YAML files and then 7.0 will allow rules to define a

transform

to fix (or at least improve) the issues the rule identified.

@careful-address-89803 Quick aside: I wrote this a couple weeks ago: https://sureshjoshi.com/development/first-pants-plugin Not sure if you're already past all that information, since you've been writing the plugin already - and that's targeted as a "getting started", but maybe there is something useful to you.

@proud-dentist-22844 Thanks for the pointers! and the path-addition thing being new in 5.4.0 explains why I didn't see it, I had ansible-lint checked out to my fork of ansible-lint so I could have a look at your reformatting MR. Which I still need to have another look at :$ I should probably test that this works with ansible-lint versions older than 5.4, then.

@careful-address-89803 Cool! I’m available here, and on Matrix/IRC in the Ansible DevTools channel. I welcome any feedback about my changes. And thanks for working on the ansible-lint plugin for pants. That would be superb for me to use and give me a way to (potentially) introduce pants at work. 😛

Finally had some time to look into this again. It looks like the ansible binary was only found because of the path-addition feature in ansible-lint 5.4, it isn't found prior to that version. Now, running a venv's python does add the venv's bin directory to the

sys.path

, which is the python module path. Not, as I thought, the system path for looking up commands (with

shutils

or

subprocess

). That's still the

os.environ

path variable, which is unmodified. So now I'm no longer confused about why things didn't work. I've going to try the other suggestions for getting multiple binaries playing together.

ansible-lint-6.0.0a0

pre release was released today. It includes the ability to reformat YAML files using

ansible-lint --write

. I would love feedback. So, if you find any issues with that or the PATH manipulation or whatever, we can get changes in before 6.0 final is released, prob in a couple weeks.

so I gave the suggestion by @enough-analyst-54434 to use the

--venv prepend

a shot. I hotwired it with a normal PexRequest with

additional_args=("--venv", "append")

, so it looks like plumbing that in to VenvPexRequest would work. I'm open to contributing, but I'm still getting around how Pants works, so I think I'd need guidance.

@careful-address-89803 how Pants works shouldn't get on your way on this one. It succumbs to more basic inspection:

$ git grep -n "\"\--venv\"" src/python/
src/python/pants/backend/python/goals/package_pex_binary.py:92:            args.extend(("--venv", "prepend"))
src/python/pants/backend/python/util_rules/pex.py:786:            "--venv",

You're interested in the 2nd hit. One option is to do like the 1st hit and just hard code

"prepend"

- I can't see any problems with this. The other option is the plumb. Looking at that line 786 there is a

VenvPexRequest

in scope (https://github.com/pantsbuild/pants/blob/94921a8edb9b142dff471784aef9da22b58cd0ab/src/python/pants/backend/python/util_rules/pex.py#L751-L786); so, if

VenvPexRequest

grew a new field carrying a prepend, append or None, you'd be done.