hey. any ideas how to you get the dependencies for...
# general
f
hey. any ideas how to you get the dependencies for a target? From the code it looks like this information is available at the target level, but retrieving a target depends on the build graph and I do not know how to get this information?
Copy code
from pants.backend.python.targets.python_target import PythonTarget
from pants.build_graph.build_graph import BuildGraph
from pants.build_graph.address import Address,

# How to get dependencies?
address = Address(lib_path, lib_name)     
# target = graph??.get_target(address)
# target = PythonTarget(name=name, address=address, build_graph=build_graph???)
dependencies = target.dependencies()
h
cc @hundreds-father-404 can you answer this? thanks.
h
Hey Adrian, what Pants version are you using? Those are types from the old v1 engine and have been removed in Pants 2.0. With the Rules API, you get the
Dependencies
of a target like this: https://www.pantsbuild.org/v2.0/docs/rules-api-and-target-api
Are you hoping to use that dependencies code in your
ContextAwareObjectFactory
for
scm_setup_py
? If so, that unfortunately won’t work.
Object
and
ContextAwareObjectFactory
are both legacy APIs which don’t have access to the Rules API. This speaks to what I think Benjy realized with you last week - we’ll need the Rules API for the power that you’re looking for. My main priority today is helping to land that.
f
I am currently using latest stable version 1.30, but can switch to any version if needed
Yes. I want to use this in my versioning function. I realized that my scm versioning subtree implementation needs to take into consideration dependencies in order to work properly ( when only a child changes, the parent version must increase In order to build a new version)
👍 1
I am open to any changes you suggest. Thank you
h
I'm wondering, are you looking at scm versioning just to figure out if any inputs to the dist have changed? Because Pants already has that invalidation logic, it knows exactly what inputs went into producing a dist, and we could possibly expose a fingerprint that represents that.
f
So I am using a versioning per library using git subtrees. My code is described here https://github.com/pantsbuild/pants/issues/10633#issuecomment-677889311 . What I want to achieve is a way to increment versions in my python libraries based on code changes. I do not want to manually increment the version in the BUILD file.
During testing the above versioning function I've hit a case where when you change lib b (lib a -depends->lib b), lib b gets a version increase, but lib a gets the same version number (because of the git subtree logic), but it point to the new version of lib b in in the install_required section of setup.py which is dangerous (you could potentially overwrite the artifact). So I am trying to add a logic of dependencies inside the versioning function, something like: when lib b changes, lib a version changes as well.
h
So I may trying to add a logic of dependencies inside the versioning function
Cool. The Rules API has a prebuilt utility to do this, find the “dependees” of any file. For example, run
./pants --changed-since=HEAD list
. From there, you can filter out any irrelevant dependees that don’t have a
setup_py
.
f
I chose git subtrees logic for versioning instead of maintaning a versioning file in the repo for multiple reasons: git is the single point of truth and from what @hundreds-father-404 told me doing file writes inside the versioning function would be dangerous because of how pants works.
h
Doing file writes is fine if you use the Rules API, which we’re going to end up needing for the functionality you want. It happens in a controlled manner that is safe and works properly with caching Are you wanting to commit to using
git subtree
, or that was more in response to what I had said, and you’d be fine using VERSION files that get checked in and are automatically managed?
f
How can I setup the rules API inside of my scm function? If not possible what are the alternatives to add this versioning function to the process of building a python library (setup-py goal)?
I would prefer git as a single point of truth, because with files writes you have to manage the lifecycle of the the whole process (read, write, commit, concurent access, etc), while with the "git approach" you only need to "read" the version from git (git tag + git commit deltas + git commit Sha)
Regarding the pants versions, I do not really know what I use. This is my pants.toml
Copy code
GLOBAL]
pants_version = "1.30.0"
v1 =  false  # Turn off the v1 execution engine.
v2 = true  # Enable the v2 execution engine.
pantsd = true  # Enable the Pants daemon for better performance.
On one hand it says I use the latest stable version 1.30, but I also use pants engine v2. Can you shed some light on my confusion to your initial question: "what pants version are you using?"
h
Yes, pardon the confusion. See https://www.pantsbuild.org/v2.0/docs/pants-1-vs-2 for an explanation of the difference, and its linked page “How to upgrade” tl;dr, you’re on Pants version 1.30, which supports both the legacy “v1 engine” and the new “v2 engine”. Pants 2.0.0a1 removes the legacy v1 engine
How can I setup the rules API inside of my scm function?
This is what I’m trying to figure out today. Benjy’s idea from last week was to add an entry point where plugin authors can customize how the
setup_py
object is parsed, with access to the Rules API. I’m looking into adding that entry point for you
f
That sounds great. Thx
h
In the meantime, how is the PyInstaller thing going? Anything we can help with?
f
I am digging into pantsbuild GitHub code to understand how everything works. Also, internally I am looking to make our pyinstaler solution generic. You could help me with a couple of things: 1. What version should I look at? 1.30x branch or master? From what I've heard from you it sounds that there are major functionality changes on pants v2. Do I need to build the solution on v2? Can it be used in 1.30? 2. Is there a dev doc to help me get familiarized with the code? 3. Can you highlight some classes or a similar approach of implementing such a use case? This would help me better understand how everything fits together and where the pyinstaler code must reside
h
1.
master
branch. We made lots of changes to the Rules API between 1.30 and 2.0 for it to be more stable 2 and 3. Yep 🙂 See https://www.pantsbuild.org/v2.0/docs/plugins-overview, and in particular https://www.pantsbuild.org/v2.0/docs/plugins-binary-goal, which includes a link to an example repo Generally, reading those docs will likely go further than looking at the source code without guidance. On the
Create binaries
guide, step #1 of creating a binary target type is a good one here. You’ll want to create a new
pyinstaller_binary
target type.
Let us know if you have any feedback on those docs - they’re fairly newly written so are not yet battle tested
f
Thx. Will look at it tomorrow