https://pantsbuild.org/ logo
Join Slack
Powered by
Does pants have any ioloop to use? I want to call ...
# development
f
Does pants have any ioloop to use? I want to call 
dependencies
function from an external library. Is that possible? I know Get and MultiGet use a custom ioloop written in rust.
h
Yeah you would use the Rules API to hook into Pants's asyncio loop What does the external library's function do? Specifically, does it have side effects like making network requests or reading from the file system? If so, you'll need to use 
@_uncacheable_rule
instead of 
@rule
f
One sec, let me take a look at: 
@_uncacheable_rule
I'm trying to find list of dependencies for a given target by using the pants v2 engine. At the same time, I don't want to spawn a new process every time I want to get list of dependencies for each target. It will be too slow for my use case. The new pants doesn't require to include the dependacies in the BUILD file unless Currently, I have a library that I wrote 2 years ago that does BUILD file parsing. I can run it like the following:
Copy code
build_parser.parse_file('<BUILD FILE PATH>')
[
            {
                'type': 'python_library',
                'name': 'rorm',
                'dependencies': ['src/python/pants/libs/goldmine:utils'],
                'sources': 'rorm.py'
            },
            {
                'type': 'python_binary',
                'name': 'utils',
                'dependencies': ['3rdparty/python:boto3'],
                'sources': ['utils.py'],
                'zip_safe': True,
                'version': 1
            },
            {
                'type': 'python_tests',
                'name': 'parser',
                'dependencies': ['src/python/pants/libs/build_parser:parser'],
                'sources': ['parser_test.py']
            },
            {
                'type': 'resources',
                'name': 'scripts',
                'sources': ['mco-dynamodblocal']
            }
        ]
Is it possible to read the dependencies for a given target by calling
Copy code
@goal_rule
async def dependencies(
        console: Console, addresses: Addresses,
        dependencies_subsystem: DependenciesSubsystem
)
from external library?
The method 
parse_file
uses Python ast to read the BUILD files. I wrote it 2 years ago when I wanted to read list of dependencies from pants, and calling pants was too slow for my use case.
h
You can't directly call a 
@goal_rule
from a plugin, but you can call the code that determines dependencies: https://www.pantsbuild.org/v2.5/docs/rules-api-and-target-api#the-dependencies-field What's the end goal here? Are you thinking of adding a new goal like 
./pants custom-workflow
? -- Taking a step back, have you tried recently to do this via 
./pants dependencies
with pantsd? Pantsd wasn't as much of a thing 2 years ago, and it didn't work as well with the v1 engine. It's meant to make "warm" runs very fast
f
I have multiple end goals. First, we have custom deployment scripts that deploy apps based on changes for the application binaries. The way I do it, I build the graph by hand, and trace changed files to each of the applications. If they can be reached, I mark it for deployment. We use labels to remove some of target nodes from the graph to not include them in every deployment. Second, we have custom concourse-ci cluster that runs our unittests shared. Pants v1 shard option had a bug and we had to implement our own sharding mechanism. We can run tests concurrently, but that will not work for on our infrastructure. Some of the "unittests" are integration tests and require to run in isolation. I know there is an option for 
--changed-since
, but I will need to test it with sharding.
Can I call 
pantsd
from Python?
h
Sorry for the late reply - when @hundreds-father-404 says "`./pants dependencies` with pantsd" he means calling 
./pants dependencies
as usual but with pantsd enabled. The idea is that this would now be fast enough to use for your purposes.
If that doesn't work we can guide you through writing a custom rule to get you what you need.
Open in Slack
PreviousNext
Powered by