https://pantsbuild.org/ logo
Join Slack
Powered by
OK. I'm going to rephrase a question I asked earli...
# general
p
OK. I'm going to rephrase a question I asked earlier. I want pants to be the primary entry point for development, replacing the Makefile & supporting scripts. One responsibility of the Makefile (and a couple of scripts) is setting up a local development environment with services like mongo, rabbitmq, and redis. How can I do setup like that from within pants? Would I need to write a pants plugin? Something that makes sure the services are running when running tests? How would service requirements (mongo, rmq, redis, etc) interact with remote execution (REAPI)? (which is something I'm interested in using eventually)
h
@happy-kitchen-89482 you've started work on documenting things like this for Django, for example, iirc. Thoughts?
h
Hmm this is always a good question and really depends on specific setup. For example, in Toolchain's internal repo we set up test databases in code: the thing that tests call to get a handle to a db will actually launch postgres if there is no db already running at that port. So that's one way, which has the advantage of being seamless and doesn't involve Pants at all. That assumes you have postgres installed on your machine of course, but that's reasonable.
For a custom plugin to work I think we'd need to do a little rewiring to make sure that tests depend on the custom setup, and that they are not treated as cacheable. Generally only special Pants rules can have side effects, and "starting a database" is one hell of a side effect.
šŸ¤£ 1
āž• 1
h
hat assumes you have postgres installed on your machine of course
Reasonable for a private org's codebase, possibly not as reasonable for an open source project. But perhaps the experience could be somewhat improved by eagerly erroring when it's not yet installed and giving instructions
h
With remote execution, you'd need those binaries to exist in the image, and there are some more complications because of the side-effecting nature of this.
And there needs to be some reasoning about whether any state needs to persist across individual test invocations
One other way to go is to have a pool of running services and each remote invocation is assigned one from the pool, with pants ensuring that no two concurrently access the same pooled service.
We already do slot management like this for local concurrency, we could extend it to remote
Basically each process runs with PANTS_CONCURRENCY_SLOT=<some small integer>
and your code can use that to pick a pool service to talk to, no matter where it runs, assuming it can access that service over the network
If the services have to be local to the machine the process is executing on, then concurrency isn't an issue but setup is
p
I'm reading and processing your comments šŸ™‚ will respond shortly...
h
No worries
p
> that assumes you have postgres installed on your machine of course
Reasonable for a private org's codebase, possibly not as reasonable for an open source project. But perhaps the experience could be somewhat improved by eagerly erroring when it's not yet installed and giving instructions
Ooh, yeah. Erroring out with instructions would be a very good experience. Then the instructions could be tailored to the active platform (CentOS, Ubuntu, Gentoo (because that's what I like to use), Mac OS X, ...)
šŸ’Æ 1
Currently, installing & running these services is very different for CI vs local development. For CI, we don't run the tests until the env is setup. So, I'm really thinking about how to improve the local dev experience without making the CI story more difficult. Adding DBs on the fly in a pytest fixture makes perfect sense, actually starting the services (super crazy side effect) is very sticky because CI vs local differs ...
So, maybe I need two plugins: ā€¢ one that injects some service availability assertions as a prerequisite to running tests (or a subset of tests? Not all of them need those services). This would provide instructions on installation and any ./pants <goal> that can be used to help with setup. ā€¢ one that adds a new goal for setting up the local dev environment - for integration tests especially, we actually need to run stackstorm itself before running the tests, this goal could start that. (Hmm - is that kosher for a goal to start "background" tasks under pantsd that continue to run while running other pants goals?)
a pool of running services and each remote invocation is assigned one from the pool
We already do slot management like this for local concurrency, we could extend it to remote
PANTS_CONCURRENCY_SLOT=<some small integer>
your code can use that to pick a pool service to talk to, no matter where it runs, assuming it can access that service over the network
So, remote execution in this way would require extra infrastructure to provide those pooled resources. The REAPI service wouldn't provide that itself.
If the services have to be local to the machine the process is executing on, then concurrency isn't an issue but setup is
This is where I imagine a pants plugin might come in? Do the pants plugins get sent over REAPI? Or would there have to be some kind of dependency on a script that would get executed to setup the REAPI environment before running the target (eg) pytest command?
That service availability assertion would also catch issues in CI where the service container doesn't actually start correctly. Cool. I think that's where I'll need to start.
e
Do the pants plugins get sent over REAPI?
The REAPI only gets sent the description of a process to run: {args, env, CWD & an optional input blob that gets materialized as the file system tree in CWD on the remote machine} and a description of the output to capture (blob paths).
So pants doesn't run on a remote machine, just some process - like ["python", "--version"]
p
using services with remote execution doesn't sound very promising. I'll probably use just the caching aspect of REAPI (once I get there) then.
e
Yeah, if the service interaction time is << service startup time there is alot of work to do on the API and implementations of it. If that is reversed though, just having the services started up as part of the REAPI image specifed's init (all implementations I know of allow an image to be specified for each execution) should work just fine.
h
I'm not sure if service availability assertions is a Pants plugin or just something your tests call (say in a pytest hook, or implicitly when they try to acquire a handle to the service in question)?
āž• 1
p
I guess I'm going for a fail-fast thing to warn as early as possible if someone is trying to test and they haven't got their env setup right. If we wait until pytest is running, then we'll end up with one failure for every file, wouldn't we? Is there any way to pair that down and get pants to bail asap?
šŸ‘ 1
(I'm also chatting about the implementation of a plugin in the #plugins channel - there were several topics in here, so I was trying to pair down the thread's topic)
h
Ah, great point Jacob. You're right that Pants would not error eagerly if the error came from Pytest. But! I do think we can add a 
--test-fail-fast
option, which could be a useful feature in general Only, I'm not sure you'd want that permanently toggled for your repo. I personally really like seeing all my test failures when iterating
p
Yes. I like running all the tests and seeing all the failures. Occassionally I'll turn on fail fast when I'm working on a particular set of tests, but I wouldn't want that on by default. If there is an error in a pants plugin, then pants exits quickly, right?
h
Yeah, the fail-fast thing is a good point
in a plugin you can get pants to fail eagerly
šŸ‘ 2
h
Occassionally I'll turn on fail fast
Using Pytest options w/o Pants, right? Trying to determine if we should add 
--test-fail-fast
to Pants. It's pretty trivial to implement (like 10 lines), only that again we're careful to add new features w/o vetting them
p
I would reach for passing the options straight to pytest, but I guess that doesn't actually help since pants is running one instance per file. I can see the utility, but maybe sit on that one for awhile longer. šŸ™‚
āœ”ļø 1
e
In this case it would be adding back a long-term feature. In Pants v1 you could generically say 
./pants --fail-fast ...
for any combination of goals.
āž• 1
2 Views
Open in Slack
PreviousNext
Powered by