Hey I want to say thanks to you guys for helping m...
# plugins
f
Hey I want to say thanks to you guys for helping me so much with this plugin writing process. The rules monad API is finally starting to click a bit in my brain. šŸŽ‰ I will reiterate one thing that I think could really help make this process easier: API docs from the code. I find myself going to the code and looking at the docstrings a lot, but i think an autopublishing readthedocs-style API guide would be a lot easier to navigate. There's really just too much detail to cover in the nice prose docs you have at pantsbuild.org right now, especially when you get into the finer points of what attributes a particular type has, etc, Anyways, thanks again for all the work and being a helpful and welcoming developer community!
šŸ™ 1
āœšŸ» 2
šŸ’Æ 3
h
You’re not at all wrong to call it The Monad API. Rules are indeed monadic - I think we avoided saying that because it might scare away people even more than we already do. Perhaps a small tooltip ā€œFYI these are monadsā€ would be helpful, though, for people familiar with the concept.
Strong +1 with auto-generated docs. Thanks for that recommendation I’m also hearing from Raul’s excellent feedback that we need more guided tutorials with the intent of showing how it all works. We have some very basic hello world stuff in the concepts, hyper-specific guides for specific plugin hooks, and then reference style prose. We are missing the in-between, that demonstrates the concepts with some guided examples
šŸ‘ 1
f
I'd leave the "monad" thing out of the docs though. That word has way too much baggage.
h
Okay, good feedback. Thanks
f
but yeah, i'm serious, this is really cool what you've guys have done, i'm glad it's starting to click for me, because i can see how i'll start developing plugins for internal use
šŸ˜„ 1
as far as the docker one...i'll try and open source it soonish, it's pretty simple actually
šŸ’Æ 1
h
The engine truly was a game changer for us with improving core Pants. Suddenly, adding a user request like
I want to include the result of
./pants package
in my test environment.
became extremely tractable. We couldn’t have had dreamed of that with the v1 engine
f
i had a colleague that wrote another docker plugin for pants 1.23 or something like that...it's much more complicated than what i wrote, and this seems a lot more powerful
there's just that initial hump of understanding the rules engine
āž• 1
h
.it’sĀ muchĀ more complicated than what i wrote, and this seems a lot more powerful
And any caching + concurrency you wanted was entirely manual and up to you to verify that it was correct
there’s just that initial hump of understanding the rules engine
It’ll hopefully only get better from here. You and a couple others are on the bleeding edge of writing plugins, meaning lots of opportunity to improve things
f
this is the promise that actually drew me to the project: i want to be able to build all kinds of different things from one set of sources
h
What do you mean by that?
f
well some of it is just what you get from having a digraph build tool...you can slice out pieces of a large repo and build different things
šŸ‘ 1
but if you run with that idea, and if you structure your dependencies right, you can defer archictectural and deployment decisions to the last minute
i could choose lambda function, flask-based microservice with gunicorn, falcon-based microservice with nginx container, CLI tool
they could all use the same underlying code, with just a thin "binary target" wrapper on top to control how they get deployed
h
That’s a really good point. And a couple users have asked about adding things like PyInstaller as an alternative to Pex. We’d love to see that happen, and it’s fairly trivial to add. We’re working on a couple blog posts for the upcoming 2.0.0 release, and part of our objective is to explain ā€œwhy would you use a tool like Pants?ā€ to people who have never tried a build tool. I think that could be a good reason to mention
f
i think a lot of organizations are in the exact place i'm at right now: you have a bunch of microservices that you may have used a cookiecutter template to make, and they mostly look alike, but now you want to start moving things to different deployment architectures like serverless options
using a more sophisticated build tool and monorepo can help you start finding commonalities and sharing migration code
šŸ‘ 1
and like i said, you can start to defer those lambda vs container kind of decisions until you actually measure what makes sense for your load and use case
h
Are your microservices in a monorepo or spread across different repos?
f
i can see one of the next plugins i write being a bridge to something like AWS Chalice, so i can generate infra code from all this as well
šŸ’Æ 1
spread across repos
šŸ‘ 1
like 30 of them...and it's mostly ML inference too šŸ˜„
trying to migrate from marathon to lambda, sagemaker, and ecs fargate
h
Cool, that’s something I personally want to de-emphasize, that Pants is specifically for monorepos. IMHO, I wouldn’t have recommended Pants 1.x to most smaller polyrepos before, but with dep inference cutting out 90% of the boilerplate + general v2 improvements, I now think Pants can be worth it even in smaller codebases.
f
maybe, but idk if the value proposition is as strong yet
for a small pyhton project, i'd probably recommend poetry still, but i think you guys are moving in the right direction
and you're defintely better than
setuptools
šŸ˜‚
šŸ˜„ 1
h
Poetry has an excellent UX and excellent error messages - we definitely would like to get closer to that experience
😌 1
(Speaking of which, please do point out any error messages that could be improved. It’s something I’m personally really passionate about, but it can be hard to keep track of I think what we need to do is make it more systemic how we write error messages. We’ve discussed things like a PR checklist, or maintaining a Shellcheck-style wiki for each error you may encounter, with a corresponding error code.)
f
rule graph errors lol (but i think you already know that)
h
Oh, yeah, the worst hah. We ourselves get confused by them and only know how to handle them from muscle memory the past few months. I think Stu has a path forward to improving them, and wants to work on it when back from paternity leave
šŸ‘ 1