https://pantsbuild.org/ logo
#general
Title
# general
w

wide-midnight-78598

04/28/2023, 1:40 AM
Has anyone come across any good strategies for monorepo versioning when you have related projects (API server, web frontend) and semi-related projects (a background service that runs semi-independently) and general tooling (e.g. CLIs to control admin). I don’t particularly like any of the solutions I’ve used in the past. Generally comes down to each project versioning itself using SemVer or whatever, and then sometimes tagged gift releases representing an important snapshot of the system
This isn’t inherently bad, but it just feels a bit janky - which might be related to handling Changelogs/changesets or, I don’t really know
e

enough-analyst-54434

04/28/2023, 1:43 AM
Versioning for whose purpose? For consumers outside the monorepo?
When I've done prod work in a monorepo - 2 different companies - there was no outside consumer and no versioning.
w

wide-midnight-78598

04/28/2023, 1:45 AM
Both, kinda. A couple packages that are public (not OSS), several internal CLI tools, and then a consumer facing web application (with Python application servers)
e

enough-analyst-54434

04/28/2023, 1:47 AM
For OSS APIs clearly you have to semver the python_distribution or equivalent, but for internal service to service I've always used protocol versioning via protobuf / thrift. I.E. You must use optional fields to evolve and have code to handle it, etc.
So, no version, just careful evolution.
w

wide-midnight-78598

04/28/2023, 1:50 AM
That is kinda where I'm landing - there's also a JSON API/SDK with the same kinda idea. Protobufs internally, JSON external. A lot of that has led to "main is always deployable" - but then when interop'ing with clients who have a more "traditional" versioning, but also don't want to use git hashes 🤯 That's why I was curious what others in the community were working on as well, so I could aggregate some ideas for my own projects and those where I'm adhering to a set of external conventions.
e.g. in my opinion, a web front-end should never have a version, that's lunacy
e

enough-analyst-54434

04/28/2023, 1:50 AM
Ah, gothca. Yeah, I mean, lots of folks will think this sounds crazy.
I mean they'll think what you just said is crazy.
I expect this is just as bad as monorepo, draws polarizing reaction for and against in my experience.
w

wide-midnight-78598

04/28/2023, 1:52 AM
Good point, might be a bit more reactionary than logical planning. To keep people happy, I've been adding versioning to a lot of the items, even though they're meaningless - but it requires a lot of manual work and changelogs
e

enough-analyst-54434

04/28/2023, 1:52 AM
I think people get caught up in short-cut thinking on this. They all agree the only criteria is don't break. If you have a way to do that, its good.
But they get hung up on versions, and whatever they associate with stability.
w

wide-midnight-78598

04/28/2023, 1:53 AM
Yep, multiple environments, each running certain versions, and so on. Not bad, not good - just their way. My current plan looks a bit like what Svelte does
e

enough-analyst-54434

04/28/2023, 1:54 AM
If you're in a monorepo context, you supply a client for your service since its trivial to consume. The client can eventually speak a version protocol with the servers iff it needs to.
So you keep the thrash in-house in your little service. Just between the client and your servers. The user of the client need not care.
w

wide-midnight-78598

04/28/2023, 1:57 AM
Yeah, that's a good point. In fact, I think I might be shooting myself in the foot with those public packages. I think the client logic is "well, those have versions and changelogs, so why not everything else" It might be better to just split those off, and treat the rest of the project as a flowing, "main is stable" unit
f

fresh-continent-76371

04/28/2023, 3:09 AM
i use a combination simply of version the external things with conventional-commits - then the versioning problem "goes" away. I am using
convco
as the binary that reads the commit logs, this worked well on previous monorepo - no version maintenance. but it does give a "hook" for rigour if needed via Semver compatibility between things.
c

cold-vr-15232

04/28/2023, 7:01 AM
We just started using a variant of CalVer, it's easier to read than git hashes and can easily be located in git without tags. "year-week-commits this week". We will see how that works in the long run.
c

curved-television-6568

04/28/2023, 12:06 PM
from a previous employment we used versioning a lot, for everything. Not a monorepo setup as such, but how the code is structured wouldn’t have changed this in any way. Nothing was public all in-house services and tools (front ends, libraries, backend services and APIs, yes micro services style setup) and the versioning was primarily (with the exception of libraries that were being consumed by other projects) for our human benefit to keep track of what was deployed where. As the code evolves we needed to keep track of which version of the code that was running where (many different test and a pre-prod environment in addition to production) and using git commits or so would’ve just been headache. So, semver it was across the board. We relied on commit comments to bump a version a “major”, “minor” or “patch” step, something like:
Fix some foo bar issue #ref ABC-123 version: patch
then when that got built, the packages previous version for the branch in question was fetched and bumped according to the changes since then (may be multiple commits, take the biggest bump from that set) and that’s the version used. after being built there’s a release deployment service that has rules with what gets deployed where when there’s a new version, or we’d bump a deployment manually. In case of roll-back that’s just as easy. @powerful-umbrella-75231 would love to get a screen shot of the deployment history graph here, if you would be able to snap one that’s not showing anything sensitive on it (maybe with some rollbacks too) 😁 ((because it just shows off how nice this can be))
w

wide-midnight-78598

04/28/2023, 12:18 PM
Thanks @curved-television-6568 - that sounds kinda like an intense version of what changeset/CLI does, except built into git commits. That’s an interesting approach which could at least save some manual work. @fresh-continent-76371 Could you explain what you mean by versioning the external things? Like, only versioning public-facing items? Or only versioning externally consumable libraries? @cold-vr-15232 I’d be interested to see how that plays out. I’ve been reading lots of blogs on something similar to that method. I’m curious how it plays out in the long run
c

curved-television-6568

04/28/2023, 12:19 PM
yea, this was the work of the CI/CD team for a couple hundred heads dev org.
w

wide-midnight-78598

04/28/2023, 12:23 PM
I think what I’ve been looking for is a “solution” (not a tool necessarily, just a strategy) to do something similar - but I feel like snapshots in GitHub might play a role. So, I could envision an automagic tool (that probably exists already) which goes through git commits, or maybe is manually kicked off periodically, figures out what needs to be version bumped on each of the packages (if those packages use versions at all), and then auto-generates/updates each package’s changelog (ala Node’s Changesets) Doing that cross language is where things get mildly more fun, or just using multiple tools to do it.
Most of my deployment workflow right now is a
pants test package experimental-deploy
Test workflow is `pants lint check test``
Versioning workflow… Not really existent just yet
c

curved-television-6568

04/28/2023, 12:26 PM
I’m not aware of something that makes the magic happen (I hope you share if you find it! 😉 ) — for changesets I kind of like what backstage.io has done https://github.com/backstage/backstage/blob/master/CONTRIBUTING.md#creating-changesets for their project (although, they have faaar too many deliverables in their changelogs, but still..)
w

wide-midnight-78598

04/28/2023, 12:29 PM
Hmm, yeah, that’s pretty cool - similar—ish to SvelteKit I think. Whatever I find, my hope is to stick it in an adhoc or shell command, so it just always runs.
👍 1
e

enough-analyst-54434

04/28/2023, 12:55 PM
Pants V1 dogfooded itself for publishing it's JVM tools: https://repo1.maven.org/maven2/org/pantsbuild/jar-tool/0.0.8/
The CHANGELOG.txt was an automatic part of the publish goal (for JVM only).
Not exactly sure if that's what you mean.
w

wide-midnight-78598

04/28/2023, 1:00 PM
Yep, something like that - I'll look into that process
g

gentle-painting-24549

04/28/2023, 3:47 PM
I’m using semantic-release to handle this and really enjoy it. It uses your commit history to determine version increments and uses your repo’s tags to define state. It handles version incrementing, git tag creation, changelog generation, GitHub releases/pre-releases, and more using a ton of plugins.
w

wide-midnight-78598

04/28/2023, 8:57 PM
That’s a neat one I’d never seen before. Sounds like it has plugins to other languages. I’m curious to what degree we can customize the commit messages that it works off of, because I’ve never written a message as per their examples.
f

fresh-continent-76371

05/01/2023, 12:28 AM
@gentle-painting-24549 - yes -
semantic-release
is good; two alternatives that d not require the node/npm kitchen sink are the following • https://github.com/bvieira/sv4git - written in go • https://github.com/convco/convco - written in rust these are small binaries that do the same thing as
semantic-release
(conventional commit derived CHANGELOG and Version bumping and git tagging)
Could you explain what you mean by versioning the external things? Like, only versioning public-facing items? Or only versioning externally consumable libraries?
"External" is • an API that has external customers, client SDKs • a product that is built and published it all comes down to - what "version" do they know they (the external customer / user) is interacting with, using and if they create a "ticket" what version to they report against.
2 Views