Trying to look ahead into maybe getting GCP Cloud Function support in pants itself. Would the preference be to re-work the awslambda backend to be more generic, and perhaps becoming a

lambdex

backend? Or would you prefer that a new backend

gcpcloudfunction

be created and duplicate code would exist, but it would be easier to allow for the different types to diverge? I think the second would also prevent a breaking change since the backend name would change?

I think it makes sense to have separate backends because we want to think in terms of user functionality. It's an implementation detail that lambdex is how we implement both With DRY, the beauty of the rules API is that it becomes really easy to factor out common helper rules here :) backends can share similar code and do so all the time

Also, it's probably not common for users to be using both GCP and AWS, so separate backends for each probably makes sense

hm… how different would the APIs actually be? if they’re identical, and can just be targeted at either provider, then a single name might make sense… but if you would need options on the target types to choose the destination, then yea: different backends probably makes sense.

I thought about this for a moment and the only thing that came to mind is that GCP requires a module of "main.py" while it sounds like AWS is configurable. That's something that could be captured in documentation? Otherwise the handler being pretty generic would leave all of the function signature conformance up to the user. lambdex pretty much is a nice way to vendor dependencies given an entrypoint, right? I don't think there's anything too specific to the different cloud providers.

I feel like having a single backend support both aws and gcp is a little risky - it works right now because the implementation is shared (and that can continue to be true as long as it makes sense), but if we had to special-case each one for some technical reason we can't foresee at the moment, it might make more sense for them to be separate registration backends, even if they are very trivial ones that just import some shared utilities.

I agree. I also think documentation wise, it makes sense to have dedicated pages

Draft PR: https://github.com/pantsbuild/pants/pull/12735

<copying from DM, I realized I should have sent here>

Amazing! It looks like this isn't sharing any code with the awslambda code, which it should and can (e.g. so you don't have two lambdex subsystems)

To factor that out, you would find a common home for this. Maybe backend/python/lambdex? Then, update those two backends to import code from there

A good starting point would be moving Lambdex to maybe backend/python/lambdex/subsystem.py, and its lockfile to lambdex/lockfile.txt

Ok, I didn't know how coupled/uncoupled we wanted it to be. Is there anything magical about the "backend" package? If a user activates the "gcpcloudfunction" backend, will they also get the lambdex stuff?

Definitely we do want as much DRY as we can for internal code and for any public APIs that are shared (i.e. the lambdex subsystem). That's one of the cool parts of the engine is how easy it is to refactor code like this

Activating a backend package will activate everything in its register.py. In gcpcloudfunction's case, it would activate the new target type, the package rule, and Lambdex for you. If a user also activates awslambda, it will activate Lambdex again but that's a no-op because it only needs to be registered twice once

There's also an integration test that makes sure activating only gcpcloudfunction will work and that it doesn't expect the user to have activated other backends