...is this something that could be solved by a locally-running buildfarm instance/container? That would be potentially interesting although I'm not certain how to do so easily.

Very interested in this as well. I went down a similar path, enamored with the idea of a pex-based Docker image, but most of my team is on MacOS (as am I) and I could not make any meaningful progress on finding a valid platform string. IIRC both PyYAML and MarkupSafe were blockers, and I figure many more blockers are awaiting if I get past those

I'm sure they are. I'm currently intrigued with the idea of running a local buildfarm (or remote) which may solve multiple problems at once, but unfortunately that's not working out terribly well yet.

I could not make any meaningful progress on finding a valid platform string. IIRC both PyYAML and MarkupSafe were blockers, and I figure many more blockers are awaiting if I get past those

Do you remember the issues? Was it that those requirements don't have wheels available for Linux? Fwit, Pants uses

--platforms

and PyYAML https://github.com/pantsbuild/pants/blob/740c7c77cf8bb075e56645caf7555fbe99a0ed5a/build-support/bin/_release_helper.py#L752-L754

John Sirois gave me some great help (using complete-platforms) and I did get past PyYAML (and MarkupSafe), but then got stuck on source distributions. Not quite wanting to build the infrastructure to build wheels for all source distributions we use, I'm doubling down on trying to learn REAPI tools and set up local (and remote) buildfarms.

Sg. @lively-exabyte-12840 do you remember the issues you had?

fwiw, cross-building via docker is what is proposed in https://github.com/pantsbuild/pants/issues/13682

@hundreds-father-404 I've got a branch where I left off on trying to use platforms, I'll revisit that soon and put together either an issue or respond back here

@lively-exabyte-12840 wheel building is needed if you will ever have deps that include pure sdist deps, full stop. Iff you don't think is the case or will be the case, you probably want to use --complete-platform instead of --platform to remove the guess work: https://pex.readthedocs.io/en/v2.1.70/buildingpex.html#complete-platform

Was just looking at that - is there a good example to follow for that? Have been having some difficulty determining the appropriate values for platform/complete-platform

You don;t determine the values for --complete-platform, which is part of the point! It's buried in those docs at https://pex.readthedocs.io/en/v2.1.70/buildingpex.html#complete-platform but you must run

pex3 interpreter inspect --markers --tags > platform.data

on the machine you're targeting to generate a complete platform data file. Then, back on your other machine

--complete-platform platform.data

Thanks, I'll give that a spin

Thanks. Like I said though, if the resolve happens to hit any distribution only available as an sdist, things will still blow up and there is nothing to do about that except pre-build a wheel for that distribution and point to it with

--find-links

or else go down your Docker route.

Good news is that with a minimal requirements.txt file only containing Flask==1.1.4, a simple test worked. Trying all dependencies next.

Yeah, assuming you're not targeting macosx.

If you are trying to use it on any platforms other than cp37-cp37m-macosx_10_14_x86_64, such as linux, then yes. (re the 10_14 part, macOS is future-compatible so newer macOS versions should at least work)

But re the cp37 part - no abi3 so dead in the water for all other interpreters.

The build farm idea has legs, but it's certainly been painful (I will say the experience has reminded me how friendly and helpful the pants community is). I got a build working with buildbarn and buildgrid (buildfarm is being fussy), and locally you can even do ssh Auth passthrough for private repos, although it's failing in one case and the experience is more frustrating than imagined; with multiple targets a single failure seems to result in immediate failure of all running jobs, and things seem to take a lot longer, and I'm worried in some cases the generated blobs will be so large that cloud based buildfarms may not be as useful as I'd like. So not a panacea, but still worth pursuing.