pantsbuild #general

Channels

welcome

github-notifications

announce

full-keyboard-92367

09/21/2021, 5:59 PM

Hey pants, I'm creating a monrepo for multiple fastapi servers. I've figured most things out, but right now I'm stuck at creating a development server. I intend to use the pex binaries for deployment using docker, but during development we can't constantly rebuild the containers. I use uvicorn for serving, there are 2 ways to start a uvicorn server, from the shell

uvicorn module:app --reload

and programmatically

uvicorn.run(app)

Unfortunately uvicorn discontinued usage of reload while programmatically running the server, so for reload feature to work(crucial in a development environment), the command has to be executed via shell. I can't for the life of me figure out how to run a shell command or create a shell entrypoint with pants. Also it would be great if you guys could provide point me to an actual real world project with pants for a reference to make sure I'm doing this correctly.

👋 2

modern-wolf-36228

09/21/2021, 9:05 PM

you can run programmatically

modern-wolf-36228

09/21/2021, 9:06 PM

uvicorn.run("<module>:<app variable>", reload=True)

modern-wolf-36228

09/21/2021, 9:08 PM

If I understood correctly what you said 😅

modern-wolf-36228

09/21/2021, 9:12 PM

The

script

field is going to be introduced in the

pex_binary

in the next release I guess, which you'll be able to use

pex_binary(script="uvicorn")

➕ 1

polite-garden-50641

09/21/2021, 10:17 PM

any idea why pants is erroring out:

salmon-planet-80683

09/21/2021, 10:17 PM

Hi folks! Have a Pants permissions issue. I have a <thing that runs jobs> - Hashicorp Nomad - that invokes

./pants

as root to execute a test. When I then try and do a

./pants

as <my own user> i, understandably, get something along the lines of

Copy code

pants.pantsd.process_manager.ProcessManager.MetadataError: failed to purge metadata directory /ROOT_TO_MY_PANTS?PROJECT/.pids/26629ce97d56/pantsd: PermissionError(13, 'Permission denied')

Would there be a way for me to execute the

purge_metadata_by_name

stuff as root so that it's not screwed up for my normal user?

hundreds-father-404

09/22/2021, 6:45 AM

I rewrote our docs on 3rd-party dependencies. The original didn't explain Pants's conceptual model very well, whereas now it all builds off of this main idea:

Pants handles dependencies with more precision than Python projects solely using workflows like requirements.txt or Poetry. Whereas normally you have a single virtual environment, Pants understands precisely which third-party dependencies each file in your project uses

https://www.pantsbuild.org/v2.7/docs/third-party-dependencies-rewrite. Feedback welcomed!

❤️ 4

curved-television-6568

09/22/2021, 8:04 AM

I’m trying to build a pex, as part of a test, but fail due to no python in the env… [SOLVED] Any ideas what I’m missing? (will try to add relevant information in the thread)

curved-television-6568

09/22/2021, 8:04 AM

The error being:

Copy code

E           Engine traceback:
E             in select
E             in pants.backend.docker.docker_build_context.create_docker_build_context
E             in pants.backend.python.goals.package_pex_binary.package_pex_binary (src/python/proj/cli:bin)
E             in pants.backend.python.util_rules.pex.create_pex
E             in pants.backend.python.util_rules.pex.build_pex (src.python.proj.cli/bin.pex)
E             in pants.engine.process.fallible_to_exec_result_or_raise
E           Traceback (most recent call last):
E             File "/private/var/folders/8j/c8jf_msj009947wyw82xvdkw0000gn/T/process-executionVc609f/src/python/pants/engine/process.py", line 258, in fallible_to_exec_result_or_raise
E               raise ProcessExecutionFailure(
E           pants.engine.process.ProcessExecutionFailure: Process 'Building src.python.proj.cli/bin.pex' failed with exit code 127.
E           stdout:
E           
E           stderr:
E           env: python: No such file or directory

curved-television-6568

09/22/2021, 8:04 AM

My test is:

Copy code

def test_packaged_pex_path(rule_runner: RuleRunner) -> None:
    rule_runner.write_files(
        {
            "src/docker/BUILD": """docker_image(dependencies=["src/python/proj/cli:bin"])""",
            "src/docker/Dockerfile": """FROM python""",
            "src/python/proj/cli/BUILD": """pex_binary(name="bin", entry_point="main.py")""",
            "src/python/proj/cli/main.py": """print("cli main")""",
        }
    )

    assert_build_context(
        rule_runner,
        Address("src/docker", target_name="docker"),
        expected_files=["src/docker/Dockerfile", "src.python.proj.cli/bin.pex"],
    )

curved-television-6568

09/22/2021, 8:06 AM

The

assert_build_context

requests this product from the rule runner:

Copy code

context = rule_runner.request(
        DockerBuildContext,
        [
            DockerBuildContextRequest(
                address=address,
                build_upstream_images=False,
            )
        ],
    )

And the rule runner has these rules:

Copy code

return RuleRunner(
        rules=[
            *context_rules(),
            *core_target_types_rules(),
            *package_pex_binary.rules(),
            *pex_from_targets.rules(),
            *target_types_rules.rules(),
            QueryRule(BuiltPackage, [PexBinaryFieldSet]),
            QueryRule(DockerBuildContext, (DockerBuildContextRequest,)),
        ],
        target_types=[DockerImage, Files, PexBinary],
    )

where most of those rules are taken from the

package_pex_binary_integration_test

curved-television-6568

09/22/2021, 8:08 AM

Oh, the answer should be staring me in the eye, of course the tests in that integration test file does what I need already…

curved-television-6568

09/22/2021, 8:09 AM

Yep, sorry, I tend to not see the answer, until I explain my issues to someone else.

🧠 1

curved-television-6568

09/22/2021, 8:09 AM

The missing piece, if any one stumbles upon this as well, was this line:

Copy code

rule_runner.set_options([], env_inherit={"PATH", "PYENV_ROOT", "HOME"})

👍 2

modern-wolf-36228

09/22/2021, 4:55 PM

script

field is already documented on

2.8.0.dev0

is already documented... but can I use it? 🤔

loud-stone-83419

09/22/2021, 11:57 PM

Hey! I have a dual pants v1 / pants v2 setup. I thought it was all good but as the v2 is growing I just spotted that pants v1 is picking up BUILD.v2. Can I configure v1 not not pick up on that? or should I change the pattern I use for v2 to be BUILDv2?

worried-salesclerk-37834

09/23/2021, 6:46 PM

Wanting to use pre-built wheels for a dependency that hasn't created one. I thought there was a way to place that in build-support and check it in, but the only docs I'm seeing are to put it in your own pypi repository. Is there a way to build with a local wheel?

modern-wolf-36228

09/23/2021, 7:09 PM

Do you guys have the reference for the Cython support mentioned here?

wooden-stone-7624

09/24/2021, 5:59 AM

Hi Team, I'm trying to build the wheels on arm64 machine but I'm facing the following error:

pants.pantsd.process_manager.ProcessManager.Timeout: exceeded timeout of 60 seconds while waiting for pantsd to start

Could you please help me with this?

👋 1

curved-television-6568

09/24/2021, 7:37 AM

Hi Community! I’m working on a draft for documentation for a new

experimental_shell_command

target for v2.8. Your feedback would be most appreciated. See this google doc: https://docs.google.com/document/d/1610tAFFCwlSR1WiclQCajwrks7fWaNEwQpfWEpkOUDo/edit?usp=sharing

❤️ 3

ambitious-student-81104

09/24/2021, 3:48 PM

Hey guys! Wondering if you have an "elevator pitch" for why use pants for python? @happy-kitchen-89482 @hundreds-father-404

🙌 1

brash-baker-91190

09/24/2021, 3:53 PM

"Do you hate Python packaging and paths? Use Pants" 😅

➕ 1

👏 1

🤣 1

busy-vase-39202

09/24/2021, 8:17 PM

Recruiting people with Pants skills or interest in learning Pants? You're welcome to post listings here. It's a great place to find skilled build engineers, data scientists, and others.

worried-salesclerk-37834

09/24/2021, 8:27 PM

Not sure of the best way to do this... I use airflow via Cloud Composer. In order to deploy the Directed Acyclic Graphs (DAGs), you zip up all of the python files containing those definitions. So there's no entrypoint or anything, just a zip of files. What I would like to do is also depend on our common library, so that some of the dags can use code from there. It doesn't look like I can just make an archive with "dependencies" on the dags' python library and have it transitively pull in my common library's code.

python_distribution

looks like it might be a way forward, but it looks like I'd have to make a separate distribution for the common library? It's relatively large, so I was hoping not to pull in the entire codebase. Is there a way I can make a zip of a current

python_library

that also pulls in all of its transitive dependency code?

ambitious-student-81104

09/24/2021, 9:20 PM

Hi folks! Quick question. I've noticed that sometimes I need to add

sources=["*.py"]

in my

python_library

target in order for my

pex_binary

in the same BUILD to run successfully, or else it would

ImportError: No module named path.to.entry_point

; but in some other cases I don't need that

source=["*.py"]

for things to work. What's the magic here?

worried-salesclerk-37834

09/27/2021, 4:02 PM

Another odd question. Is it possible to have pants build a pex without third party dependencies? Just infer first-party? I tried to add "!//:airflow" to both the top-level python library and also to pex_binary, but it still tried to build with the dependency.

user

09/27/2021, 4:09 PM

https://twitter.com/pantsbuild/status/1442521828564426756 Pants 2.7.0 is now available! Python tool lockfiles, Yapf, Docker, and

./pants peek

. Check out the blog post and notes for more information: https://t.co/aF5iedD70V https://t.co/bgt9By8KxS Twitter

🎉 1

brash-baker-91190

09/27/2021, 4:28 PM

I've got an odd failure to build a wheel with native compilation in our CI/CD environment; see thread for more

brave-furniture-86963

09/27/2021, 7:11 PM

Hi all! What determines how many

requirements.pex

and

pytest_runner.pex

get built during

./pants test ::

? I have defined 9

python_tests

targets in all in our repository, yet I’m seeing over 26

pytest_runner.pex

and over 31

requirements.pex

built (also these numbers change depending on how I structure my requirements.txt files).