alert-psychiatrist-1410201/29/2023, 8:01 AM
happy-kitchen-8948201/29/2023, 6:44 PM
happy-kitchen-8948201/30/2023, 2:55 AM
ancient-france-4290901/30/2023, 4:28 PM
the output is buffered then dumped immediately, is there a way to do that?
high-yak-8589901/30/2023, 7:56 PM
so I know my installation is good. It requires a
installation on the system which are both available under my
. Pants seems to be picking things up fine because the server is attempted to start, but fails the boot process. I think sandboxing is the most obvious thing at play here. Any ideas for environment variable differences or similar to look at? When I preserve sandboxes, the generated
executes without error.
faint-businessperson-8690301/31/2023, 12:45 AM
target deterministically (in 🧵).
adorable-engine-7173601/31/2023, 10:04 AM
script from the repo, and added
- im getting a flake - seemingly only in CI and not local, although that could just be because im testing more in CI - typical error is:
i can open a ticket but i wanted to rule out that im doing something incorrect first
✓ mypy succeeded. FATAL: exception not rethrown FATAL: exception not rethrown /home/runner/work/_temp/f3725c23-b66e-48a4-b13a-9692347a0610.sh: line 1: 1684 Aborted (core dumped) ./scripts/pants check :: Error: Process completed with exit code 134.
ancient-france-4290901/31/2023, 12:19 PM
adventurous-journalist-7292001/31/2023, 12:46 PM
thousands-plumber-3325501/31/2023, 12:52 PM
high-energy-7311901/31/2023, 12:59 PM
high-energy-7311901/31/2023, 1:00 PM
curved-microphone-3945501/31/2023, 2:46 PM
My on-prem REAPI server are working correctly and exposed correctly (Tested with Bazel example) and when I try the
remote_cache_read = true remote_cache_write = true remote_store_address = "<grpc://cache.local:8980>" remote_instance_name = "worker"
with Pants, I get errors about the Environment missing Python (which is normal at this state) but we don't want remote execution, only the caching. When running any command
I don't see any hit on my server or logs in Pants that tell its trying to do something with the remote cache, Am I missing something here ? I am following this https://www.pantsbuild.org/v2.15/docs/remote-caching
lint, fmt, test, package, publish
lemon-oxygen-7249801/31/2023, 3:21 PM
. And because GitHub default runners have pretty low performance, the Pants CI is on the average 4 to 5 times slower, because it uses a single machine. I wonder where I should be headed to speed this up. For the moment we don't cache Pants' state and use
./pants --changed-since=origin/main --changed-dependees=transitive test
, because downloading the cache was quite slow. Ways forward that I see: 1. Use persistent runners, keep Pants' state 2. Use remote executors 3. Use a remote cache Any experience to share on what would yield the biggest speedup?
brief-flag-3558601/31/2023, 3:35 PM
It’s using the right environment, but doesn’t use docker at all (details in 🧵)
~/bin/pants package clients/prefect:prefect_client 00:16:55.48 [INFO] Completed: Building clients.prefect/prefect_client@environment=local_docker.pex with 2 requirements: prefect[aws,postgres,redis,templates,viz]<0.16.0,>=0.15.6, redis-py-cluster<3.0.0,>=2 00:16:55.63 [INFO] Completed: Building clients.prefect/prefect_client@environment=local.pex with 2 requirements: prefect[aws,postgres,redis,templates,viz]<0.16.0,>=0.15.6, redis-py-cluster<3.0.0,>=2 00:16:55.69 [INFO] Wrote dist/clients.prefect/prefect_client@environment=local_docker.pex 00:16:55.69 [INFO] Wrote dist/clients.prefect/prefect_client@environment=local.pex
bitter-ability-3219001/31/2023, 3:38 PM
Started failing due to the sha/len being different. And then today, they decided to revert the change. So just FYI to those of y'all fielding off errors from Pants talking about unexpected length/SHA on GitHub tarball URLs.
http_source( filename = "pybind11", len = 569736, sha256 = "8103bf774f02ff7a063c017eb1adac51c17af9c2c08363830618ad648f09ad1f", url = "<https://github.com/pybind/pybind11/archive/refs/tags/v2.4.3.tar.gz>", ),
brief-flag-3558601/31/2023, 3:39 PM
# pants.toml [GLOBAL] pants_version = "2.15.0rc2" pantsd = false print_stacktrace = true pythonpath = ["%(buildroot)s/build-support/plugins"] colors = true build_patterns = ["PANTS", "PANTS.root"] build_file_prelude_globs = ["build-support/macros.py"] backend_packages = [ "pants.backend.python.lint.pyupgrade", "pants.backend.python.lint.autoflake", "pants.backend.python.lint.docformatter", "pants.backend.python.lint.isort", "pants.backend.python.lint.black", "pants.backend.python.lint.flake8", "pants.backend.python.lint.pylint", "pants.backend.python.typecheck.mypy", "pants.backend.python", "pants.backend.shell.lint.shellcheck", "pants.backend.shell.lint.shfmt", "pants.backend.shell", "pants.backend.awslambda.python", "pants.backend.plugin_development", # for the pants_requirements macro "pants.backend.docker", "pants.backend.docker.lint.hadolint", ] [python] interpreter_constraints = [">=3.8,<3.10"] interpreter_versions_universe = ["3.8", "3.9"] [python-bootstrap] search_path = ["<PATH>"] names = ["python3.9", "python3.8", "python3", "python"] .... [environments-preview.names] local_docker = "//:local_docker" local = "//:local"
... docker_environment( name="local_docker", image="python:3.9.13-slim-buster", fallback_environment="local", docker_executable_search_paths=["<PATH>"] ) local_environment( name="local", ) # __defaults__(all=dict(environment="docker")) ...
brief-flag-3558601/31/2023, 3:39 PM
python_sources() pex_binary( name="prefect_client", entry_point="prefect_client.py", environment=parametrize("local_docker", "local") )
dazzling-elephant-3376601/31/2023, 3:56 PM
which imports from
Running this locally, everything works completely fine ✅
import lib.business_rules # noqa def test_thing(): assert 1 == 1
But running the same test in CI gives an error ❌
$ ./pants test --force lib/platform/tests/test_advice_models.py 15:51:25.09 [INFO] Completed: Run Pytest - lib/platform/tests/test_advice_models.py - succeeded. Partition: platform ✓ lib/platform/tests/test_advice_models.py succeeded in 0.85s.
My GH action is invoking the exact test file (the same way I would run locally)
==================================== ERRORS ==================================== __________ ERROR collecting lib/platform/tests/test_advice_models.py ___________ ImportError while importing test module '/tmp/pants-sandbox-5gjBP7/lib/platform/tests/test_advice_models.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) lib/platform/tests/test_advice_models.py:1: in <module> import lib.business_rules # noqa E ModuleNotFoundError: No module named 'lib.business_rules'
Checking the output of
jobs: build_and_test_python: runs-on: ubuntu-latest strategy: matrix: python-version: [3.9] steps: - uses: actions/checkout@v3 - uses: pantsbuild/actions/init-pants@v2 - name: Test run: | ./pants test lib/platform/tests/test_advice_models.py if: always() # We want the log even on failures.
Includes things in
./pants dependencies --transitive lib/platform/tests/test_advice_models.py
so unsure why CI is unable to find this module
It’s a long shot (apologies), but any idea how I could begin to debug what’s going on here? (I’m on pants 2.15.0rc2 )
lib/business_rules/__init__.py lib/business_rules/engine.py lib/business_rules/operators.py
adorable-engine-7173601/31/2023, 6:33 PM
- does anyone have any ideas how this could be acheived?
proud-dentist-2284401/31/2023, 10:44 PM
target depends a
target (a packagable target) via the
field instead of the
field, will the
transitively depend on everything the
depends on (like the deps inferred from entry_points)?  https://www.pantsbuild.org/docs/python-test-goal#testing-your-packaging-pipeline I want to depend (well, actually infer a dep from within a plugin) on the underlying code without actually generating the wheel.
dazzling-dinner-7642001/31/2023, 10:54 PM
are used by non-developers in parts of the company that will depend on the algorithms written in
. This will also allow a more formal development process into
to be treated similar as numpy and pandas for internal development. Everything up to this point had been built exclusively in a windows environment. Setting up to use pants to handle our dependencies, testing, and distribution we have started working in WSL using ubuntu 22.04. My understanding that in addition to being able to run pants this should help us remove any incurred dependencies on windows enabling easier distribution to other operating systems or servers. The questions we have at the moment are: • What is the best way to distribute standalone python apps to internal workers on windows machines? ◦ We have been reviewing the docs but the support seems to centered towards pex (https://www.pantsbuild.org/docs/python-distributions) • Some members of the development team work in pycharm while others us vs code. Our current plan to to leave most development flows unaltered in the previous windows environments with pycharm and handle testing/distribution separately in ubuntu or using the WSL extension for vs code. Does this raise any red flags? In addition, we would very much appreciate any feedback regarding our proposed structure and plan this far! Let me know if there are any other details that would help in understanding our questions or if you have any questions for us!
faint-dress-6498902/01/2023, 12:07 AM
application over to our pants repo. Previously, I was running these test inside a docker image built with the databricks runtime as their base image. Java exists on said image as it is a dependency of spark and thus databricks. Now the following fixture no longer works as Java isn't install on my machine:
Is there a way for pants to manage the setup/installation of java for this test suite?
@pytest.fixture(scope="session") def spark_session() -> SparkSession: return ( SparkSession.builder.master("local") .appName("spark session example") .getOrCreate() )
lemon-cpu-3431202/01/2023, 1:02 AM
Then make changes to
./pants run src/python/app/run.py
or any of its transitive dependencies, I would expect pants to restart the run command. What am I missing here?
lemon-cpu-3431202/01/2023, 1:07 AM
enough-analyst-5443402/01/2023, 1:07 AM
enough-analyst-5443402/01/2023, 1:09 AM
lemon-cpu-3431202/01/2023, 1:09 AM