pantsbuild #general

Channels

welcome

github-notifications

announce

fancy-queen-20734

04/14/2020, 8:06 PM

Why is it then that I see > 1000 lines of black linter calls even when I don’t touch python? Do pull requests use a different/individual cache?

rapid-crayon-8232

04/15/2020, 3:02 PM

hey guys, I'm trying out the v2 version of pants for python, and I have a question (maybe stupid) about the dependencies resolution, so I've run

./pants test packages/::

from a cold repo, and when downloading the requirements in 3rdParty I noticed this:

fancy-queen-20734

04/15/2020, 4:48 PM

toml errors are kinda dense

broad-airplane-1877

04/15/2020, 5:13 PM

If I find problems in pantsbuild.org documentation, should I create a GitHub issue?

rapid-crayon-8232

04/15/2020, 5:34 PM

I've seen that repl now supports

IPython

(super cool btw), are there any plans to then support a repl with a jupyter notebook ?

rapid-crayon-8232

04/15/2020, 5:51 PM

I don't know if it's a known bug but when running the

binary

goal on multiple files, pants don't automatically ignore other goals, works well with the test goal btw

When you glob over files or targets, Pants knows to ignore ones that aren't relevant to the requested goal.

and throws the following error:

Copy code

ERROR: The `run` and `binary` goals only work with the following target types: ['python_binary']

You used packages-python-pants/helloworld/greet:greet with target type 'python_library'.

bulky-evening-62934

04/16/2020, 9:19 AM

Hi, I’m using

1.27.0.dev3

, it seems that pants will always build the binary for current platform even I set

platforms = ['linux-x86_64-cp-37-cp37m']

BUILD

, is it expected behavior ? And is it possible to pass platform as an option when running

pants binary

instead of using the value in BUILD file ?

bulky-evening-62934

04/16/2020, 11:58 AM

Also, for local developement, I used to create venv by dependencies, dev dependencies, and install intra-repo dependencies by edit mode, than use that venv in VSCode for definition jumping, code evaluate and formatter/linting (black, isort, etc..) , it’s possible to output a venv from pants for a specific binary/application ? or any other suggested dev workflow on VSCode with Pants ?

❤️ 2

bulky-evening-62934

04/16/2020, 2:16 PM

Another question, python_binary could only accept 1 file in sources ?

ERROR: The 'sources' field in target XXXX:XXXX must have 0 or 1 files, but it had 2 files.

could I include more than 1 file for pants binary ?

fancy-queen-20734

04/16/2020, 7:03 PM

Ran into a bunch of time outs in a integration test shard run

fancy-queen-20734

04/16/2020, 9:01 PM

Could I get another reviewing pass on the jvm-platforms strict patch? https://github.com/pantsbuild/pants/pull/9287

fancy-queen-20734

04/17/2020, 12:05 AM

Found a fairly straightforward bug https://github.com/pantsbuild/pants/pull/9567

bulky-evening-62934

04/17/2020, 5:25 PM

Hi all, v2_ui is very nice, but I met some issues with my vscode terminal (for example, when I run

./pants target-types --details=python_binary

, it will jump to the head of terminal, might be a issue with vscode terminal)… but if I disable v2_ui, there will be very limited information, for example, running

./pants binary --binary-level=trace XXXX

, it will only show

Wrote dist/app.pex

, less information is very clean, but do you also consider adding a bit more loggings like fetch third party dependencies, building pex file, etc… which might be useful when making binary takes long time …

bulky-evening-62934

04/17/2020, 5:31 PM

another question,

pants

is a bash script from https://github.com/pantsbuild/setup/blob/gh-pages/pants, it seems to be very stable (4 changes during last year) so we don’t need to regularly update it when upgrading the pants version ?

bulky-evening-62934

04/17/2020, 6:22 PM

For

python-repo

(thanks for the fixing last week), it works perfectly, I have,

Copy code

[python-repos]
indexes = [
  "<https://pypi.org/simple/>",
  "<http://localhost:9696/simple>"
]

is there any logic about the priority for those 2 indexes, let’s say we have the package of the same name (but might build with different cpu instructions) on those 2 indexes, which one will be picked ?

💯 1

bulky-evening-62934

04/17/2020, 6:32 PM

For the name of

python_binary

, the built pex file name is exactly the same string of name field in BUILD, so it’s possible that I got same file name

app.pex

from

./pants binary app1:app

and

./pants binary app2:app

(if I have defined both library and binary in that BUILD, I cannot use something like

./pants binary app1

), any suggestion for the best practice of the name in

python_binary

bulky-evening-62934

04/20/2020, 6:46 PM

Hi all, from https://github.com/pantsbuild/pants/blob/a154c84eb02092c397c15dc00915bbf7cc89fd40/src/python/pants/base/build_root.py#L21, pants detect the root of repo by checking whether cwd or the closest ancestor has one of these 3 files. So basically, I could put pants script in another place (included in $PATH), and create a empty pants file in the root of my repo ? (

./pants

is not as easy as just typing pants ; ) )

aloof-angle-91616

04/20/2020, 6:46 PM

absolutely!!

aloof-angle-91616

04/20/2020, 6:47 PM

the buildroot (where everything in pants is relative to) is determined (by definition) by walking upward from the current directory to find a file named

pants

, as you discovered

bulky-evening-62934

04/21/2020, 10:59 AM

Hi, I have,

Copy code

[python-setup]
interpreter_constraints = ["CPython>=3.7"]
interpreter_search_paths = ["<PYENV>"]

And I’m wondering what’s the rule of pants to select the interpreter if I have multiple versions in pyenv ? I’m asking this because I have both

3.7.5

and

anaconda3-2019.10

in pyenv, for pants, I hope it could be always the first one got selected, but pants sometimes use the second one (which had some issues with python.h …)

calm-artist-46894

04/21/2020, 9:12 PM

Cache (build everything but share cache cross build) vs Incremental Build (only build based on git change), which one is more favorable in case of monorepo? Related question, many of the intermediate result (e.g. pyprep, source, requirements) are not in the

.cache

folder (they can be found in

.pants.d

folder), hence even with remote cache setup, those step would still be re-evaluated. Is this by design?

adamant-mouse-92628

04/22/2020, 1:26 AM

from

pants goals

adamant-mouse-92628

04/22/2020, 1:26 AM

Copy code

Bootstrapping pants_deps with requirements:
  /Users/eax/svn/pants/3rdparty/python/requirements.txt
  /Users/eax/svn/pants/3rdparty/python/twitter/commons/requirements.txt
  /Users/eax/svn/pants/pants-plugins/3rdparty/python/requirements.txt
  /Users/eax/svn/pants/contrib/python/src/python/pants/contrib/python/checks/checker/3rdparty/requirements.txt
Using python at /usr/bin/python3
Using base prefix '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7'
New python executable in /Users/eax/svn/pants/build-support/virtualenvs/Darwin/pants_dev_deps.py37.venv/bin/python3
Also creating executable in /Users/eax/svn/pants/build-support/virtualenvs/Darwin/pants_dev_deps.py37.venv/bin/python
Traceback (most recent call last):
  File "/Users/eax/svn/pants/build-support/virtualenv.dist/virtualenv.py", line 2619, in <module>
    main()
  File "/Users/eax/svn/pants/build-support/virtualenv.dist/virtualenv.py", line 867, in main
    symlink=options.symlink,
  File "/Users/eax/svn/pants/build-support/virtualenv.dist/virtualenv.py", line 1142, in create_environment
    install_python(home_dir, lib_dir, inc_dir, bin_dir, site_packages=site_packages, clear=clear, symlink=symlink)
  File "/Users/eax/svn/pants/build-support/virtualenv.dist/virtualenv.py", line 1607, in install_python
    shutil.copy(original_python, py_executable)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/shutil.py", line 245, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/shutil.py", line 103, in copyfile
    if _samefile(src, dst):
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/shutil.py", line 88, in _samefile
    return os.path.samefile(src, dst)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/genericpath.py", line 96, in samefile
    s1 = os.stat(f1)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType`

adamant-mouse-92628

04/22/2020, 1:26 AM

am I missing something?

adamant-mouse-92628

04/22/2020, 1:27 AM

hrm...

∴PY=/opt/local/bin/python3.7 pants goals

seems to have actually worked

😶 1

adamant-mouse-92628

04/22/2020, 1:29 AM

except for

adamant-mouse-92628

04/22/2020, 1:29 AM

Copy code

Installed package `cargo-ensure-installed v0.2.1` (executable `cargo-ensure-installed`)
Traceback (most recent call last):
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 454, in select
    downloaded_file = self._binary_tool_fetcher.fetch_binary(fetch_request)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 255, in fetch_binary
    self._do_fetch(bootstrapped_binary_path, file_name, urls)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 242, in _do_fetch
    with self._select_binary_stream(file_name, urls) as binary_tool_stream:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 238, in _select_binary_stream
    raise self.BinaryNotFound(name, accumulated_errors)
__main__.BinaryNotFound: Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 569, in <module>
    print(select(sys.argv))
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 565, in select
    return binary_util.select(binary_request)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 456, in select
    raise self.BinaryResolutionError(binary_request, e)
__main__.BinaryResolutionError: ("Error resolving binary request BinaryRequest(supportdir='bin/cmake', version='3.9.5', name='cmake', platform_dependent=True, external_url_generator=None, archiver=<pants.fs.archive.TarArchiver object at 0x10ce88390>): Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)", BinaryNotFound('Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)'))

Failed to build native engine.
 85.55s user 23.92s system 106% cpu 1:42.44 total; max RSS 630644Ki
 85.57s user 23.95s system 106% cpu 1:42.50 total; max RSS 630644Ki
[22347 18:29:16.337 eax@tw-mbp-eax ~/svn/pants !1!]∴PY=/opt/local/bin/python3.7 pants goals                         (git:pants)-[eax/maybe-junit-413]
Traceback (most recent call last):
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 454, in select
    downloaded_file = self._binary_tool_fetcher.fetch_binary(fetch_request)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 255, in fetch_binary
    self._do_fetch(bootstrapped_binary_path, file_name, urls)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 242, in _do_fetch
    with self._select_binary_stream(file_name, urls) as binary_tool_stream:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 238, in _select_binary_stream
    raise self.BinaryNotFound(name, accumulated_errors)
__main__.BinaryNotFound: Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 569, in <module>
    print(select(sys.argv))
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 565, in select
    return binary_util.select(binary_request)
  File "/Users/eax/svn/pants/src/python/pants/binaries/binary_util.py", line 456, in select
    raise self.BinaryResolutionError(binary_request, e)
__main__.BinaryResolutionError: ("Error resolving binary request BinaryRequest(supportdir='bin/cmake', version='3.9.5', name='cmake', platform_dependent=True, external_url_generator=None, archiver=<pants.fs.archive.TarArchiver object at 0x10d1c7c10>): Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)", BinaryNotFound('Failed to fetch cmake.tar.gz binary from any source: (Failed to fetch binary from <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz>: Fetch of <https://binaries.pantsbuild.org/bin/cmake/mac/10.13/3.9.5/cmake.tar.gz> failed with status code 400)'))

Failed to build native engine.

adamant-mouse-92628

04/22/2020, 1:29 AM

😕

aloof-angle-91616

04/22/2020, 1:30 AM

does the URL work when you curl it?

adamant-mouse-92628

04/22/2020, 1:32 AM

ah, ~/.netrc it is. One of these days I will succesfully bootstrap pants