For a new user with default limit. It worked after they changed it, but not a great first interaction. We at least have a section saying what to do when you encounter it on the install page: https://pants.readme.io/docs/installation

it’s fairly likely that 256 is too low, yea. but that’s not due to any known leak. if you take a look at

lsof -p

for a stable pantsd instance (not because this has anything to do with pantsd, but just because that’s the easiest way to inspect what we use in a stable state), you’ll see about 400 open files, depending on whether pants was loaded from a pex or wheels, or sources, etc

it could be reduced, but 256 … is just too low.

you’ll see lots of references to the lmdb database: that’s because we shard it 16 ways, and we have 3 different databases.

BUT, as i’m spieling, it does look like we might have two or three copies open. and that’s suspicious. we should only have one handle to the

Store

.

…actually, no. that’s the appropriate number of handles. each one is in a different mode.

Maye we can add

ulimit -n 10000

in the runner script, at least on macos, since we know with ~certainty that every macos user will encounter this.

mm, and i forgot that last time i noticed that you can call that directly from libraries

yea, worth doing.

FWIW I just got a new macbook running 10.15 and it looks like the default file handle limit got bumped to 2560

heh, the lazy 1 character change CL

that’s great news.

sorry to ask two questions in one day; currently hitting something I’m struggling to diagnose on my own. I’m trying to upgrade to pants 1.26 from 1.22 and seeing this exception. From my reading, checkstyle.py can’t propagate None to pex_build_util.py, so I guess it’s something happening downstream of that.

woops, got stacktrace got detected as a binary by slack, here it is:

why is pants deleting content from my init.py files?

Hi there, I am currently upgrading from pants 1.8 to 1.28 (yay), but am running into a slight issue with a requirement library. Most of our code is python3, but we do have a requirement on the old futures python2 backport. When I try to build with pants, it attempts to pull the futures module from the python3 repositories. I have tried these two things: 1. Setting the

interpreter_constraints

to include CPython2.7 2. Adding compatibility to the python_requirement_library like so:

python_requirement_library(
  name='futures',
  requirements=[python_requirement(name='futures', requirement='futures==3.2.0', compatibility='CPython>=2.7,<3')])

Any suggestions on how to pull that requirement from the python2 repositories or can you point me to some relevant documentation? Thanks in advance 🙂

have any of you seen this before?

15:27:16 00:49   [pyprep]
15:27:16 00:49     [interpreter]
                   Invalidated 602 targets.
15:27:20 00:53     [build-local-dists]
15:27:20 00:53     [requirements]
                   Invalidated 149 targets.ERROR: Double requirement given: six<2,>=1.9.0 (already in six==1.15.0, name='six')

happening somewhere in `~/.cache/pants/setup/bootstrap-Darwin-x86_64/1.26.0_py37/lib/python3.7/site-packages/pex/resolver.py”, line 521, in resolve_distributions I tried making sure the bootstrap’s pip is upgraded 20.1.1, no luck.

I have another question unrelated to my previous above. I have an egg file for a 3rd party dependency that is specified as a

python_requirement_library

. Within our pants.toml file, we have the following:

[python-repos]
repos = [
    "%(buildroot)s/3rdparty/repos"
  ]

In pants 1.8, that egg file was being picked up from the repos directory, but in 1.28 it is not able to be found. Do I need to specify it some other way?

Hi there, I got a question about coverage html report. We are currently on pants v.1.28 in pure v2 mode. After executing pytest, the junit xml results is generated successfully. How do I generate html coverage report on top of that? Pants was able to generate both xml and html report previously on v1 mode.

Curious if anyone else has seen this?

00:13:05:335 [DEBUG] pants.engine.internals.scheduler:pid=15743: computed 1 nodes in 1.049953 seconds. there are 12121 total nodes.
00:13:05:357 [ERROR] Build graph construction failed: ExecutionError 1 Exception encountered:
  Exception: Snapshot failed: Throw(Error { kind: Io(Os { code: 28, kind: Other, message: "No space left on device" }), paths: [] }, "<pants native internals>")

One other problem I've run into is output files. We have a task that produces an output and a binary that pulls in the task as a dependency. In 1.8, this output file was being wrapped into the resulting pex file, but is not occurring in 1.28. Has something changed underneath with how output files are handled? Example:

example_task(
  name='compile',
  sources=['./**/*'],
  output='output.cpython-36m-x86_64-linux-gnu.so',
  dependencies=[
    '3rdparty:numpy
    ]
)


In another BUILD file:

python_library(
  name='impl',
  sources=['cli/*.py'],
  dependencies=[
    'submodules/plugins/plugins:example_task',
  ]
)

python_binary(
  name='cli',
  zip_safe=False,
  sources=['cli/cli.py'],
  dependencies=[':impl'],
  tags={'default-build', 'docker-build'}
)

Im on an old version of pantsbuild (hopefully that’s not causing the problem), but I am a bit confused about how

[jar-dependency-management

use_managed

is supposed to work. I “pinned” a version of netty there, and when I use

./pants --resolver-resolver=ivy export --output-file=/tmp/a :app

I see that version. However, when I run

./pants resolve.ivy --open --no-cache-read :app

I see a different, newer version of netty. How do I know which version is actually being used?

hmm we lost our test coverage output after upgrading to 1.23.0, we had this:

[test.pytest]
coverage=auto

is that still the correct way to do it?

Hello, how can I find the binary file that is built from a

pants binary

goal?

Hi all,  We want to release Pants 2.0.0 in the next few weeks (that is, some time in mid-July).  This will be the first version to be based entirely on the v2 engine. We've created a document to discuss this and other issues: https://docs.google.com/document/d/1JgybVPtadftK0UXxmSdzmfTr8PmEvYwR2edwonHP66k/edit?usp=sharing Note that the initial release will focus on Python, and will not yet have support for JVM/Node/Go.  The document above outlines the reasoning for this, the timeline for adding v2 support for these and other languages, and the workarounds for using Pants v1 until that happens. Your comments, on this and all other issues, are welcome! We'll form a specific plan and create GitHub issues to track it based on your feedback.  Thanks in advance,  Benjy PS Also, if you'd like to be a launch partner for 2.0 (meaning we help you adopt it, and you give us feedback) let me know!

https://twitter.com/pantsbuild/status/1270762092316864512 The launch of Pants 2.0 is on the horizon! We'd love your feedback on the plan: https://groups.google.com/d/topic/pants-devel/CKKX8Ay-EmY/discussion #monorepo #python Twitter

Hi everyone! We've been using Pants for our Python-based machine learning/data science monorepo to test and build multiple libraries and PEX executables that share common dependencies. Now we are running the pants test and binary goals as part of our CI/CD pipeline (Gitlab CI). To take advantage of Pants' caching system, we persist the

.cache

and

.pants.d

directories from one CI pipeline to the next. However, we are now facing the issue that the

.pants.d

directory regularly grows to 15+GB, which is a lot to download/upload in each CI stage and we are actually running into disk space issues with shared Gitlab runners. Does someone know if there is a way to purge or clean the cache directories without losing the ability to only rebuild the changed projects within the monorepo? We've naively tried to only persist the

.cache

directory from one run to the next, however, that lead to every project being rebuilt in every CI run. Any help or input would be appreciated!

Hey @better-nail-12700 i'm working on using pants for an ML monorepo too! Have you encountered any problems that are specific to ML libraries or processes? Could you share any insights you've learned through doing this?