Hey, after updating from pants 1.25 to 1.29 we see this error sometimes in our CI runs:

Build graph construction failed: ExecutionError 1 Exception encountered:
  Exception: Snapshot failed: Throw { val: Error { kind: Io(Os { code: 28, kind: Other, message: "No space left on device" }), paths: [] }, python_traceback: "Traceback (no traceback):\n  <pants native internals>\nException: Error { kind: Io(Os { code: 28, kind: Other, message: \"No space left on device\" }), paths: [] }", engine_traceback: ["Fingerprinting: <SOME SCALA FILE>.scala"] }

pants is invoked in gitlab-ci inside a docker container. The host has enough diskspace available. Where is it trying to write?

@plain-river-51682 A couple of questions: • Does the job work sometimes on your CI and fails sometimes for the same tests/builds? • What is your setting for

pants_workdir

? • Is the repo mounted as read-only in the docker container?

Does the job work sometimes on your CI and fails sometimes for the same tests/builds?

yes, when we retry the same job, it suddenly succeeds. Though every retry gets checked out in a new docker container

Is the repo mounted as read-only in the docker container?

the repo and all paths are fully writeable

What is your setting for 

pants_workdir

?

we havent touched this, so I guess it's using the default?

The default is

${REPO_PATH}/.pants.d

. This shouldn't be an issue if the repo is read-write mounted.

Ya, this path is writable.

Under macOS we've had to enter a custom

ulimit

(17000). I see

docker run

has an option to set this for containers (

--ulimit rtprio=<value>

). It is a stab in the dark, but if it is easy to set there is no harm in trying.

worth confirming that you’re not hitting https://www.pantsbuild.org/docs/troubleshooting#no-space-left-on-device-error-while-watching-files

that might be it, we give it a try!