Question: Is it possible to set pytest's root directory to somewhere other than

/tmp

when running test with pants? Our test code is having trouble reading

docker-compose.yml

(the purpose of this file is to create a test redis cluster) when launching snap-based docker, since snap based docker can't access to

/tmp

.

I think @rhythmic-morning-87313 already figured out the solution here: https://pantsbuild.slack.com/archives/C046T6T9U/p1652695366750999?thread_ts=1652686882.500409&cid=C046T6T9U ?

Unfortunatly that didn't work. I've already set

[GLOBAL].local_execution_root_dir

, but regardless of that option, pants still creates new workdir under

/tmp

:(

Is this the pants workdir itself, or a tmpdir your test is creating?

It’s a directory for pytest, which contains a copy of the testing codes and resources (as defined on BUILD file).

Hmm, try setting TMPDIR using this option: https://www.pantsbuild.org/docs/reference-subprocess-environment#section-env-vars ?

Tried but no luck 😞 added TMPDIR env var with

--subprocess-environment-env-vars="['TMPDIR=$(pwd)/.tmp']"

flag, but pytest still starts from

/tmp

. I’ll keep look for some other options.

maybe should we change an option for

pex

to override its current working directory?

TMPDIR

seems to have no effects in this case.

I find @rhythmic-morning-87313's suggestion of setting

local_execution_root_dir

definitely works. @victorious-animal-71161 here's what I ran on Pants main with commentary:

$ mkdir bob
$ ./pants \
    --local-execution-root-dir=bob \
    --pants-ignore=bob \
    --pytest-args="[]" \
    test \
    --force \
    --output=all \
    src/python/pants/util/strutil_test.py \
    -- -vvs
10:20:09.03 [INFO] Completed: Run Pytest - src/python/pants/util/strutil_test.py:tests succeeded.
============================= test session starts ==============================
platform linux -- Python 3.7.13, pytest-7.0.1, pluggy-1.0.0 -- /home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/06648b60/venv/bin/python3.7
cachedir: .pytest_cache
metadata: {'Python': '3.7.13', 'Platform': 'Linux-5.18.10-arch1-1-x86_64-with-arch', 'Packages': {'pytest': '7.0.1', 'py': '1.11.0', 'pluggy': '1.0.0'}, 'Plugins': {'metadata': '2.0.1', 'html': '3.1.1', 'cov': '3.0.0', 'asyncio': '0.18.3', 'icdiff': '0.5'}}
rootdir: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK, configfile: pyproject.toml
plugins: metadata-2.0.1, html-3.1.1, cov-3.0.0, asyncio-0.18.3, icdiff-0.5
asyncio: mode=auto
collecting ... collected 17 items

src/python/pants/util/strutil_test.py::test_pluralize PASSED
...
src/python/pants/util/strutil_test.py::test_fmt_memory_sizes[2199023255552-2048GiB] PASSED

- generated xml file: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK/src.python.pants.util.strutil_test.py.tests.xml -
============================== 17 passed in 0.13s ==============================



✓ src/python/pants/util/strutil_test.py:tests succeeded in 0.28s.

+ The

mkdir bob

was needed due to a bug (https://github.com/pantsbuild/pants/issues/16119) + The

--pants-ignore=bob

was needed since

bob

is a relative directory (in the build root) and is otherwise watched by pantsd which leads to continual re-starts. + The `--pytest-args="[]"`is to turn pytest headers back on where rootdir is printed (The Pants repo turns those headers off with

--no-header

in its

pants.toml

) + The

--force --output=all

is to make sure the test re-runs so we can see the result of the experiment. + The

-- -vvs

is to pass pytest some verbosity args directly. You'll notice the result of the experiment in the pytest header - it sees:

rootdir: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK, configfile: pyproject.toml

@victorious-animal-71161 what version of Pants are you using?

in the interactive mode

@rhythmic-morning-87313 are you debugging https://github.com/pantsbuild/pants/issues/16119 here with your last few comments?

(i believe Kyujin is sleeping)

Yeah - I treat this as an async medium. I figure folks will read at an appropriate time in their timezone.

no, i'm just trying to figure out where the hell

/tmp/.tmpxxxxx/.tmp

comes from when

local_execution_root_dir

is set to use

%(build_root)s/.tmp

Ah, gotcha.

(note: Snap-based docker does not allow using

/tmp

in the host to mount inside the container..)

I still don't follow. In your screenshot the pytest rootdir is shown as in /tmp. In my example above, it is not.

image.png

@rhythmic-morning-87313 how are you running Pants?

before the pytest process starts

Sure - yes - you have not re-directed --local-execution-root-dir so that is expected.

it's configured in pants.toml

Can you try on the CLI like I did?

image.png

Then we can work backwards from there.

ok

Yes, please read my whole example. I filed a bug for that. You need to pre-created the dir.

image.png

Ok, I want to back up slightly. Kyujin claimed if pytest rootdir could be moved off /tmp, things would be good. My example shows that works. Your example is showing something else IIUC. Another leak?

hmm

The `eprintln!`macro is probably what you want. Do you have snap-based docker locally so you can actually test you're solving the OP problem? See: https://doc.rust-lang.org/std/macro.eprintln.html

yes, we have the same development setups

Ah, great.

now compiling..

Look in

.pants.d/pants.log

or run with

--no-pantsd

still nothing..

Ok, I'm not following your live debug session too closely, but when you've gathered your investigation to some conclusions or final set of questions, feel free to loop me back in with a summary issue or PR. I'm happy to review either.

i'm still not sure whether this is an expected behavior or not

Great - thank you.

and.. i just found that this makes my pytest session to behave differently in interactive/non-interactive modes

The infinite changed is what requires

--pants-ignore=bob

in my example above - it is expected. The interactive bit is the key thing I was missing. I was not using that in my testing above. So I think the summary is non-interactive works fine? But

--debug

does not, which is tracked now by the issue you filed.

oh, adding

--pants-ignore=bob

resolves the infinite loop

@rhythmic-morning-87313 as an aside, you're successfully using Pants on Linux arm with a custom build of Pants from sources?

yeah, i've not seen any issues specific to linux arm setup

Ok, great, I'll mention your success in another thread. Thank you.