https://pantsbuild.org/ logo
#general
Title
# general
v

victorious-animal-71161

06/11/2022, 11:40 AM
Question: Is it possible to set pytest's root directory to somewhere other than
/tmp
when running test with pants? Our test code is having trouble reading
docker-compose.yml
(the purpose of this file is to create a test redis cluster) when launching snap-based docker, since snap based docker can't access to
/tmp
.
Looks like root directory is created by this line...
h

happy-kitchen-89482

06/11/2022, 11:54 AM
v

victorious-animal-71161

06/11/2022, 12:06 PM
Unfortunatly that didn't work. I've already set
[GLOBAL].local_execution_root_dir
, but regardless of that option, pants still creates new workdir under
/tmp
:(
h

happy-kitchen-89482

06/12/2022, 9:23 AM
Is this the pants workdir itself, or a tmpdir your test is creating?
What is the actual path ?
v

victorious-animal-71161

06/13/2022, 2:33 AM
It’s a directory for pytest, which contains a copy of the testing codes and resources (as defined on BUILD file).
Screen Shot 2022-06-13 at 11.33.31 AM.png
the path described at
rootdir
column
h

happy-kitchen-89482

06/18/2022, 6:25 PM
v

victorious-animal-71161

06/20/2022, 1:25 PM
Tried but no luck 😞 added TMPDIR env var with
--subprocess-environment-env-vars="['TMPDIR=$(pwd)/.tmp']"
flag, but pytest still starts from
/tmp
. I’ll keep look for some other options.
r

rhythmic-morning-87313

07/10/2022, 2:05 PM
maybe should we change an option for
pex
to override its current working directory?
TMPDIR
seems to have no effects in this case.
pytest
itself does not assume anything about the current working directory.
The
[GLOBAL].local_execution_root_dir
setting in
pants.toml
has no effect neither.
image.png
maybe more related with this case? hmm
but it is about changing the named cache directory to the same filesystem where /tmp resides
image.png
maybe this could be the source
hmmmm
I added extra logging to this function
then suddnely it began to use the correct path;
image.png
after restarting pantds several times it began again..
image.png
the cwd of the
setup_pex_cli_process()
seems to be legitimate.. hmm
hmmmm
this seems to be race condition
image.png
depending on existence of these highlighted 3 lines
the result of
Path.cwd()
in my test fixture differs
.........
hmm
image.png
the source of this path seems to be a temporary venv
i'm tracking through
pex.py
,
process.py
, ... to determine the source of this tmp directory, but still i cannot find it
...
i discovered the source of all the paths under it.. but not itself...
hmmm
image.png
image.png
image.png
maybe SANDBOX_ROOT...?
VENVSCRIPT SANDBOX_ROOT = /tmp/.tmpebTVkM/.
....
e

enough-analyst-54434

07/10/2022, 4:26 PM
I find @rhythmic-morning-87313's suggestion of setting
local_execution_root_dir
definitely works. @victorious-animal-71161 here's what I ran on Pants main with commentary:
Copy code
$ mkdir bob
$ ./pants \
    --local-execution-root-dir=bob \
    --pants-ignore=bob \
    --pytest-args="[]" \
    test \
    --force \
    --output=all \
    src/python/pants/util/strutil_test.py \
    -- -vvs
10:20:09.03 [INFO] Completed: Run Pytest - src/python/pants/util/strutil_test.py:tests succeeded.
============================= test session starts ==============================
platform linux -- Python 3.7.13, pytest-7.0.1, pluggy-1.0.0 -- /home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/06648b60/venv/bin/python3.7
cachedir: .pytest_cache
metadata: {'Python': '3.7.13', 'Platform': 'Linux-5.18.10-arch1-1-x86_64-with-arch', 'Packages': {'pytest': '7.0.1', 'py': '1.11.0', 'pluggy': '1.0.0'}, 'Plugins': {'metadata': '2.0.1', 'html': '3.1.1', 'cov': '3.0.0', 'asyncio': '0.18.3', 'icdiff': '0.5'}}
rootdir: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK, configfile: pyproject.toml
plugins: metadata-2.0.1, html-3.1.1, cov-3.0.0, asyncio-0.18.3, icdiff-0.5
asyncio: mode=auto
collecting ... collected 17 items

src/python/pants/util/strutil_test.py::test_pluralize PASSED
...
src/python/pants/util/strutil_test.py::test_fmt_memory_sizes[2199023255552-2048GiB] PASSED

- generated xml file: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK/src.python.pants.util.strutil_test.py.tests.xml -
============================== 17 passed in 0.13s ==============================



✓ src/python/pants/util/strutil_test.py:tests succeeded in 0.28s.
+ The
mkdir bob
was needed due to a bug (https://github.com/pantsbuild/pants/issues/16119) + The
--pants-ignore=bob
was needed since
bob
is a relative directory (in the build root) and is otherwise watched by pantsd which leads to continual re-starts. + The `--pytest-args="[]"`is to turn pytest headers back on where rootdir is printed (The Pants repo turns those headers off with
--no-header
in its
pants.toml
) + The
--force --output=all
is to make sure the test re-runs so we can see the result of the experiment. + The
-- -vvs
is to pass pytest some verbosity args directly. You'll notice the result of the experiment in the pytest header - it sees:
rootdir: /home/jsirois/dev/pantsbuild/jsirois-pants/bob/pants-sandbox-ppykzK, configfile: pyproject.toml
@victorious-animal-71161 what version of Pants are you using?
r

rhythmic-morning-87313

07/10/2022, 4:32 PM
in the interactive mode
<http://intrinsic.rs|intrinsic.rs>
seems to create
maybe_tempdir
and
image.png
run_in_workspace
is expected to be set false when run interactively
e

enough-analyst-54434

07/10/2022, 4:33 PM
@rhythmic-morning-87313 are you debugging https://github.com/pantsbuild/pants/issues/16119 here with your last few comments?
r

rhythmic-morning-87313

07/10/2022, 4:33 PM
(i believe Kyujin is sleeping)
e

enough-analyst-54434

07/10/2022, 4:34 PM
Yeah - I treat this as an async medium. I figure folks will read at an appropriate time in their timezone.
r

rhythmic-morning-87313

07/10/2022, 4:34 PM
no, i'm just trying to figure out where the hell
/tmp/.tmpxxxxx/.tmp
comes from when
local_execution_root_dir
is set to use
%(build_root)s/.tmp
e

enough-analyst-54434

07/10/2022, 4:35 PM
Ah, gotcha.
r

rhythmic-morning-87313

07/10/2022, 4:35 PM
(note: Snap-based docker does not allow using
/tmp
in the host to mount inside the container..)
image.png
in the pants' python side, there seems no explicit assignment of the working directory of a venv pex process
now i'm suspecting the rust engine side...
e

enough-analyst-54434

07/10/2022, 4:37 PM
I still don't follow. In your screenshot the pytest rootdir is shown as in /tmp. In my example above, it is not.
r

rhythmic-morning-87313

07/10/2022, 4:38 PM
image.png
here is the first point i can observe
/tmp
path in the pants code
e

enough-analyst-54434

07/10/2022, 4:38 PM
@rhythmic-morning-87313 how are you running Pants?
I listed exactly how I ran Pants in my example above.
r

rhythmic-morning-87313

07/10/2022, 4:39 PM
before the pytest process starts
Copy code
./pants-local test --debug tests/agent/docker/test_agent.py:tests -- -k test_auto_pull_digest_when_digest_matching -s -v
my pants is based on 2.12.0 with linux-aarch64 patches
e

enough-analyst-54434

07/10/2022, 4:40 PM
Sure - yes - you have not re-directed --local-execution-root-dir so that is expected.
r

rhythmic-morning-87313

07/10/2022, 4:40 PM
it's configured in pants.toml
e

enough-analyst-54434

07/10/2022, 4:40 PM
Can you try on the CLI like I did?
r

rhythmic-morning-87313

07/10/2022, 4:40 PM
image.png
e

enough-analyst-54434

07/10/2022, 4:40 PM
Then we can work backwards from there.
r

rhythmic-morning-87313

07/10/2022, 4:41 PM
ok
image.png
e

enough-analyst-54434

07/10/2022, 4:41 PM
Yes, please read my whole example. I filed a bug for that. You need to pre-created the dir.
r

rhythmic-morning-87313

07/10/2022, 4:42 PM
image.png
and...
image.png
it still uses
/tmp/.tmpxxxx
i showed the missing dir error to indicate that the cli arg has effects
e

enough-analyst-54434

07/10/2022, 4:44 PM
Ok, I want to back up slightly. Kyujin claimed if pytest rootdir could be moved off /tmp, things would be good. My example shows that works. Your example is showing something else IIUC. Another leak?
r

rhythmic-morning-87313

07/10/2022, 4:45 PM
hmm
i'm new to rust, and how could i add some logs in the engine code?
around
interactive_process()
in
<http://intrinsic.rs|intrinsic.rs>
i'm suspecting two places:
interactive_process()
and some bits of immutable digest stuffs
where it seems to create temp dirs implicitly
e

enough-analyst-54434

07/10/2022, 4:47 PM
The `eprintln!`macro is probably what you want. Do you have snap-based docker locally so you can actually test you're solving the OP problem? See: https://doc.rust-lang.org/std/macro.eprintln.html
r

rhythmic-morning-87313

07/10/2022, 4:48 PM
yes, we have the same development setups
e

enough-analyst-54434

07/10/2022, 4:48 PM
Ah, great.
Thanks for poking at this.
r

rhythmic-morning-87313

07/10/2022, 4:51 PM
now compiling..
ahha i need to use
:?
or
:#?
... building again
ok it ran...
but there seems to be no place the above logs printed...
grepping the console output does not have them. hm
e

enough-analyst-54434

07/10/2022, 4:58 PM
Look in
.pants.d/pants.log
or run with
--no-pantsd
👀 1
r

rhythmic-morning-87313

07/10/2022, 5:00 PM
still nothing..
ah
pants.log has it
image.png
yes, it's coming from here
e

enough-analyst-54434

07/10/2022, 5:02 PM
Ok, I'm not following your live debug session too closely, but when you've gathered your investigation to some conclusions or final set of questions, feel free to loop me back in with a summary issue or PR. I'm happy to review either.
r

rhythmic-morning-87313

07/10/2022, 5:03 PM
i'm still not sure whether this is an expected behavior or not
but i think i have enough data to post a github issue
e

enough-analyst-54434

07/10/2022, 5:03 PM
Great - thank you.
r

rhythmic-morning-87313

07/10/2022, 5:07 PM
and.. i just found that this makes my pytest session to behave differently in interactive/non-interactive modes
i originally hit this problem because my pytest session got stuck (timeout) in GitHub Action workflow and just added
--debug
flag to see what happens locally
using the same setup, removing
--debug
makes...
image.png
infinite stream of "filesystem changed" logs
i'm not sure whether this issue originates from the same source or not...
e

enough-analyst-54434

07/10/2022, 5:33 PM
The infinite changed is what requires
--pants-ignore=bob
in my example above - it is expected. The interactive bit is the key thing I was missing. I was not using that in my testing above. So I think the summary is non-interactive works fine? But
--debug
does not, which is tracked now by the issue you filed.
r

rhythmic-morning-87313

07/10/2022, 5:36 PM
oh, adding
--pants-ignore=bob
resolves the infinite loop
e

enough-analyst-54434

07/10/2022, 5:38 PM
@rhythmic-morning-87313 as an aside, you're successfully using Pants on Linux arm with a custom build of Pants from sources?
r

rhythmic-morning-87313

07/10/2022, 5:44 PM
yeah, i've not seen any issues specific to linux arm setup
e

enough-analyst-54434

07/10/2022, 5:46 PM
Ok, great, I'll mention your success in another thread. Thank you.
🙌 1
5 Views