https://pantsbuild.org/ logo
#general
Title
# general
a

acceptable-guitar-79854

10/21/2020, 8:55 PM
I have a few questions about the file system stuff: • what is the difference between what goes into
~/.cache/pants/named_caches/pex_root
and
~/.pex/
? • is there a way to set where
named_caches/
,
lmdb_store
go? e.g. somewhere on NFS
h

hundreds-father-404

10/21/2020, 9:03 PM
~/.pex
is when you use Pex outside of Pants.
~/.cache/pants/named_caches/pex_root
is the same thing, but isolated.
is there a way to set where
Yes, I really need to finish https://www.pantsbuild.org/docs/troubleshooting#how-to-change-your-cache-directory. Change
named_caches_dir
and
local_store_dir
in the
[GLOBAL]
scope
❤️ 1
a

acceptable-guitar-79854

10/21/2020, 9:09 PM
great, thank you
h

hundreds-father-404

10/23/2020, 11:34 PM
❤️ 1
✔️ 1
a

acceptable-guitar-79854

10/24/2020, 6:52 PM
thank you @hundreds-father-404 this looks great and concise, I would still have a couple of related questions: 1. why
~/.pex
is populated when running pants? i don't know why it appears there when pants already uses
named_caches
2. what happens if multiple concurrent builds point to the same
lmdb_store
/
named_caches
say somewhere on NFS
❤️ 1
3. the difference with
$PANTS_HOME
(i know the answer to 3 but just thought that it may be relevant)
h

hundreds-father-404

10/25/2020, 5:28 PM
1. I wouldn’t expect you to have that folder from running Pants, and it’s a bug if you do. Have you ever run Pex directly? Like
pip install pex
, then
pex -o my_pex req1 req2
?. 2. That should be safe afaict.
lmdb_store
uses the Lightning Memory-Mapped Database scheme: https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database, and can handle multiple projects writing at the same time.
named_caches
is exclusively append-only caches, i.e. that avoid overwriting anything. (This is why it’s so tricky for us to leverage MyPy’s cache in a safe way, which mutates) 3. I think
$PANTS_HOME
is only used by the
./pants
script to determine where to put
~/.cache/pants/setup
. Perhaps we should rename it to something like
PANTS_SETUP_CACHE
.
a

acceptable-guitar-79854

10/25/2020, 7:50 PM
Great answer, thank you @hundreds-father-404 Yes, that's what I observed PANTS_HOME does. I just validated again: 1.
rm -rf ~/.pex/
2.
du -sh ~/.pex/
-> no such file or directory 3.
./pants fmt ::
• initialising pantsd • finding interpreter to satisfy constraints • run isort, docformatter (without building isort, docformatter) 4.
du -sh ~/.pex/
-> 17M
1
h

hundreds-father-404

10/25/2020, 7:56 PM
🙏 1
👍 1
I reproduce that, thanks for the find! That’s a bug that we’re using
~/.pex
when running PEXes (but not when building them)
👍 1
Fix: https://github.com/pantsbuild/pants/pull/11040 I’m really glad you caught that before 2.0.0 early this week. Thanks for the find!
🙏 1
❤️ 1
a

acceptable-guitar-79854

10/25/2020, 8:29 PM
that was really really fast!! Thank you @hundreds-father-404
🐇 1