https://pantsbuild.org/ logo
#general
Title
# general
n

narrow-activity-17405

07/14/2021, 1:20 PM
Within CI, I'm running tests on changed files (
./pants --changed-since=origin/master --changed-dependees=transitive test
). Some of the code within the repo depends on files stored using git lfs, but most of it doesn't. Right now, we are checking out lfs for every pull request, which generates unnecessary bandwidth. It would be great if (somehow) I could run lfs checkout conditionally - only for tests that need it. Any ideas on how this could be done please? :-)
h

happy-kitchen-89482

07/14/2021, 3:26 PM
Hmmmmmmm, just to be sure I understand - the LFS files are needed for the
--changed-since
check? Or do they only come into play later, when the test is run?
w

witty-crayon-22786

07/14/2021, 4:27 PM
that would depend on whether
git-lfs
makes placeholder files or not
… actually. sorry. it depends on whether
git diff --name-only
does the right thing when the contents have not been fetched.
but regardless, i think that we could definitely add support for either 1) poking
git-lfs
after doing a
--changed
calculation, 2) poking
git-lfs
while expanding globs
đź‘Ť 1
@narrow-activity-17405: in the meantime though, do know that you can split this and do something like:
./pants --changed-since=origin/master --changed-dependees=transitive filedeps --transitive
to get all affected files, make sure that
git-lfs
has materialized them, and then pass the list of files in to
./pants test
as a file with --spec-files
2) poking 
git-lfs
 while expanding globs
and… actually, this is maybe workable from a performance perspective, now that i think about it. if there are placeholder files, you don’t need to ask
git-lfs
to materialize them unless you want to capture their content: and that is a separate operation where it wouldn’t be too bad to interact with
git-lfs
(especially if you were able to do it lazily, where you only poked
git-lfs
if after capturing the file it was shaped like a placeholder file… i.e. contained a hash or w/e)
n

narrow-activity-17405

07/15/2021, 5:32 AM
@happy-kitchen-89482 The LFS files come into play when tests run.
Actually, tests are not directly depending on LFS files. It is one of their
runtime_package_dependencies
which needs them.
@witty-crayon-22786 Thanks for info and tips. Not sure if I got everything right, but at least I tried to run
./pants --changed-since=origin/master --changed-dependees=transitive filedeps --transitive
and it gives me quite a lot of files even when there are no changes - is this expected? :-)
w

witty-crayon-22786

07/15/2021, 3:50 PM
if there are no changes since master, nothing should be reported.
you can look at the “root” things that were identified by just running
./pants --changed-since=origin/master list
the two different “transitive” flags walk up and down from the roots, respectively
6 Views