https://pantsbuild.org/ logo
#general
Title
# general
r

rough-vase-83553

10/19/2022, 7:27 PM
I'm seeing a weird probably-bug after adding a
files(name = "all-the-files", sources = ["**"])
target to the repo root (don't ask me why simple smile). I'm guessing it's because we have some paths in the repository that contain asterisk (don't ask me why simple smile).
Copy code
InvalidFieldException: The 'source' field in target //some/project/data/[*]/_SUCCESS:../../../../../all-the-files should not include `*` globs, but was set to some/project/data/[*]/_SUCCESS. Instead, use a literal file path (relative to the BUILD file).
The actual path is
some/project/data/*/_SUCCESS
.
I don't actually have any BUILD files covering this directory or its parents below the root
h

hundreds-father-404

10/19/2022, 7:30 PM
you can set something like this:
Copy code
[GLOBAL]
pants_ignore.add = ["some/project/data/*"]
I'm not surprised that Pants is upset with a file path with
*
embedded in it
👍 1
r

rough-vase-83553

10/19/2022, 7:31 PM
That's a good unblocker for now. Although eventually this might pose a problem for adoption and might not be what the team intended so just LYK.
👍 1
h

hundreds-father-404

10/19/2022, 7:32 PM
Mind sharing the use case for
*
in a file path?
😅 1
r

rough-vase-83553

10/19/2022, 7:33 PM
I'd have to hunt down the codeowners and ask them 😉. Which I might eventually do but maybe not out of the blue before the build system gains actual traction around here.
h

hundreds-father-404

10/19/2022, 7:47 PM
Okay - if you're motivated enough, I suspect it's plausible to make the codecheck more robust to allow
*
to work. Maybe something like escaping it. (We still want to error generally for
*
because it's a core assumption that a
file
target respond to exactly 1 file) This is the error you're hitting: https://github.com/pantsbuild/pants/blob/08410e5b94d3e3aa86b2538ee1991d964e27e72c/src/python/pants/engine/target.py#L2184-L2193
👀 1
r

rough-vase-83553

10/19/2022, 8:55 PM
Oh I see. This is kind of a fail-fast to prevent accidental use of globs where paths are required
Not too familiar with the architecture but would it be possible to check whether the file actually exists before making that assertion?
I suspect not... in which case you'd need to make this message appear when the filesystem lookup fails
h

hundreds-father-404

10/19/2022, 9:01 PM
Oh I see. This is kind of a fail-fast to prevent accidental use of globs where paths are required
Exactly
Not too familiar with the architecture but would it be possible to check whether the file actually exists before making that assertion
Yeah, although there's more complexity here that the Rust engine will interpret
*
as a glob to match everything in that directory. Which is not the semantics you're looking for here -- you want
*
to be interpreted as a literal character Which is where I think escaping is relevant, figuring out how you can escape
*
so that the engine treats it as a literal and not a glob
👀 1
r

rough-vase-83553

10/19/2022, 9:04 PM
Yeah it seems like the standard
\
escape is what you'd use in a glob engine. At least based on the fact
echo **/\*/**
works as I'd expect on my shell.
h

hundreds-father-404

10/19/2022, 9:07 PM
you could experiment quickly with this by using
./pants count-loc '<path>'
, where the
''
is to avoid shell escaping from happening. That will have the Rust engine's glob semantics kick in. Maybe set up something like this:
Copy code
dir/
dir/a.txt
dir/b.txt
dir/*/
dir/*/a.txt
dir/*/b.txt
experiment with how quoting gets different results
once you find the right way to escape
*
, then a patch would be welcomed to update https://github.com/pantsbuild/pants/blob/08410e5b94d3e3aa86b2538ee1991d964e27e72c/src/python/pants/engine/target.py#L2184-L2193 to have a more advanced string check, like banning
*
except for the allowlisted value of
\*
or whatever it is
r

rough-vase-83553

10/19/2022, 9:10 PM
Hmm so you're suggesting that the user would e.g. do
file(source="my/asterisk/\*/path")
instead of
file(source="my/asterisk/*/path")
?
h

hundreds-father-404

10/19/2022, 9:11 PM
yeah, but I'm not sure what the escape is in this case
r

rough-vase-83553

10/19/2022, 9:11 PM
And similarly
files
would expand out globs into the escaped paths rather than the literal result of the glob expansion
Makes sense. I'll summarize this discussion in an issue
h

hundreds-father-404

10/19/2022, 9:12 PM
which it may already be doing? from top of the thread
some/project/data/[*]/_SUCCESS
👍 1
or are the
[]
in the actual file path?
r

rough-vase-83553

10/19/2022, 9:13 PM
No the macro (?) seems to add that itself. Yeah wondering why that's being used rather than
\
5 Views