https://pantsbuild.org/ logo
Join Slack
Powered by
Hello everyone, I have a BUILD file under a `.ci` ...
# general
w
Hello everyone, I have a BUILD file under a 
.ci
folder and I can’t get it to show up when I run 
list
or any other goal. I have this bad feeling that if it’s a hidden folder pants ignores it but just checking here first.
h
It likely is. Run 
./pants help-advanced | grep -A 10 'pants_ignore'
It defaults to ignoring files/folders that start with 
.
, in addition to reading from your 
.gitignore
. You can add back the value by setting:
Copy code
[GLOBAL]
pants_ignore.add = ["!.ci/*"]
w
😞 didnt work.. I tried even redefining the 
pants_ignore
which is now like this:
Copy code
default: [
          ".*/",
          "/dist/"
      ]
      current value: [
          "/dist/"
      ] (from pants.toml)
but can’t find the 
.ci
folder (it’s not on my .gitignore)
I actually put my helm charts/terraform all under a 
services
folder but DevOps folks want me to use their “standard” which are: 
.terraform
and 
.helm
together with that 
.ci
folder so I’m not sure what to do haha
h
For 
./pants list
, to confirm, do you have BUILD files in those directories? Pants will only output things with 
./pants list
if there is a target that refers to those source files
w
Yes, I do have a BUILD file inside 
.ci
folder and 
./pants list ::
isnt showing it.. and when I try 
./pants list .ci::
it throws an error
h
Try this for the sake of debugging 
./pants --pants-ignore='[]' --no-pants-ignore-use-gitignore list .ci::
Also at the risk of being pedantic, double check that there are targets in the BUILD file
h
I think I can replicate this issue myself
👍 1
if I make a directory in the root of the pants repo called 
zzz
with a 
BUILD
file with the contents 
python_library()
and nothing else
hten 
./pants list ::
will display a 
zzz
entry
w
Copy code
Engine traceback:
  in select
  in `list` goal
  in pants.engine.internals.build_files.strip_address_origins
  in Find targets from input specs
  in pants.engine.internals.build_files.addresses_with_origins_from_address_specs
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/native.py", line 69, in generator_send
    res = func.send(arg)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/build_files.py", line 214, in addresses_with_origins_from_address_specs
    addr_target_pairs_for_spec = glob_spec.matching_addresses(addr_families_for_spec)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/base/specs.py", line 130, in matching_addresses
    raise ResolveError(f"Address spec '{self}' does not match any targets.")
pants.base.exceptions.ResolveError: Address spec '.ci::' does not match any targets.
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 253, in run
    engine_result = self._run_v2()
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 164, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 187, in _maybe_run_v2_body
    poll_delay=(0.1 if poll else None),
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 142, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 568, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 532, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in select
  in `list` goal
  in pants.engine.internals.build_files.strip_address_origins
  in Find targets from input specs
  in pants.engine.internals.build_files.addresses_with_origins_from_address_specs
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/native.py", line 69, in generator_send
    res = func.send(arg)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/build_files.py", line 214, in addresses_with_origins_from_address_specs
    addr_target_pairs_for_spec = glob_spec.matching_addresses(addr_families_for_spec)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/base/specs.py", line 130, in matching_addresses
    raise ResolveError(f"Address spec '{self}' does not match any targets.")
pants.base.exceptions.ResolveError: Address spec '.ci::' does not match any targets.
h
if I rename it to 
.zzz
it disappears, even if I add a line in 
pants_ignroe.add
"!/.zzz/*",
👍 2
so maybe this is a bug with the pants_ignore functionality
w
oh no 🐛
🐛 1
h
It sounds like it is. Thank you for the report, Thales. Let’s definitely prioritize fixing in beta2, and likely backporting to 1.30.2rc1
👍 2
h
I'll start digging into this @hundreds-father-404
h
Thank you Greg!
h
so I think what might be going on here
is that we have 
".*/"
as a default for the 
--pants-ignore
option in global_options.py
and this is somehow blocking overrides defined afterwards by 
pants.toml
h
I don’t think that’s it. If I override 
pants_ignore
in 
pants.toml
, then print what we pass to Rust, I get
Copy code
['/build-support/virtualenvs/', '/.pants.d', '/dist', '/.pids']
Given this diff:
Copy code
diff --git a/pants.toml b/pants.toml
index f014b0c47..965aad8cf 100644
--- a/pants.toml
+++ b/pants.toml
@@ -27,18 +27,18 @@ pantsd_invalidation_globs.add = [
   "src/rust/engine/**/*.toml",
 ]
 # Path patterns to ignore for filesystem operations on top of the builtin patterns.
-pants_ignore.add = [
+pants_ignore = [
   # venv directories under build-support.
   "/build-support/virtualenvs/",
-  "/build-support/*.venv/",
+#  "/build-support/*.venv/",
   # An absolute symlink to the Pants Rust toolchain sources.
-  "/build-support/bin/native/src",
+#  "/build-support/bin/native/src",
   # We shouldn't walk or watch the rust compiler artifacts because it is slow.
-  "/src/rust/engine/target",
+#  "/src/rust/engine/target",
   # We want to .gitignore Java .class files, but pants should pay attention to them.
-  "!*.class",
+#  "!*.class",
   # We also want to override the .gitignore'd pants.pex file
-  "!/pants.pex",
+#  "!/pants.pex",
 ]
 
 [source]
diff --git a/src/python/pants/init/options_initializer.py b/src/python/pants/init/options_initializer.py
index c94c47709..752eeb2c3 100644
--- a/src/python/pants/init/options_initializer.py
+++ b/src/python/pants/init/options_initializer.py
@@ -107,6 +107,7 @@ class OptionsInitializer:
         add(global_options.pants_distdir)
         add(global_options.pants_subprocessdir)
 
+        print(pants_ignore)
         return pants_ignore
 
     @staticmethod
h
@hundreds-father-404 does the problem still occur when completely removing 
.*/
from pants-ignore?
I'll try it myself in a sec
ah, actually, I think that something else might be adding 
.*
to the ignore struct, even if that pants option is unset
👍 1
huh we call 
GitIgnoreStyleExcludes::create_with_gitignore_file
a lot
👍 1
that seems unnecessary...
uh, maybe the problem is that we should be using --build-ignore and not --pants-ignore...
@wooden-thailand-8386 I think you might want to be using 
--build-ignore
option instead of the 
--pants-ignore
option
I am actually not sure why the distinction exists between these two options. @hundreds-father-404?
w
tried it with build-ignore, same thing
h
hm, ok
h
Greg, 
build_ignore
is to ignore 
BUILD
files in particular. It’s useful, for example, if you are still using Pants v1 for JVM and have a v1 config file that says ignore 
src/python
and a v2 config file that says ignore 
src/java
I think this is a bug in 
pants_ignore
h
ok, so with the current state of the codebase, if you add the directory itself to 
pants_ignore.add
, and not every file in that subdir, I believe that will work
that is, add 
!/.ci
, rather than 
!/.ci/*
@wooden-thailand-8386
h
When debugging, also try running 
./pants count-loc '.ci/*'
. This will check the behavior of 
--pants-ignore
Greg and I realized 
--build-ignore
is relevant, indeed. Even if 
--pants-ignore
is correct, if 
--build-ignore
is saying to ignore BUILD files, it will look to Pants like there are no targets in that directory. The above command will help to better isolate if either one or both of the options are the issue
h
it's definitely possible to get into a state where 
./pants count-loc '.ci/*'
will work, but 
.ci
won't show up in 
./pants list ::
if 
pants_ignore.add
has 
!/.ci
but 
build_ignore
isn't touched
it's also the case that if you have 
build_ignore.add = ["!/.ci"]
, the default 
build_ignore
.*/
is getting applied after the custom 
.add
, so the explicit allow of 
.ci
doesn't affect the final matching behavior
that is a legitimate bug I believe
h
it’s also the case that if you have build_ignore.add = [“!/.ci”], the default build_ignore .*/ is getting applied after the custom .add
Does the ordering impact things? That’s generic options code, that 
my_opt.add
appends to the end of the prior values
h
as far as I'm aware, ordering matters
"Each line in a 
gitignore
file specifies a pattern. When deciding whether to ignore a path, Git normally checks 
gitignore
patterns from multiple sources, with the following order of precedence, from highest to lowest (within one level of precedence, the last matching pattern decides the outcome):"
👍 1
that file also has a section on pattern format, which we probably want to explicitly reference in our own docs on these options
@wooden-thailand-8386 so, specifically, right now it is necessary to 1) add 
!/.ci/
to 
pants_ignore.add
and 2) to remove the default 
".*/"
glob from the 
build-ignore
option by adding 
build_ignore.remove = [".*/"]
to your cargo.toml. that should make it possible to see 
.ci
show up in pants goals
🤩 1
and I think based on this conversation that we want to remove that problematic 
.*/
as a default glob in the 
--build-ignore
option, so in the next release that has that change, 2) will no longer be necessary
👍 1
w
Awesome! I just turned on my work laptop to add those changes already haha, thanks for all the digging around to find that fix.
btw, you guys are on the East Coast right?
West*
h
the bulk of us are in the bay area, a few of us are elsewhere. I'm personally in the bay area
w
haha ok, that makes sense 🙂 I’m on East Coast so if I dont reply it’s probably bc it’s already late for me
h
no worries 🙂
w
Should we wait the fix on v2 to add that information into the documentation?
h
I think probably. But would love any feedback on https://www.pantsbuild.org/docs/troubleshooting#pants-cannot-find-a-file-in-your-project when you have a chance
h
yeah, I think it's good not to change the documentation until there's a release that has the fix in it. removing the default is very simple, but it still needs to go through the CI process and get put into a release
h
As we get closer to a release candidate, we’ll likely pick up the cadence for releases so that there’s less time to wait for bug fixes
👍 2
w
Copy code
❯ ./pants list .ci::
.ci:its_alive
😄 it worked
h
Yay! Love the name. Thanks for being patient with us in trying to figure this one out
👍 2
w
ha that’s great, one less friction with the devops team here, they already dislike me enough
🙃 1
kinda odd that no one here seems to have ever heard of 
pants
not even 
bazel
h
wonderful, glad it's working
w
yeah, srsly thanks for all the work
❤️ 1
h
Let us know if there are changes we can make to help them like you and like Pants more haha Honestly, that feedback is gold. We get lots of feedback from folks who like Pants enough to join Slack or comment on GitHub etc. But it’s challenging for us to hear the feedback of the summer intern who’s confused wth a target is. (This was me two years ago at Foursquare when first using Pants)
w
haha makes sense, yeah there’s this push enterprise-wise into monorepo.. my team is definitely an early adopter and we’re struggling with a lot of constraints and resistance from devops/cloud ops teams that support us
I’m trying to convince everyone that using pants is awesome 🤣
❤️ 2
Hopefully it’ll become the default tool and we might be able to think about using toolchain 🙂 @happy-kitchen-89482 would be happy haha
🔥 1
h
Hey Thales, beta2 also includes a fix that 
build_ignore
is now empty by default. I’m not sure if you ended up setting that, but if so, you can remove it. You likely still need 
!.ci
in 
pants_ignore
though.
w
oh ok, I did set that haha, thanks for letting me know
h
Thanks for the report and for helping us to figure out what was going on last week!
❤️ 1
3 Views
Open in Slack
PreviousNext
Powered by