https://pantsbuild.org/ logo
#general
Title
# general
w

wooden-thailand-8386

09/24/2020, 6:03 PM
Hello everyone, I have a BUILD file under a
.ci
folder and I can’t get it to show up when I run
list
or any other goal. I have this bad feeling that if it’s a hidden folder pants ignores it but just checking here first.
h

hundreds-father-404

09/24/2020, 6:10 PM
It likely is. Run
./pants help-advanced | grep -A 10 'pants_ignore'
It defaults to ignoring files/folders that start with
.
, in addition to reading from your
.gitignore
. You can add back the value by setting:
Copy code
[GLOBAL]
pants_ignore.add = ["!.ci/*"]
w

wooden-thailand-8386

09/24/2020, 6:20 PM
😞 didnt work.. I tried even redefining the
pants_ignore
which is now like this:
Copy code
default: [
          ".*/",
          "/dist/"
      ]
      current value: [
          "/dist/"
      ] (from pants.toml)
but can’t find the
.ci
folder (it’s not on my .gitignore)
I actually put my helm charts/terraform all under a
services
folder but DevOps folks want me to use their “standard” which are:
.terraform
and
.helm
together with that
.ci
folder so I’m not sure what to do haha
h

hundreds-father-404

09/24/2020, 6:30 PM
For
./pants list
, to confirm, do you have BUILD files in those directories? Pants will only output things with
./pants list
if there is a target that refers to those source files
w

wooden-thailand-8386

09/24/2020, 6:31 PM
Yes, I do have a BUILD file inside
.ci
folder and
./pants list ::
isnt showing it.. and when I try
./pants list .ci::
it throws an error
h

hundreds-father-404

09/24/2020, 6:32 PM
Try this for the sake of debugging
./pants --pants-ignore='[]' --no-pants-ignore-use-gitignore list .ci::
Also at the risk of being pedantic, double check that there are targets in the BUILD file
h

hundreds-breakfast-49010

09/24/2020, 6:34 PM
I think I can replicate this issue myself
👍 1
if I make a directory in the root of the pants repo called
zzz
with a
BUILD
file with the contents
python_library()
and nothing else
hten
./pants list ::
will display a
zzz
entry
w

wooden-thailand-8386

09/24/2020, 6:35 PM
Copy code
Engine traceback:
  in select
  in `list` goal
  in pants.engine.internals.build_files.strip_address_origins
  in Find targets from input specs
  in pants.engine.internals.build_files.addresses_with_origins_from_address_specs
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/native.py", line 69, in generator_send
    res = func.send(arg)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/build_files.py", line 214, in addresses_with_origins_from_address_specs
    addr_target_pairs_for_spec = glob_spec.matching_addresses(addr_families_for_spec)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/base/specs.py", line 130, in matching_addresses
    raise ResolveError(f"Address spec '{self}' does not match any targets.")
pants.base.exceptions.ResolveError: Address spec '.ci::' does not match any targets.
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 253, in run
    engine_result = self._run_v2()
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 164, in _run_v2
    return self._maybe_run_v2_body(goals, poll=False)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 187, in _maybe_run_v2_body
    poll_delay=(0.1 if poll else None),
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 142, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 568, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 532, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in select
  in `list` goal
  in pants.engine.internals.build_files.strip_address_origins
  in Find targets from input specs
  in pants.engine.internals.build_files.addresses_with_origins_from_address_specs
Traceback (most recent call last):
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/native.py", line 69, in generator_send
    res = func.send(arg)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/engine/internals/build_files.py", line 214, in addresses_with_origins_from_address_specs
    addr_target_pairs_for_spec = glob_spec.matching_addresses(addr_families_for_spec)
  File "/Users/lzfnyy/.cache/pants/setup/bootstrap-Darwin-x86_64/2.0.0b1_py37/lib/python3.7/site-packages/pants/base/specs.py", line 130, in matching_addresses
    raise ResolveError(f"Address spec '{self}' does not match any targets.")
pants.base.exceptions.ResolveError: Address spec '.ci::' does not match any targets.
h

hundreds-breakfast-49010

09/24/2020, 6:35 PM
if I rename it to
.zzz
it disappears, even if I add a line in
pants_ignroe.add
"!/.zzz/*",
👍 2
so maybe this is a bug with the pants_ignore functionality
w

wooden-thailand-8386

09/24/2020, 6:36 PM
oh no 🐛
🐛 1
h

hundreds-father-404

09/24/2020, 6:36 PM
It sounds like it is. Thank you for the report, Thales. Let’s definitely prioritize fixing in beta2, and likely backporting to 1.30.2rc1
👍 2
h

hundreds-breakfast-49010

09/24/2020, 6:38 PM
I'll start digging into this @hundreds-father-404
h

hundreds-father-404

09/24/2020, 6:40 PM
Thank you Greg!
h

hundreds-breakfast-49010

09/24/2020, 8:26 PM
so I think what might be going on here
is that we have
".*/"
as a default for the
--pants-ignore
option in global_options.py
and this is somehow blocking overrides defined afterwards by
pants.toml
h

hundreds-father-404

09/24/2020, 8:33 PM
I don’t think that’s it. If I override
pants_ignore
in
pants.toml
, then print what we pass to Rust, I get
Copy code
['/build-support/virtualenvs/', '/.pants.d', '/dist', '/.pids']
Given this diff:
Copy code
diff --git a/pants.toml b/pants.toml
index f014b0c47..965aad8cf 100644
--- a/pants.toml
+++ b/pants.toml
@@ -27,18 +27,18 @@ pantsd_invalidation_globs.add = [
   "src/rust/engine/**/*.toml",
 ]
 # Path patterns to ignore for filesystem operations on top of the builtin patterns.
-pants_ignore.add = [
+pants_ignore = [
   # venv directories under build-support.
   "/build-support/virtualenvs/",
-  "/build-support/*.venv/",
+#  "/build-support/*.venv/",
   # An absolute symlink to the Pants Rust toolchain sources.
-  "/build-support/bin/native/src",
+#  "/build-support/bin/native/src",
   # We shouldn't walk or watch the rust compiler artifacts because it is slow.
-  "/src/rust/engine/target",
+#  "/src/rust/engine/target",
   # We want to .gitignore Java .class files, but pants should pay attention to them.
-  "!*.class",
+#  "!*.class",
   # We also want to override the .gitignore'd pants.pex file
-  "!/pants.pex",
+#  "!/pants.pex",
 ]
 
 [source]
diff --git a/src/python/pants/init/options_initializer.py b/src/python/pants/init/options_initializer.py
index c94c47709..752eeb2c3 100644
--- a/src/python/pants/init/options_initializer.py
+++ b/src/python/pants/init/options_initializer.py
@@ -107,6 +107,7 @@ class OptionsInitializer:
         add(global_options.pants_distdir)
         add(global_options.pants_subprocessdir)
 
+        print(pants_ignore)
         return pants_ignore
 
     @staticmethod
h

hundreds-breakfast-49010

09/24/2020, 8:36 PM
@hundreds-father-404 does the problem still occur when completely removing
.*/
from pants-ignore?
I'll try it myself in a sec
ah, actually, I think that something else might be adding
.*
to the ignore struct, even if that pants option is unset
👍 1
huh we call
GitIgnoreStyleExcludes::create_with_gitignore_file
a lot
👍 1
that seems unnecessary...
uh, maybe the problem is that we should be using --build-ignore and not --pants-ignore...
@wooden-thailand-8386 I think you might want to be using
--build-ignore
option instead of the
--pants-ignore
option
I am actually not sure why the distinction exists between these two options. @hundreds-father-404?
w

wooden-thailand-8386

09/24/2020, 9:48 PM
tried it with build-ignore, same thing
h

hundreds-breakfast-49010

09/24/2020, 9:53 PM
hm, ok
h

hundreds-father-404

09/24/2020, 10:20 PM
Greg,
build_ignore
is to ignore
BUILD
files in particular. It’s useful, for example, if you are still using Pants v1 for JVM and have a v1 config file that says ignore
src/python
and a v2 config file that says ignore
src/java
I think this is a bug in
pants_ignore
h

hundreds-breakfast-49010

09/24/2020, 11:43 PM
ok, so with the current state of the codebase, if you add the directory itself to
pants_ignore.add
, and not every file in that subdir, I believe that will work
that is, add
!/.ci
, rather than
!/.ci/*
@wooden-thailand-8386
h

hundreds-father-404

09/24/2020, 11:45 PM
When debugging, also try running
./pants count-loc '.ci/*'
. This will check the behavior of
--pants-ignore
Greg and I realized
--build-ignore
is relevant, indeed. Even if
--pants-ignore
is correct, if
--build-ignore
is saying to ignore BUILD files, it will look to Pants like there are no targets in that directory. The above command will help to better isolate if either one or both of the options are the issue
h

hundreds-breakfast-49010

09/24/2020, 11:47 PM
it's definitely possible to get into a state where
./pants count-loc '.ci/*'
will work, but
.ci
won't show up in
./pants list ::
if
pants_ignore.add
has
!/.ci
but
build_ignore
isn't touched
it's also the case that if you have
build_ignore.add = ["!/.ci"]
, the default
build_ignore
.*/
is getting applied after the custom
.add
, so the explicit allow of
.ci
doesn't affect the final matching behavior
that is a legitimate bug I believe
h

hundreds-father-404

09/24/2020, 11:50 PM
it’s also the case that if you have build_ignore.add = [“!/.ci”], the default build_ignore .*/ is getting applied after the custom .add
Does the ordering impact things? That’s generic options code, that
my_opt.add
appends to the end of the prior values
h

hundreds-breakfast-49010

09/24/2020, 11:52 PM
as far as I'm aware, ordering matters
"Each line in a
gitignore
file specifies a pattern. When deciding whether to ignore a path, Git normally checks
gitignore
patterns from multiple sources, with the following order of precedence, from highest to lowest (within one level of precedence, the last matching pattern decides the outcome):"
👍 1
that file also has a section on pattern format, which we probably want to explicitly reference in our own docs on these options
@wooden-thailand-8386 so, specifically, right now it is necessary to 1) add
!/.ci/
to
pants_ignore.add
and 2) to remove the default
".*/"
glob from the
build-ignore
option by adding
build_ignore.remove = [".*/"]
to your cargo.toml. that should make it possible to see
.ci
show up in pants goals
🤩 1
and I think based on this conversation that we want to remove that problematic
.*/
as a default glob in the
--build-ignore
option, so in the next release that has that change, 2) will no longer be necessary
👍 1
w

wooden-thailand-8386

09/25/2020, 12:20 AM
Awesome! I just turned on my work laptop to add those changes already haha, thanks for all the digging around to find that fix.
btw, you guys are on the East Coast right?
West*
h

hundreds-breakfast-49010

09/25/2020, 12:21 AM
the bulk of us are in the bay area, a few of us are elsewhere. I'm personally in the bay area
w

wooden-thailand-8386

09/25/2020, 12:22 AM
haha ok, that makes sense 🙂 I’m on East Coast so if I dont reply it’s probably bc it’s already late for me
h

hundreds-breakfast-49010

09/25/2020, 12:23 AM
no worries 🙂
w

wooden-thailand-8386

09/25/2020, 12:23 AM
Should we wait the fix on v2 to add that information into the documentation?
h

hundreds-father-404

09/25/2020, 12:24 AM
I think probably. But would love any feedback on https://www.pantsbuild.org/docs/troubleshooting#pants-cannot-find-a-file-in-your-project when you have a chance
h

hundreds-breakfast-49010

09/25/2020, 12:24 AM
yeah, I think it's good not to change the documentation until there's a release that has the fix in it. removing the default is very simple, but it still needs to go through the CI process and get put into a release
h

hundreds-father-404

09/25/2020, 12:24 AM
As we get closer to a release candidate, we’ll likely pick up the cadence for releases so that there’s less time to wait for bug fixes
👍 2
w

wooden-thailand-8386

09/25/2020, 12:26 AM
Copy code
❯ ./pants list .ci::
.ci:its_alive
😄 it worked
h

hundreds-father-404

09/25/2020, 12:27 AM
Yay! Love the name. Thanks for being patient with us in trying to figure this one out
👍 2
w

wooden-thailand-8386

09/25/2020, 12:27 AM
ha that’s great, one less friction with the devops team here, they already dislike me enough
🙃 1
kinda odd that no one here seems to have ever heard of
pants
not even
bazel
h

hundreds-breakfast-49010

09/25/2020, 12:28 AM
wonderful, glad it's working
w

wooden-thailand-8386

09/25/2020, 12:28 AM
yeah, srsly thanks for all the work
❤️ 1
h

hundreds-father-404

09/25/2020, 12:29 AM
Let us know if there are changes we can make to help them like you and like Pants more haha Honestly, that feedback is gold. We get lots of feedback from folks who like Pants enough to join Slack or comment on GitHub etc. But it’s challenging for us to hear the feedback of the summer intern who’s confused wth a target is. (This was me two years ago at Foursquare when first using Pants)
w

wooden-thailand-8386

09/25/2020, 12:30 AM
haha makes sense, yeah there’s this push enterprise-wise into monorepo.. my team is definitely an early adopter and we’re struggling with a lot of constraints and resistance from devops/cloud ops teams that support us
I’m trying to convince everyone that using pants is awesome 🤣
❤️ 2
Hopefully it’ll become the default tool and we might be able to think about using toolchain 🙂 @happy-kitchen-89482 would be happy haha
🔥 1
h

hundreds-father-404

09/28/2020, 7:02 PM
Hey Thales, beta2 also includes a fix that
build_ignore
is now empty by default. I’m not sure if you ended up setting that, but if so, you can remove it. You likely still need
!.ci
in
pants_ignore
though.
w

wooden-thailand-8386

09/28/2020, 7:02 PM
oh ok, I did set that haha, thanks for letting me know
h

hundreds-father-404

09/28/2020, 7:03 PM
Thanks for the report and for helping us to figure out what was going on last week!
❤️ 1
3 Views