https://pantsbuild.org/ logo
Join Slack
Powered by
Not sure if a known issue but using a spec-file wi...
# general
w
Not sure if a known issue but using a spec-file with the below content in 1.26.0 results in a 
Exception message: Both address specs and filesystem specs given. Please use only one type of spec.
filespec.txt
happy to file an issue
the rest of the. erro output gives:
Copy code
Exception message: Both address specs and filesystem specs given. Please use only one type of spec.

Address specs: src/subscription_new/BUILD:BUILD
Filesystem specs: 3rdparty/python/requirements.txt, Pipfile, Pipfile.lock, deploy.sh, pants-docker.ini, src/common/dataframe/test/validate_test.py, src/common/dataframe/validate.py, src/conviva_v2/BUILD, src/conviva_v2/config.yml, src/conviva_v2/conviva_daily_backfill_dag.py, src/conviva_v2/conviva_daily_ingestion.py, src/conviva_v2/helpers.py, src/gdpr_requests/dag.py, src/sfmc_import/BUILD, src/sfmc_import/db_migrations/7a4efabb1dda_20200513_add_sendlog_fields.py, src/sfmc_import/schemas/sendlog.py, src/subscription/check_data.py, src/subscription/dynamo_point_in_time.py, src/subscription/sublime_transform.py, src/subscription/subscription_ingest.py, src/subscription/subscription_state_backfill_dag.py, src/subscription/tests/config_test.py, src/subscription/tests/dynamo_test.py, src/subscription/tests/subscription_ingest_test.py, src/subscription/tests/test_check_data.py, src/subscription/tests/test_sublime_tranform.py, src/subscription/tests/test_utils_funcs.py, src/subscription/tests/utils_for_tests.py, src/subscription/utils.py, src/subscription_new/README.md, src/subscription_new/__init__.py, src/subscription_new/config/__init__.py, src/subscription_new/config/config.py, src/subscription_new/config/config.yml, src/subscription_new/dynamo.py, src/subscription_new/dynamo_to_json.py, src/subscription_new/dynamo_to_parquet.py, src/subscription_new/fixtures/reprocess_hive_data.json, src/subscription_new/fixtures/reprocess_hive_data_expected.json, src/subscription_new/fixtures/unittest_state_engine.json, src/subscription_new/job_config_prod.yml, src/subscription_new/job_config_test.yml, src/subscription_new/schema/__init__.py, src/subscription_new/schema/dynamo_subscription_schema.py, src/subscription_new/schema/state_schema.py, src/subscription_new/scripts/migrate_dynamo_est.py, src/subscription_new/scripts/migrate_dynamo_v2.py, src/subscription_new/scripts/seed_start_date.py, src/subscription_new/scripts/subscription_load_table.sql, src/subscription_new/squirlconf/common.yaml, src/subscription_new/squirlconf/int.yaml, src/subscription_new/squirlconf/prod.yaml, src/subscription_new/squirlconf/qa.yaml, src/subscription_new/sublime_transform.py, src/subscription_new/subscription_backfill_dag.py, src/subscription_new/subscription_dag.py, src/subscription_new/subscription_data_check.py, src/subscription_new/subscription_transform.py, src/subscription_new/tests/__init__.py, src/subscription_new/tests/test_check_data.py, src/subscription_new/tests/test_config.py, src/subscription_new/tests/test_dynamo.py, src/subscription_new/tests/test_sublime_tranform.py, src/subscription_new/tests/test_subscription_transform.py, src/subscription_new/tests/test_utils.py, src/subscription_new/tests/utils.py, src/subscription_new/utils.py, src/watch/data_reconciliation/BUILD
not sure why it thinks theres a 
build:build
address
h
An issue would be great! Thank you. I’ve seen this once or twice. I suspected you had a directory like 
src/subscriptions
that was getting interpreted as an address spec, because this is shorthand for 
src/subscriptions:subscriptions
. But it doesn’t look like that’s what is happening here.
w
is it possible its interpreting 
src/sfmc_import/BUILD
as 
src/sfmc_import/BUILD:BUILD
?
b/c that would. be an issue
h
let me load it up with 
./pants repl
to test
w
and it looks closer to what the error message is
h
Indeed:
Copy code
▶ ./v2 repl --shell=ipython src/python/pants/base/cmd_line_spec_parser.py

In [1]: from pants.base.cmd_line_spec_parser import CmdLineSpecParser

In [2]: from pants.base.build_environment import get_buildroot

In [3]: spec_parser = CmdLineSpecParser(get_buildroot())

In [4]: spec_parser.parse_spec("src/sfmc_import/BUILD")
Out[4]: SingleAddress(directory='src/sfmc_import/BUILD', name='BUILD')
w
funny this hasn't cropped up before, seems like it would happen quickly
h
So, the heuristic 
if PurePath(spec).suffix
won’t apply, but I would expect the next heuristic of 
if Path(self._root_dir, spec_path).is_file():
to apply. Could you please try running this from your build root:
Copy code
$ python  # open a repl
from pathlib import Path
(Path.cwd() / "src/sfmc_import/BUILD").is_file()
w
True
🤔 1
ah
i think its b/c, during local testing i may. have been behind master
and the script does a 
git diff <version>
that probably detected changed files that didn't. exist on my filesystem
if i replalcethe path with the file it. was failing for
i get False
h
Ohh, yes, that is it
w
i mean still seems fragile, i.e. we delete targets
we cant use 
git diff
h
Hm, I’ll update the ticket - that’s an interesting problem of how to disambiguate between files vs. directories when you can’t call 
Path.exists()
on it because of deletion.
Possibly, we special case 
BUILD
, but that isn’t great because: 1) I believe you could have a directory with that name 2) If you were to delete 
Pipfile
, for example, it wouldn’t work properly.
A more general solution is to remove the shorthand of 
folder/
expanding to 
folder:folder
. That’s been proposed but not explored fully because it has huge implications - for example, that shorthand is really useful in BUILD files for the 
dependencies
field.
w
sure makes sense
not sure how difficult it would be to allow shorthannd in BUILD file definitions but not via the CLI, but that sounds like a reasonable possibility
h
Definitely feasible implementation wise - the hesitation is more conceptual, that we try to avoid a value meaning one thing in one place, but another thing somewhere else. Bad UX to have to remember special cases
w
sure of course
h
hm I don’t think some kind of special syntax to disambiguate, like ending the arg with a 
!
, would help. You’re giving Pants a machine generated value, and it would be very clunky to get Git to prepend a special symbol
w
right
why is mixing targets and file specs not allowed?
h
Ambiguity in Pants’ rule graph. At a high level, a 
@goal_rule
like 
test
will ask for 
Targets
. There are two paths for the rule graph to fulfill that: 1) FilesystemSpecs -> OwnersOf -> Targets 2) AddressSpecs -> Targets If both were provided at the same time, then we would have two ways to get the 
Targets
needed by the goal, and Pants wouldn’t know what to do. (We could possibly rework things to work around, this, though.) -- But even if we allowed address and file specs at the same time, you would still hit a similar issue. Pants would still interpret 
src/sfmc_import/BUILD
as 
src/sfmc_import/BUILD:BUILD
, and then it would complain that it can’t find any folder called 
src/sfmc_import/BUILD
with the target 
:BUILD
Hm, maybe the answer is outside of Pants. See https://stackoverflow.com/questions/6894322/how-to-make-git-diff-and-git-log-ignore-new-and-deleted-files for a flag that will allow you to filter out all deleted files
Alternatively, you can use 
./pants --changed-since
. Pants knows what to do with deleted files if you use that approach.
w
i'll spend some. time with this thanks
❤️ 1
4 Views
Open in Slack
PreviousNext
Powered by