https://pantsbuild.org/ logo
Join Slack
Powered by
Different question: I believe I am struggling with...
# general
f
Different question: I believe I am struggling with pants crashing when trying to find standard packages ie 
attrs
in our internal 
artifactory
. My best guess is that our artifactory is returning an empty list or an error that is then causing pants to fail. Is there a better way I should be handling this? I'm currently adding things with 
[python-repos].indexes.add
h
Hmm, can you post the output and stack trace?
(redacted if necessary)
f
Copy code
./pants test ::                                                                                                                                               eopt-core/git/pants !
14:47:51.22 [INFO] Initializing scheduler...
14:47:51.91 [INFO] Scheduler initialized.
14:48:12.78 [INFO] Completed: Building pytest.pex from pytest_default_lockfile.txt
<snipped several hopefully unrelated warnings>
14:48:13.99 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 236, in _run_inner
    return self._perform_run(goals)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 175, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 192, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 538, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.SCHkFQ/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 506, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building pytest.pex from pytest_default_lockfile.txt' failed with exit code 1.
stdout:

stderr:
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    crit = self.state.criteria[name]
KeyError: 'attrs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/commands/download.py", line 130, in run
    requirement_set = resolver.resolve(
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 121, in resolve
    self._result = resolver.resolve(
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    name, crit = self._merge_into_criterion(r, parent=None)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    crit = Criterion.from_requirement(self._p, requirement, parent)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement
    if not cands:
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 124, in __bool__
    return bool(self._sequence)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in __bool__
    return any(self)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 220, in iter_index_candidates
    result = self._finder.find_best_candidate(
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 882, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 825, in find_all_candidates
    package_links = self.process_project_url(
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/package_finder.py", line 789, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 618, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 430, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/index/collector.py", line 127, in _get_html_response
    resp = session.get(
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/network/session.py", line 428, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 662, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
    username = ask_input("User for {}: ".format(netloc))
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/02b35deb/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 262, in ask_input
    return input(message)
EOFError: EOF when reading a line
pid 67423 -> /Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/1d2a3f1b3961efbfba4c4373717c72d09d116d95/0c41b5bd1d055c543f99ad078c876aa51077c621/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/tdimiduk/.cache/pants/named_caches/pex_root --log /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionAL8XB4/.tmp/tmpct79fcd6/pip.log download --dest /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionAL8XB4/.tmp/tmpi7qtb_2w/Users.tdimiduk..pyenv.versions.3.8.12.bin.python3.8 --no-deps --requirement pytest_default_lockfile.txt --index-url <https://pypi.org/simple/> --extra-index-url <https://artifactory>.<snipped>/simple --extra-index-url <https://artifactory>.<snipped>/simple --retries 5 --timeout 15 exited with 2 and STDERR:
None
I can mess around and eventually get past this by running with whatever is failing commented out until I get some things cached, then adding them back in, but I got back in this hole when I blew away my caches trying to see if that was related to a different problem
h
Looks like this is crashing inside Pip...
Which is weird
Which Pants version are you using?
And how sure are you that the URLs you're adding belong in 
[python-repos].indexes
as opposed to 
[python-repos].repos
?
I.e., is it an index a la https://pypi.org/simple/, or is it a find-links situation?
f
yes they are 
artifactory.<....>/simple
so I'm pretty sure they belong in indexes
and when I run pip outside of pants I pass them with 
--extra-index-url
h
Right, makes sense then
Weird that it's failing inside Pip
Presumably running Pip directly against that artifactory works
f
yes
that was one of the early things I tried. I've been trying to reproduce with the pex commands, but getting other issues, I'll probably have to see if I can distill this into a simpler case to debug this
👍 1
h
That would be super-helpful
e
Looks like it's this set of things: + https://github.com/pypa/pip/issues/9557 + https://github.com/pypa/pip/issues/3931 + https://github.com/pypa/pip/pull/10033 Pex uses a vendored copy of Pip 20.3.4 which does not have that PR fix. I'll see if that patch applies ~cleanly.
This cherry-picked cleanly: https://github.com/pantsbuild/pip/pull/10 I'll try to get a Pex release out with this today.
@freezing-vegetable-92896 to confirm: Do you have multiple indexes configured with auth?
f
I've got multiple indexes configured, but I believe only one of them with auth. Currently it is public pypi, one authenticated artifactory, and one artifactory that doesn't need auth
e
Ok. I'll stare at the Pip PR a bit more to see if your setup sounds related still. Thanks.
f
The packages I'm having fail to install are from public pypi, not our private one. But if I remove the extra index url for both of our private repos then I can install from public pypi just fine
e
So, looking at your backtrace more closely and the vendored Pip its running against, the KeyError looks like its expected / handled: https://github.com/pantsbuild/pex/blob/099fd9955c3a0696526d148027f070504dfb0da7/pex/vendor/_vendored/pip/pip/_vendor/resolvelib/resolvers.py#L1[…]73 Its in the handling that an HTTP request is made and it looks like there is an auth challenge + no auth configured -> So Pip prompts for a password which fails since this is non-interactive. That would seem to indicate some auth is not in the 
--index
configuration but coming from your environment or ~/.netrc or pip.conf and its not getting picked up. Does that ring true?
f
yes, I've been attempting to give auth from 
~/.netrc
, but that has been finickier than than I'de prefer
I struggled with figuring out what the recommended pattern for using auth with pants is. With pip we were using 
https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory/...
but that didn't seem to work with pants (unless I was running into a different problem)
e
That is the reccomendation. Pants though strives for hermetcity and does not allow arbitrary env vars to pass through to processes it runs - its all tightly controlled. So you need to let pants know to pass those env vars through. Just a sec for config...
The closest matching high level doc is here, but in the context of a custom proxy: https://www.pantsbuild.org/docs/python-third-party-dependencies#advanced-usage
So you want to add this to pants.toml:
Copy code
[subprocess-environment]
env_vars.add = [
  "ARTIFACTORY_USERNAME",
  "ARTIFACTORY_PASSWORD",
]
You should be good to go after that.
f
thanks. I'll try that out
I get a similar error
Copy code
./pants test ::                                                                                                                                                eopt-core/git/pants
11:26:51.13 [INFO] Initializing scheduler...
11:26:51.51 [INFO] Scheduler initialized.
11:27:10.80 [INFO] Completed: Resolving pytest.pex from pytest_default_lockfile.txt
11:27:10.85 [INFO] Completed: Resolving constraints.txt
11:27:10.88 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 236, in _run_inner
    return self._perform_run(goals)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 175, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 192, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 538, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 506, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Resolving pytest.pex from pytest_default_lockfile.txt' failed with exit code 1.
stdout:

stderr:
User for <http://artifactory.teslamotors.com|artifactory.teslamotors.com>: ERROR: Exception:
Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    crit = self.state.criteria[name]
KeyError: 'attrs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/commands/download.py", line 131, in run
    reqs, check_supported_wheels=True
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    name, crit = self._merge_into_criterion(r, parent=None)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    crit = Criterion.from_requirement(self._p, requirement, parent)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement
    if not cands:
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 124, in __bool__
    return bool(self._sequence)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in __bool__
    return any(self)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 223, in iter_index_candidates
    hashes=hashes,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 882, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 826, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 789, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 618, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 430, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 144, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/session.py", line 428, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 662, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
    username = ask_input("User for {}: ".format(netloc))
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 262, in ask_input
    return input(message)
EOFError: EOF when reading a line
pid 87579 -> /Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/784b6347ec6f56ab06909a22fe2c6920eded90f0/e1598f4d48718c5c37e988d295dc18143bab9570/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/tdimiduk/.cache/pants/named_caches/pex_root --log /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionycbZAg/.tmp/tmp95otzw8r/pip.log download --dest /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionycbZAg/.tmp/tmpeaf1qk2p/Users.tdimiduk..pyenv.versions.3.6.12.bin.python3.6 --no-deps --requirement pytest_default_lockfile.txt --index-url <https://pypi.org/simple/> --extra-index-url https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory<snip>/simple --extra-index-url <https://artifactory><snip>/simple --retries 5 --timeout 15 exited with 2 and STDERR:
None
e
Hrm. So you're sure that 2nd artifactory url needs no auth?
And, to confirm, the pants.toml snippet I gave is to be used iff the env vars are defined in your environment - say by direnv or ~/.profile or what have you. If not, then then the values need to go in that config, like "ARTIFACTORY_USERNAME=you_probably_dont_want_to_expose_secrets_this_way_but_just_checking".
f
yeah, I've got the secrets defined in my env. If I don't then I get errors about accessing a private package
But now when I get the secrets lined up correctly, I get an error looking for a standard python package that should be in the public pypi
I can get a similar error if I remove the second artifactory entirely ./pants test :: eopt-core/git/pants ! 114741.47 [INFO] Initializing scheduler... 114741.85 [INFO] Scheduler initialized. 114804.51 [INFO] Completed: Resolving pytest.pex from pytest_default_lockfile.txt 114809.40 [INFO] Completed: Resolving constraints.txt 114809.46 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError) File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 236, in _run_inner return self._perform_run(goals) File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 175, in _perform_run return self._perform_run_body(goals, poll=False) File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 192, in _perform_run_body return self.graph_session.run_goal_rules( File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules exit_code = self.scheduler_session.run_goal_rule( File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 538, in run_goal_rule self._raise_on_error([t for _, t in throws]) File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 506, in _raise_on_error raise ExecutionError( Exception message: 1 Exception encountered: ProcessExecutionFailure: Process 'Resolving constraints.txt' failed with exit code 1. stdout: stderr: User for artifactory.teslamotors.com: ERROR: Exception: Traceback (most recent call last): File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion crit = self.state.criteria[name] KeyError: 'werkzeug' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 223, in _main status = self.run(options, args) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper return func(self, options, args) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/commands/download.py", line 131, in run reqs, check_supported_wheels=True File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve requirements, max_rounds=try_to_avoid_resolution_too_deep, File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve name, crit = self._merge_into_criterion(r, parent=None) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion crit = Criterion.from_requirement(self._p, requirement, parent) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement if not cands: File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 124, in bool return bool(self._sequence) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in bool return any(self) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 223, in iter_index_candidates hashes=hashes, File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 882, in find_best_candidate candidates = self.find_all_candidates(project_name) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 826, in find_all_candidates project_url, link_evaluator=link_evaluator, File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 789, in process_project_url html_page = self._link_collector.fetch_page(project_url) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 618, in fetch_page return _get_html_page(location, session=self.session) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 430, in _get_html_page resp = _get_html_response(url, session=session) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 144, in _get_html_response "Cache-Control": "max-age=0", File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 555, in get return self.request('GET', url, **kwargs) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/session.py", line 428, in request return super(PipSession, self).request(method, url, *args, **kwargs) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 662, in send r = dispatch_hook('response', hooks, r, **kwargs) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook _hook_data = hook(hook_data, **kwargs) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 256, in handle_401 username, password, save = self._prompt_for_password(parsed.netloc) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password username = ask_input("User for {}: ".format(netloc)) File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 262, in ask_input return input(message) EOFError: EOF when reading a line pid 88594 -> /Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/784b6347ec6f56ab06909a22fe2c6920eded90f0/e1598f4d48718c5c37e988d295dc18143bab9570/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/tdimiduk/.cache/pants/named_caches/pex_root --log /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionOzRlth/.tmp/tmp375ivgvd/pip.log download --dest /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionOzRlth/.tmp/tmpkzsr38o1/Users.tdimiduk..pyenv.versions.3.6.12.bin.python3.6 --constraint constraints.txt <snip> Werkzeug==2.0.2 <snip> --index-url https://pypi.org/simple/ --extra-index-url https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory<snip>/simple --retries 5 --timeout 15 exited with 2 and STDERR: None
e
Yeah "User for artifactory.teslamotors.com ..." means you're getting prompted so auth isn't lined up still somehow.
f
it seems to be resolving things in a different order and failing on 
werkzeug
instead, but again it should be able to find that in public pypi
e
Ok, to help me understand - I'm a little lost: You've found a way to get auth working and then hit werkzeug issue?
f
Is there an easy way to confirm the environment variables are getting in correctly?
I thought I had auth working, since it is failing on a a package that isn't private
but as you point out, maybe I don't actually have it working correctly
Though I also don't understand why it should have problems installing werkzeug or attrs based on failures to auth with a private repo
e
To confirm you can run with 
--no-process-execution-local-cleanup
which will both print out process execution 
/tmp/...
dir names and leave them there for you to examine. The last such one should be the right one. Inside that dir will be a 
__run.sh
script. You can add an echo in that script and run it to check.
If we can debug the auth 1st, that would be best even if its a bit frustrating. I think mixing debugging of both things will just muddy.
I have to step away for a good chunk of the afternoon, so you may want to draw others' attention if you come up with the debug info re env vars; otherwise I'll check back in this evening.
f
The issue does appear to resolve when I get the credentials correct. Weird, but I guess won't be a problem once we get the credentials in good shape
now I'm struggling with it looking like shell variable expansion isn't working in the pex commands. If I hardcode my username and password in the 
pants.toml
file things work correctly
And the environment variables do get into my 
__run.py
file with the correct values, but they seem to not be properly expanded for the command call
e
Can you share your 
__run.sh
with the sensitive env var values redacted?
f
Things are coming through as
Copy code
--no-pypi $'--index=<https://pypi.org/simple/>' $'--index=<https://artifactory><snip>/simple'
If I remove the single quotes 
'
around the url, it no longer prompts me for a password. My bash isn't that strong, but I think single quotes 
'
prevent variable expansion (I'm not sure what the extra 
$
does) so I'm suspicious of that
Copy code
/Users/tdimiduk/.pyenv/versions/3.6.12/bin/python3.6 ./pex --python-path $'/Users/tdimiduk/.pyenv/versions/3.6.12/bin:/Users/tdimiduk/.pyenv/versions/3.6.8/bin:/Users/tdimiduk/.pyenv/versions/3.8.12/bin:/Users/tdimiduk/.pyenv/versions/3.8.3/bin:/Users/tdimiduk/.poetry/bin:/usr/local/Cellar/python/3.8.8_1/bin:/Users/tdimiduk/bin:/Users/tdimiduk/.pyenv/bin:~/bin:/Users/tdimiduk/dev/dataworks/bazel-bin/telemetry-one-off/src/main/scala/com/tesla/data/oneoff/telemetry:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin' --tmpdir .tmp --output-file repository.pex --include-tools --no-pypi $'--index=<https://pypi.org/simple/>' --index=https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory.teslamotors.com/artifactory/api/pypi/stst-pypi-local/simple '--index=https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory.teslamotors.com/artifactory/api/pypi/fa-dataworks-release-local/simple' --resolver-version pip-2020-resolver --python /Users/tdimiduk/.pyenv/versions/3.6.12/bin/python3.6 --no-emit-warnings --jobs 4 --manylinux manylinux2014 $'--sources-directory=source_files' --constraints constraints.txt $'CVXcanon==0.1.2' $'Cython==0.29.2' $'Flask==1.0.2' $'Jinja2==3.0.2' $'MarkupSafe==2.0.1' $'PyHive==0.6.1' $'PyJWT==1.7.1' $'PyMySQL==0.9.3' $'PyYAML==6.0' $'Pygments==2.10.0' $'SQLAlchemy==1.3.20' $'SecretStorage==3.3.1' $'Werkzeug==2.0.2' $'amqp==5.0.6' $'appdirs==1.4.4' $'attrs==19.3.0' $'autobidder-proto==0.20.0+a9e5a9d.520' $'aws-sam-translator==1.39.0' $'aws-xray-sdk==2.8.0' $'backports.zoneinfo==0.2.1' $'beautifulsoup4==4.5.0' $'bill-estimator==0.3.0+0d87a66.16' $'billiard==3.6.4.0' $'black==20.8b1' $'bleach==4.1.0' $'boto3-stubs==1.12.39.0' $'boto3==1.12.39' $'boto==2.49.0' $'botocore==1.15.49' $'cached-property==1.5.2' $'cachetools==4.2.4' $'cattrs==1.0.0' $'celery==5.1.2' $'certifi==2021.10.8' $'cffi==1.15.0' $'cfn-lint==0.54.4' $'chardet==3.0.4' $'charset-normalizer==2.0.7' $'click-didyoumean==0.3.0' $'click-plugins==1.1.1' $'click-repl==0.2.0' $'click==7.1.2' $'colorama==0.4.4' $'common-proto==0.2.0+30df4bd.474' $'configparser==5.0.2' $'control-api-proto==0.3.0+30df4bd.470' $'coverage==6.1.1' $'cryptography==3.1' $'cvxpy==1.0.11' $'cycler==0.11.0' $'dataclasses==0.8' $'datalake-presto-client==2.1.10.post4609' $'datatank-client==2.1.10.post4609' $'dataworks-bouncer-auth==2.1.10.post4609' $'dataworks-common==2.1.10.post4609' $'decorator==4.4.2' $'deepdiff==5.5.0' $'dill==0.3.4' $'docker==5.0.3' $'docutils==0.15.2' $'ecdsa==0.17.0' $'ecos==2.0.7.post1' $'energy-device-proto==1.2.0+30df4bd.471' $'et-xmlfile==1.1.0' $'execnet==1.9.0' $'fastcache==1.1.0' $'fsspec==2021.10.1' $'future==0.18.2' $'gateway-asset-manifest-proto==0.2.0+30df4bd.472' $'google-auth==2.3.2' $'googleapis-proto==1.7.0+5a0a612.486' $'grpcio==1.27.2' $'html5lib==1.1' $'hypothesis==5.8.0' $'idna==2.7' $'importlib-metadata==4.8.1' $'importlib-resources==3.3.1' $'iniconfig==1.1.1' $'isodate==0.5.4' $'itsdangerous==2.0.1' $'jeepney==0.7.1' $'jmespath==0.10.0' $'joblib==1.1.0' $'jsondiff==1.1.2' $'jsonpatch==1.32' $'jsonpointer==2.1' $'jsonschema==3.2.0' $'junit-xml==1.9' $'jwt==1.2' $'keyring==23.2.1' $'kiwisolver==1.3.1' $'kombu==5.1.0' $'kubernetes==11.0.0' $'lxml==3.6.1' $'marshmallow==3.0.0b16' $'matplotlib==3.1.1' $'mock==2.0.0' $'moto==1.3.14' $'multiprocess==0.70.12.2' $'mypy-boto3-cloudformation==1.12.39.0' $'mypy-boto3-dynamodb==1.12.39.0' $'mypy-boto3-ec2==1.12.39.0' $'mypy-boto3-lambda==1.12.39.0' $'mypy-boto3-rds==1.12.39.0' $'mypy-boto3-s3==1.12.39.0' $'mypy-boto3-secretsmanager==1.12.39.0' $'mypy-boto3-sns==1.12.39.0' $'mypy-boto3-sqs==1.12.39.0' $'mypy-boto3==1.12.39.0' $'mypy-extensions==0.4.3' $'mypy==0.790' $'networkx==2.5.1' $'numpy==1.19.1' $'oauthlib==3.1.1' $'openpyxl==3.0.7' $'ordered-set==4.0.2' $'osqp==0.6.2.post0' $'packaging==21.2' $'pandas==1.1.5' $'pathspec==0.9.0' $'pbr==5.6.0' $'pkginfo==1.7.1' $'pluggy==0.13.1' $'prompt-toolkit==3.0.21' $'protobuf==3.8.0' $'psycopg2-binary==2.9.1' $'py==1.10.0' $'pyarrow==4.0.1' $'pyasn1-modules==0.2.8' $'pyasn1==0.4.8' $'pycparser==2.20' $'pyhocon==0.3.57' $'pyiso==0.3.19' $'pyparsing==2.4.7' $'pyrsistent==0.18.0' $'pytest-cov==2.10.1' $'pytest-forked==1.3.0' $'pytest-xdist==2.1.0' $'pytest==6.1.2' $'python-dateutil==2.8.2' $'python-jose==3.3.0' $'python-json-logger==2.0.1' $'pytz-deprecation-shim==0.1.0.post0' $'pytz==2018.5' $'qdldl==0.1.5.post0' $'readme-renderer==30.0' $'regex==2021.11.1' $'requests-oauthlib==1.3.0' $'requests-toolbelt==0.9.1' $'requests==2.20.1' $'responses==0.10.15' $'rfc3986==1.5.0' $'rsa==4.7.2' $'ruamel.yaml.clib==0.2.6' $'ruamel.yaml==0.16.13' $'s3fs==0.4.2' $'s3transfer==0.3.7' $'scikit-learn==0.23.0' $'scipy==1.5.4' $'scs==2.1.4' $'seaborn==0.10.0' $'sentry-sdk==1.4.3' $'shared-protocol-proto==1.1.0+30df4bd.473' $'six==1.16.0' $'sortedcontainers==2.4.0' $'sqlalchemy-stubs==0.3' $'sshpubkeys==3.3.1' $'tabulate==0.8.9' $'threadpoolctl==3.0.0' $'toml==0.10.2' $'tqdm==4.60.0' $'twine==3.4.2' $'typed-ast==1.4.3' $'typing-extensions==3.7.4.1' $'typing==3.7.4.3' $'tzdata==2021.5' $'tzlocal==4.1' $'urllib3==1.24.3' $'vine==5.0.0' $'wcwidth==0.2.5' $'webencodings==0.5.1' $'websocket-client==1.2.1' $'wrapt==1.13.3' $'xlrd==1.2.0' $'xmltodict==0.12.0' $'xpress==8.10.2' $'zipp==3.6.0'
e
Don't get too distracted by the detailed contents of 
__run.sh
- its just for debugging. The actual process invokation is ~just like that but directly from Rust code.
f
The environment variables are just above that, and I've confirmed they are the correct values
Ok, I definitely was getting distracted by them
but the error from pants is still pretty similar:
Copy code
17:49:03.87 [INFO] preserving local process execution dir `"/private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionFRby3J"` for "Resolving constraints.txt"
17:49:10.46 [INFO] Completed: Resolving pytest.pex from pytest_default_lockfile.txt
17:49:10.58 [INFO] Completed: Resolving constraints.txt
17:49:10.62 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
17:49:10.62 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 236, in _run_inner
    return self._perform_run(goals)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 175, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 192, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 538, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/tdimiduk/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.tm5F4q/install/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 506, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Resolving pytest.pex from pytest_default_lockfile.txt' failed with exit code 1.
stdout:

stderr:
User for <http://artifactory.teslamotors.com|artifactory.teslamotors.com>: ERROR: Exception:
Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 171, in _merge_into_criterion
    crit = self.state.criteria[name]
KeyError: 'attrs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/commands/download.py", line 131, in run
    reqs, check_supported_wheels=True
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 122, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 453, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 318, in resolve
    name, crit = self._merge_into_criterion(r, parent=None)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _merge_into_criterion
    crit = Criterion.from_requirement(self._p, requirement, parent)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 82, in from_requirement
    if not cands:
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/resolvelib/structs.py", line 124, in __bool__
    return bool(self._sequence)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 99, in __bool__
    return any(self)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 223, in iter_index_candidates
    hashes=hashes,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 882, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 826, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/package_finder.py", line 789, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 618, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 430, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/index/collector.py", line 144, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/session.py", line 428, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/sessions.py", line 662, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
    username = ask_input("User for {}: ".format(netloc))
  File "/Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/short/ff5b2036/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 262, in ask_input
    return input(message)
EOFError: EOF when reading a line
pid 10683 -> /Users/tdimiduk/.cache/pants/named_caches/pex_root/venvs/784b6347ec6f56ab06909a22fe2c6920eded90f0/e1598f4d48718c5c37e988d295dc18143bab9570/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/tdimiduk/.cache/pants/named_caches/pex_root --log /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionHsUbgR/.tmp/tmp_stsdit8/pip.log download --dest /private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionHsUbgR/.tmp/tmp7qi3lnn_/Users.tdimiduk..pyenv.versions.3.6.12.bin.python3.6 --no-deps --requirement pytest_default_lockfile.txt --index-url <https://pypi.org/simple/> --extra-index-url https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory.teslamotors.com/artifactory/api/pypi/stst-pypi-local/simple --extra-index-url https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@artifactory.teslamotors.com/artifactory/api/pypi/fa-dataworks-release-local/simple --retries 5 --timeout 15 exited with 2 and STDERR:
None
And when I look in 
/private/var/folders/t2/cfkkkdyx5ln1lr1w3_5_xvt189jt__/T/process-executionFRby3J
I see
Copy code
#!/bin/bash
# This command line should execute the same process as pants did internally.
export ARTIFACTORY_PASSWORD=<correct value> ARTIFACTORY_USERNAME=tdimiduk ...
when I run that ```..._/T/process-executionFRby3J❯ ./__run.sh User for artifactory.teslamotors.com:```
e
Ah - damn. My bad. Right. Pex interpolates if passed in requirements.txt. Pants, though, doesn't interpolate in pants.toml. Very sorry to space on this. You'd need your environment to 
export PANTS_PYTHON_REPOS_INDEXES="+[\"https://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}:...\",\"<next one>\"]"
. In other words, an alternate way to set any Pants config is via 
PANTS_<option scope>_<option name>=
and lists have the special add syntax 
+[...]
Does that make sense? Sorry for the goose chase!
f
Oh, can I just put the lines back in my requirements.txt instead of pants.toml? that would be much simpler
e
Unfortunately - no, Pants does not pass your requirements.txt as-is to Pex, it parses out the requirements and ignores all else 😕
f
I see, so that would work with pex, but not with pants. I thought I had tried it a while ago
e
So, the current state of affairs is both awkward and confusing. Sorry it took a while to get there - I should know better. But, in summary, you need to configure 
[python-repos] indexes
, but via env var to accomplish this today.
f
It looks like what you proposed will probably work, fighting some other issues, but I think it is converging
And yeah, I'll chime in there
It's probably at least a docs bug
e
Agreed.
Thanks for battling through this.
f
if nothing else, the solution you proposed here might want to be in the docs on the section for custom repositories
Thanks for the help
I've got it working now (took a few tries to construct the 
PANTS_PYTHON_REPOS_INDEXES
correctly). Thanks for your help @enough-analyst-54434
5 Views
Open in Slack
PreviousNext
Powered by