limited-insurance-37393
11/19/2020, 8:38 PMhundreds-father-404
11/19/2020, 8:48 PMpoetry.lock
- we’re working on adding first-class support for consuming that file
In the meantime, the simplest approach is to run poetry export
to generate a requirements.txt file in the root of your project (the “build root”).
Then, add a BUILD file at the build root that says python_requirements()
, which is a macro that converts each entry in requirements.txt
into a Pants target
https://www.pantsbuild.org/docs/python-third-party-dependencieslimited-insurance-37393
11/19/2020, 8:49 PMhundreds-father-404
11/19/2020, 8:50 PM./pants list ::
hundreds-father-404
11/19/2020, 8:50 PMpython_tests
targets, for examplelimited-insurance-37393
11/19/2020, 8:53 PMlimited-insurance-37393
11/19/2020, 8:56 PMlib
or resource
targets. For example: https://github.com/mitodl/ol-data-pipelines/blob/putting_on_pants/src/ol_data_pipelines/edx/solids.py#L25-L33limited-insurance-37393
11/19/2020, 8:56 PM./pants dependencies --dependencies-type=source-and-3rdparty src/ol_data_pipelines/edx/
//:dagster
//:httpx
src/ol_data_pipelines/edx/__init__.py
src/ol_data_pipelines/edx/api_client.py
src/ol_data_pipelines/edx/repositories.py
src/ol_data_pipelines/edx/schedule.py
src/ol_data_pipelines/edx/solids.py
dagster==0.9.14
httpx==0.16.1
hundreds-father-404
11/19/2020, 9:01 PMsrc/BUILD
and src/ol_data_pipelines/{edx,lib,resources}/BUILD
. All of those have python_library()
targets which have the same files in the sources()
field. This means that dep inference won’t work for imports of those files, as Pants can’t disambiguate which of the two targets you want to use for metadata
It’s valid to have only src/BUILD
with the **/*.py
sources. Alternatively, we generally recommend having 1 BUILD file in each directory with nothing more than this line:
python_library()
That uses the default sources
field of ["*.py", "!*_test.py", "!test_*.py"]
https://www.pantsbuild.org/docs/targets#target-granularityhundreds-father-404
11/19/2020, 9:03 PMAll of those have python_library() targets which have the same files in the sources() field.You can see which targets “own” a file by running
./pants list path/to/file.py
. Generally, you want it to be only 1 valuelimited-insurance-37393
11/19/2020, 9:08 PMhundreds-father-404
11/19/2020, 9:12 PMsrc
, as you aren’t importing src.ol_data_pipelines
, but instead ol_data_pipelines
src
, so you could leave it undefined too)limited-insurance-37393
11/19/2020, 9:12 PMedx
package to be a target that I package as an independently deployable unit, does that mean that it should have a BUILD
definition in that directory, rather than just having the one definition at the root of src
?limited-insurance-37393
11/19/2020, 9:15 PMlib
and resource
modules in the dependency list.limited-insurance-37393
11/19/2020, 9:15 PMwitty-crayon-22786
11/19/2020, 9:16 PM./pants dependencies
is not transitive by default: if you’d like to see everything depended on transitively, need to pass --transitive
witty-crayon-22786
11/19/2020, 9:18 PMresources
need to be depended on explicitly, because pants cannot infer their usagewitty-crayon-22786
11/19/2020, 9:18 PMhundreds-father-404
11/19/2020, 9:18 PMwitty-crayon-22786
11/19/2020, 9:19 PMwitty-crayon-22786
11/19/2020, 9:19 PMhundreds-father-404
11/19/2020, 9:45 PM./pants
bash script defaults to running with Py36, then 37, then 38
To fix, run curl -L -o ./pants <https://pantsbuild.github.io/setup/pants>
to pull in recent changes we made to the script, then change lines 140 and 141 to only use 3.8
We’re fixing this so Pants always does the right thing, regardless of what interpreter the tool is run with.
(Black and MyPy have this same issue - the typed-ast
library did not add support for Py38, so the only way to parse an Py38 AST is to run with 3.8)hundreds-father-404
11/19/2020, 9:45 PMlimited-insurance-37393
11/19/2020, 9:55 PMlimited-insurance-37393
11/19/2020, 9:58 PMhundreds-father-404
11/19/2020, 10:00 PM./pants dependencies src/ol_data_pipelines/edx/api_client.py
and hadn’t double checked it actually made first-party imports 🤦)
Sorry about the trouble with Py38 - definitely a rough edge that will be fixed after this PR later todaylimited-insurance-37393
11/19/2020, 10:00 PMhundreds-father-404
11/19/2020, 10:32 PMhundreds-father-404
11/23/2020, 5:14 PMSo, for the case where I want the edx package to be a target that I package as an independently deployable unit, does that mean that it should have a BUILD definition in that directory, rather than just having the one definition at the root of src?Pardon the delay with this one. So, the
python_library
is sort of a misnomer. It doesn’t imply any thing about how you deploy your code, it’s nothing more than a way to set metadata on some Python source code. A more apt name would be python_srcs
, for example.
The reason to use more granular vs. coarser python_library
targets is how fine-grained you want your metadata to be. For example, you might want to only set one file to have overridden Python interpreter constraints, vs. an entire subtree. We generally recommend 1 BUILD file per directory because it tends to be the simplest heuristic and scales well whenever you do need to add metadata
For your deployables, you explicitly say which dependencies you want in it, then those and all their transitive deps will be pulled in. You can either depend on a whole python_library
target, which is shorthand for depending on all its files in the sources
field, or you can pick out certain files from it using “file addresses” to be more fine-grained. https://www.pantsbuild.org/docs/targets#dependencies-and-dependency-inferencelimited-insurance-37393
11/23/2020, 6:40 PMlimited-insurance-37393
11/23/2020, 6:40 PMhundreds-father-404
11/23/2020, 6:42 PMversion
limited-insurance-37393
11/23/2020, 6:45 PM