https://pantsbuild.org/ logo
Join Slack
Powered by
here is my use case - currently using poetry to de...
# general
a
here is my use case • currently using poetry to deploy a mono lambda -> multiple lambdas in one folder that get build as a singular zip file and used with api gateway proxy mapping • what I am trying to figure out is If I can use pants to do the following ◦ split each lambda into a separate folder ◦ as poetry helps with the dependency management would like to keep using that if possible ◦ somehow using a CI/CD to create a zip file for each of the new lambda folders thanks in advance
r
Yes I think this should be possible with pants. • define a 
python_awslambda
target inside each folder • poetry has some nice features but you can use pants directly to manage all the dependencies. Pants can read the pyproject.toml and only pick the necessary dependencies for each lambda, So you can use a single source of dependencies. Read more here how pants manages 3rd party dependencies Read this about 
python_awslambda
https://www.pantsbuild.org/docs/awslambda-python
h
BTW you can continue to use your existing pyproject.toml, with a 
poetry_requirements
target, and as @refined-addition-53644 says, Pants will take the correct subset of it for each lambda. That way you have a single consistent set of deps in the repo, but each lambda won’t have to package and ship stuff it doesn’t need. So basically you define the list of possible requirements using poetry syntax, but you let Pants use that list in its own way.
a
so when packaging the application is it possible to package it without the dependencies.
also thanks for the quick response
also given that if pants is able to detect dependencies well. will it support mixn i.e. 3rd party dependencies defined by pyproject.toml and then rely on Pants detecting me using a shared folder
reason why am asking is because in poetry its not possible to define local folder as dependency and get it packaged together
Hi @happy-kitchen-89482, @refined-addition-53644 with a set up like this. am I missing something from a config perspective. also how do I manage my lock files with this as each folder should and will contain its own lockfile
r
The point is to not define separate lockfile for each folder unless there is some compelling reason (like you need different versions of same 3rd party). Define a common lockfile and pants will handle everything else like picking only necessary 3rd party dependencies for a specific lambda.
Copy code
# my pants.toml
[python.resolves]
common = "3rdparty/python/common.lock"

# BUILD with poetry_requirements (should be where pyproject.toml is)
poetry_requirements(resolve="common")

# generate lockfile first by running
./pants generate-lockfiles --resolve=common
Also you don’t need to define local dependencies like the way poetry does. As long as you import that module/package in your lambda code, pants will package it for you. In some cases where this doesn’t happen, you can do it explicitly when defining python_awslambda target. I would strongly recommend following the tutorials because I feel like they explain it better.
a
ok ace thanks. let me have a crack at it. also given my set up do I need to define a python_test in the build. as it currently complains when I i define it.
r
Also you need to use same resolve for any 
python_sources
, 
python_test
or 
python_awslambda
where you want to fetch 3rd party packages from the 
common
resolve.
Copy code
python_sources(resolve="common")
In order to avoid typing resolve in every target you can use 
___defaults___
as explained here I have something like this in my top level BUILD file
Copy code
# define a default resolve for the all the targets below
__defaults__(all=dict(resolve="common"))
By the way 
common
is very generic name. I don’t exactly use that name. So pick something more informative, if you would like
Yes you need to define 
python_tests
if you want to run your tests using pants (which is recommended).
a
ok let me try the readme and get back to you
hi followed the the backed-end awslambda tutorial and I got stuck. am I still missing something. giving this setup
and this pants.toml file
example BUILD for lambda-one and lambda-two
Copy code
__defaults__(all=dict(resolve="common"))
poetry_requirements(
    name="poetry",
    resolve="common",
)

# The default `sources` field will include our handler file.
python_sources(name="lib")

python_awslambda(
    name="lambda-one",
    runtime="python3.9",
    # Pants will convert this to `project.lambda_example:example_handler`.
    handler="lambda_handler.py:lambda_handler",
)
output from running 
./pants roots
Copy code
.
lambda-one
lambda-two
shared-code
i get the following error when i run the package command either as
Copy code
8:49.71 [ERROR] 1 Exception encountered:

  Exception: Unmatched glob from CLI arguments: "lambda-one/lambda-one/lambda_handler.py"

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
Copy code
./pants package lambda-one/lambda_handler.py
14:58:33.04 [ERROR] 1 Exception encountered:                                                                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                                                                    
  Exception: Unmatched glob from CLI arguments: "lambda-one/lambda_handler.py"                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                    
Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
@refined-addition-53644 am i sitll missing set up?
r
your set-up looks fine. For building a specific lambda target inside 
lambda-one/lambda_one
, you should run
Copy code
./pants package lambda-one/lambda_one:<lambda_target_name>

# So in this case it should be
./pants package lambda-one/lambda_one:lambda-one
h
Or I think you can do 
/pants package lambda-one/lambda-one/lambda_handler.py
? That might work
But 
./pants package lambda-one/lambda_handler.py
won’t work because that path is not a correct path from the cwd to the file
12 Views
Open in Slack
PreviousNext
Powered by