I have created pex for my EMR serverless. when I unzip the pex I can see a module inside .deps/jsonschema.. but the script which is importing it is not able find this module. I provide main.py and pex to emr with boto3 from a lambda. I created pex_binary in build. does anyone faced similar issues? if someone suggest better way to run pyspark script on emr when dependency is also on other repos of the Monorepo.

(answered over at https://pantsbuild.slack.com/archives/C087V4P1T/p1741911628864519?thread_ts=1741854887.676689&cid=C087V4P1T 😄)

@broad-processor-92400 Thanks it worked and i was able to use the libs but now i have another problem as my libs are build with >+3.10 python version and emr supports the 3.9 by default. it there a way I can provide main.py and pex to emr and this pex will have the venv for 3.10 , so emr will use the venv instead default.

I know nothing about the specifics of EMR. If EMR is using a Python 3.9 interpreter, at the very least you’ll need to find a EMR (or related service) option to change this. A 3.9 interpreter cannot use a 3.10+ venv Alternatively, if EMR is stuck on 3.9, that’s what you’ll need to build.