#17747 Deploy pex files to Databricks jobs/pipelines using dbx Issue created by ericfeunekes Is your feature request related to a problem? Please describe. I would like to use pants to create pex files for each databricks job that I create (a python script) and then deploy those files to the applicable job clusters. Describe the solution you'd like The dbx plugin would look at the dbx

conf

files to determine which files relate to each job, it would build each of those scripts into a pex file and then deploy those to databricks to be installed when the job starts (likely using an init script) Describe alternatives you've considered Currently doing this with the dbx integration with Poetry. dbx builds a single

whl

file and deploys that to be installed on all the job clusters that use the package. The problem with this approach is that the build gets bigger and bigger as new jobs are added and the number of dependencies expands, even if those dependencies only relate to a single job. This means install times and therefore job startup times get longer. pantsbuild/pants