https://pantsbuild.org/ logo
Powered by
h

high-yak-85899

05/19/2022, 4:11 AM
How would I go about setting up my own definition of 
publish
for a pex binary?
Seems like plugin territory
h

hundreds-father-404

05/19/2022, 4:14 AM
yeah, see src/python/pants/backend/python/goals/publish.py for Pants's Twine implementation
h

high-yak-85899

05/19/2022, 4:31 AM
thanks. Lots of reading to do. I want to make it easy to publish my pex binaries to a generic repository in our hosted Artifactory system
I think after I peruse the existing target/rule docs, I'll be in a good place to see how docker things work and model my changes after that to have a custom extension of 
pex_binary
targets
❤️ 1
h

hundreds-father-404

05/19/2022, 4:36 AM
let us know if you have questions! I'm anxious to get better plugin docs already..
h

high-yak-85899

05/19/2022, 4:45 AM
I do love how strongly typed everything is. It makes it so much more intuitive to follow.
➕ 2
One quick question: Trying to follow the principle of placing plugins separate from in-repo code. My plugin is going to need a third-party library and it would be nice if I used the same library version between my plugin and first party code. Any proper way to keep those synchronized?
Actually, I think I see in the docs where this is addressed
But next question: I'd like vscode to have a clue about the things I'm using when developing a plugin in. What do the pros normally do for developing pants code? I'm following https://www.pantsbuild.org/docs/plugins-overview#building-in-repo-plugins-with-pants and naively tried to do 
./pants export ::
but that didn't work.
h

hundreds-father-404

05/19/2022, 5:04 AM
Generally, we think it's a good idea to use a dedicated 
[python].resolves
for your pants-plugin to keep plugin code decoupled from internal code But, with multiple resolves, you can use the same input 
python_requirement
target in multiple resolves. For example, say you're using `requirements.txt`:
Copy code
python_requirements(
   overrides={"django": {"resolve": paramatrize("pants-plugin", "python-default"}},
)
and naively tried to do ./pants export :: but that didn't work.
do you have the 
pants_requirements()
target generator set up?
h

high-yak-85899

05/19/2022, 5:05 AM
Yup, I did that and it failed at
Copy code
A distribution for pantsbuild-pants-testutil could not be resolved in this environment.
h

hundreds-father-404

05/19/2022, 5:06 AM
anything more to it?
h

high-yak-85899

05/19/2022, 5:07 AM
Not much 😞
Copy code
(mono-venv) ~/devel/monorepo (pex-binary-publish-plugin)$ ./pants export ::
23:00:51.06 [INFO] Initialization options changed: reinitializing scheduler...
23:00:51.25 [INFO] Scheduler initialized.
23:01:01.88 [INFO] Starting: Installing constraints.txt for the resolve `python-default`
23:02:18.04 [INFO] Completed: Installing constraints.txt for the resolve `python-default`
23:02:18.04 [INFO] Starting: Extracting 87 requirements to build requirements.pex from python-default_lockfile.pex: Flask-SimpleLDAP==1.4.0, Flask-SocketIO==5.0.1, Flask==1.1.2, PyVISA-py>=0.3.1, PyVISA>=1.8, SQLAlchemy<2,>=1.4.3... (1716 characters truncated)
23:02:19.78 [INFO] Completed: Extracting 87 requirements to build requirements.pex from python-default_lockfile.pex: Flask-SimpleLDAP==1.4.0, Flask-SocketIO==5.0.1, Flask==1.1.2, PyVISA-py>=0.3.1, PyVISA>=1.8, SQLAlchemy<2,>=1.4.3... (1716 characters truncated)
23:02:19.78 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Extracting 87 requirements to build requirements.pex from python-default_lockfile.pex: Flask-SimpleLDAP==1.4.0, Flask-SocketIO==5.0.1, Flask==1.1.2, PyVISA-py>=0.3.1, PyVISA>=1.8, SQLAlchemy<2,>=1.4.31, aiotask_context, ansible-runner==2.0.3, ansible<3,>=2.10, asciitree>=0.3.3, asn1<3,>=2.5, astropy<6.0,>=5.0, asyncssh<2.9,>=2.4.2, awscli==1.19.0, bcrypt>=3.1.3, boto3==1.17.0, botocore<2,>=1.20.0, click-option-group>=0.5.2, click<8.0,>=7.1.2, cloud_sptheme==1.10.1.post20200504175005, coloredlogs>=6.0, conscript<1,>=0.1.3, cryptography<37.0,>=36.0, docker==5.0.0, dohq-artifactory<1,>=0.8, elasticsearch<8,>=7, ephem>=3.7.6.0, eventlet==0.30.2, fire>=0.1.2, flatdict>=4.0.1, freezegun>=1.1.0, google-api-python-client>=1.6.5, grafana-api>=1.0.3, graphviz>=0.8.4, gspread>=3.0.1, influxdb<6,>=5.2.0, ipython<8,>=7.20, jinja2<4.0,>=3.0, marshmallow==3.13.0, matplotlib>=0.0.1, numpy<1.22.0,>=1.20.1, oauth2client>=4.1.2, openpyxl>=3.0.9, pandas<=1.3.5,>=1.0, pantsbuild.pants.testutil<2.12,>=2.11.0rc0, pantsbuild.pants<2.12,>=2.11.0rc0, paramiko<3.0,>=2.0, pexpect>=4.4.0, phabricator==0.8.1, pika<2,>=1.2.1, portpicker<2,>=1.2.0, prompt-toolkit<4,>=2, protobuf==3.19.1, psutil>=5.8.0, psycopg2-binary==2.8.6, pyModbusTCP>=0.1.8, pyOpenSSL==20.0.1, pyftpdlib>=1.5.5, pyparsing<3, pyserial-asyncio>=0.4, pyserial>=3.4, pysofa2, python-can>=3.1.1, python-dateutil<3,>=2.8.0, pytz>=2017.3, pyusb==1.2.1, pyyaml>=5.1, pyzmq<20,>=19.0.2, requests<3,>=2.27, rtl_tools@ <git+ssh://git@git.corp.astranis.com/source/rtl_tools.git@ea9d0ea6187058b50401b20f03b91365382f1552>, scipy>=1.1.0, setuptools<62,>=61.2.0, sgp4<3,>=2, skyfield<2,>=1.42, slackclient>=2.7.2, sortedcontainers==2.4.0, sphinx-click<4,>=3.0.2, sphinx-rtd-theme<2,>=1.0, sphinx<5,>=4.0, tabulate>=0.8.1, testing.postgresql==1.3.0, typeguard==2.12.1, types-PyYAML==6.0.0, types-paramiko<3,>=2.8.13, typing-extensions<5,>=4.1.1, urllib3==1.26, yubihsm<3,>=2.1' failed with exit code 1.
stdout:

stderr:
A distribution for pantsbuild-pants-testutil could not be resolved in this environment.



Use `--no-process-cleanup` to preserve process chroots for inspection.
h

hundreds-father-404

05/19/2022, 5:12 AM
hm, unfortunate. what is your 
[python].interpreter_constraints
?
h

high-yak-85899

05/19/2022, 5:15 AM
interpreter_constraints = ["CPython>=3.8,<3.9"]
. I think this was the default? Can't remember
h

hundreds-father-404

05/19/2022, 5:21 AM
okay then that's not it. the wheel only works w/ >=3.7,<3.10. But yours is a subset
h

high-yak-85899

05/19/2022, 5:25 AM
Yeah, if I do a normal 
pip install <the constraints it has>
it works, but it has conflicts with other constraints. Maybe a separate lockfile for plugin stuff will handle this.
👍 1
🤞 1
I think that could help, but is there a way to only export one resolve? Right now, 
export
doesn't work for my normal first party resolves (separate issue).
But as far as I'm aware, I only know of 
./pants export ::
. Should I do 
./pants export pants-plugins/::
?
👍 1
Answer seems to be yes. Feel free to unsubscribe from my scratch pad 🙂
h

hundreds-father-404

05/19/2022, 5:48 AM
exactly, and 
export
creates one venv per resolve. you can switch in your IDE between which you want activated
h

high-yak-85899

05/19/2022, 5:51 AM
One issue that I did run into is that I want pants to generate the lock for pants-plugins and not for normal repo code, but I only get one 
resolves_generate_lockfiles
boolean to set. Easy enough to set to 
true
when I need to and run 
generate-lockfiles --resolve=pants-plugins
and then change back, but something to be aware of. We're hoping to switch to the pex-made lockfiles soon, but have a couple things blocking that for now.
It's kind of confusing that extending a field means I add a callable 
def rules()
to my 
register.py
module. With such intent to separate the notion of target and rule, why I have to do that has gone over my head. Nevertheless, in ~ 1 hour I've 1) set up a new plugin directly 2) generated a lockfile for it 3) bootstrapped the environment so I can develop it and 4) successfully added my own custom field to 
PexBinary
targets. Now on to implementation, but definitely a credit to the docs that I can get that far that quickly.
❤️ 1
w

wide-midnight-78598

05/19/2022, 11:59 AM
4 Views
Powered by