The test also fails in CI with the same error https://travis-ci.org/pantsbuild/pants/jobs/621528735?utm_medium=notification&utm_source=github_status

It looks like the

cwd

points to different places for v1 and v2. As ./pants is resolved as directory.

if I type

./pants binary --help

, one of the top-level options it displays help about is:

--binary-output-file=<path> (default: None)
    Output to this file.  If unspecified, outputs to stdout.

it looks like this is a generic option on any

ConsoleTask

, but I don't think this makes sense for

binary

, which I think implies we don't want it to exist on the v2 version?

yes!

it's actually a very interesting idea though

i would like to be able to tell pants to produce a binary at a specific path instead of knowing the target name and globbing

dist/

so it looks like some tests (v2 tests I think) are using

Console

with mock stdout and stderr passed into them as the test console, and other tests (maybe v1-only tests?) are using a special

MockConsole

data structure

is there any good reason for this? since I need to rearchitect how Conosle works to fix https://github.com/pantsbuild/pants/issues/8062 I wonder if it also makes sense to get rid of

MockConsole

(perhaps in a follow-up commit)

MockConsole existed first, but I don't think that it needs to

so yea, sounds reasonable

Hey folks, im writing my own target for publish config settings and i'd like to mark the target as needing changes when the config settings change. More context in thread.

Hi all, would love your feedback on this design doc for "v2 plugins": https://docs.google.com/document/d/1E6a7nKq5Lf1G-VRS987ZT77KYkR5DT8JNxb9x4I4xEA/edit?usp=sharing

Maybe not the best place to put it, but any intuition on why the v1

context.target_roots

is a list and not a set?

Hi, I'm trying to build a pex that references a

python_requirements_library

and getting this. Whats the best way to debug it?

Exception message: Package SourcePackage(u'file:///aiq/build/.pants.d/python-setup/resolved_requirements/CPython-2.7.6/mysqlclient-1.4.6.tar.gz') is not translateable by ChainedTranslator(WheelTranslator, EggTranslator, SourceTranslator)

-ldebug

doesnt tell me much. I do have a build cache but disabled it, and

python-repos

is only pointing to a local dir with only that wheel. And this works fine:

pip wheel --no-cache-dir mysqlclient==1.4.6 --wheel-dir=./
pip install ./mysqlclient-xxx-none-linux.whl

I'm on ubuntu14 and ub18 and noticed in one case, it builds a

none

wheel, in another case it builds

cp27mu

. But fails to

pants1.7 binary :mypex

in both cases.

interesting tidbit for anyone interested in options and help display:

dmcclanahan@tw-mbp-dmcclanahan: ~/tools/pants-v3 0:40:50 $? 1
X> ./pants options | g pex.build
Scrubbed PYTHONPATH=/Users/dmcclanahan/tools/pants-v3/src/python: from the environment.
(standard input):1385:pex-builder-wrapper.generate_ipex = False (from HARDCODED)
(standard input):1386:pex-builder-wrapper.ipex_injected_pex_version = 1.6.12 (from HARDCODED)
(standard input):1387:pex-builder-wrapper.setuptools_version = 40.6.3 (from HARDCODED)

./pants options  2.72s user 0.79s system 80% cpu 4.390 total
dmcclanahan@tw-mbp-dmcclanahan: ~/tools/pants-v3 0:41:16
X> ./pants --pex-builder-wrapper-binary-py-generate-ipex options | g pex.build
Scrubbed PYTHONPATH=/Users/dmcclanahan/tools/pants-v3/src/python: from the environment.
(standard input):1385:pex-builder-wrapper.generate_ipex = False (from HARDCODED)
(standard input):1386:pex-builder-wrapper.ipex_injected_pex_version = 1.6.12 (from HARDCODED)
(standard input):1387:pex-builder-wrapper.setuptools_version = 40.6.3 (from HARDCODED)
(standard input):1388:pex-builder-wrapper.binary.py.generate_ipex = True (from FLAG)

./pants --pex-builder-wrapper-binary-py-generate-ipex options  2.55s user 0.63s system 87% cpu 3.654 total

--pex-builder-wrapper-binary-py-generate-ipex

causes the option to be visible in the

binary.py

scope. there's some intelligence about displaying options here that we could apply to the output of

./pants help

(which displays every single option in all its possible scopes)

^that

--generate-ipex

option is from this open PR (https://github.com/pantsbuild/pants/pull/8793)

Hi, is there a way to bundle a set of mock files with my tests ? I've looked to define them as

files

and adding them as a dependency to the tests but they are still not found or not in the correct path, thx.

Hi, have a question on tasks. We're working on some support for publishing python targets, and running into some issues with the wiring. We have the following: `job_target`: this is conceptually similar to a

python_binary

. Thas a single 'main'

python_library

, an entrypoint, and some other params `publish_task`: this is similar to the existing publish, but uses twine. All these steps work well individually. The issue here is getting them to play nicely with each other. What I'd like to do is call our

deploy-job

goal with the

job_target

as a target. This would then in turn call our publish goal, which calls setup.py. The issue that I'm running into is that when setup.py is called, it fails with the error :

FAILURE: setup-py target(s) must provide an artifact.

I imagine this is because it is looking at the

job_target

for a provides argument, which it doesn't have. What we're looking for is to somehow specify that the

round_manager.require_data('python_dists')

line can inform setup.py libraries that this needs to be run against are the python_library deps of

job_target

I've dabbled in the python_binary code to see how this is done, but couldn't exactly grab a hold. Happy to provide more details as well

Hi folks, I tested my simple Presto UDF project with Maven: https://github.com/jiegzhan/prestoUdfs/tree/master/src/main, then I uploaded "prestoUdfs-1.0-SNAPSHOT.jar" to my EMR Presto cluster, it worked. When I ran "jar tf prestoUdfs-1.0-SNAPSHOT.jar", it looks like this:

META-INF/
META-INF/MANIFEST.MF
META-INF/services/
com/
com/roku/
com/roku/dea/
com/roku/dea/presto/
com/roku/dea/presto/udfs/
META-INF/services/com.facebook.presto.spi.Plugin
com/roku/dea/presto/udfs/UdfPlugin.class
com/roku/dea/presto/udfs/ProductUDF.class
META-INF/maven/
META-INF/maven/com.roku.dea/
META-INF/maven/com.roku.dea/prestoUdfs/
META-INF/maven/com.roku.dea/prestoUdfs/pom.xml
META-INF/maven/com.roku.dea/prestoUdfs/pom.properties

Since our team is using pants to build the project, so I copied my files to Pants project folder, and have a BUILD file like this:

resources(name='prestoUdfs-lib-resources',
  sources=['src/main/resources/META-INF/services/com.facebook.presto.spi.Plugin']
)

java_library(
    name='prestoUdfs-lib',
    dependencies=[provided('3rdparty/java/com/facebook:presto-spi'),
                  provided('3rdparty/java/com/google:guava')
                 ],
    sources=globs('*.java'),
)

jvm_binary(
    name='prestoUdfs-artifacts',
    dependencies=[':prestoUdfs-lib',
                  ':prestoUdfs-lib-resources'],
)

Then I built the project, it says

WARN] Globs did not match. Excludes were: []. Unmatched globs were: ["src/main/java/com/roku/dea/presto/udfs/src/main/resources/META-INF/services/com.facebook.presto.spi.Plugin"].

Then I moved META-INF folder to the same folder as .java files, and changed BUILD file to

resources(name='prestoUdfs-lib-resources',
  sources=['META-INF/services/com.facebook.presto.spi.Plugin']
)

java_library(
    name='prestoUdfs-lib',
    dependencies=[provided('3rdparty/java/com/facebook:presto-spi'),
                  provided('3rdparty/java/com/google:guava')
                 ],
    sources=globs('*.java'),
)

jvm_binary(
    name='prestoUdfs-artifacts',
    dependencies=[':prestoUdfs-lib',
                  ':prestoUdfs-lib-resources'],
)

Now my jar file looks like

META-INF/
META-INF/MANIFEST.MF
com/
com/roku/
com/roku/dea/
com/roku/dea/presto/
com/roku/dea/presto/udfs/
com/roku/dea/presto/udfs/META-INF/
com/roku/dea/presto/udfs/META-INF/services/
com/roku/dea/presto/udfs/META-INF/services/com.facebook.presto.spi.Plugin
com/roku/dea/presto/udfs/UdfPlugin.class
compile_classpath/
compile_classpath/src.main.java.com.roku.dea.presto.udfs.prestoUdfs-lib.txt
com/roku/dea/presto/udfs/ProductUDF.class

Looks like META-INF/services/com.facebook.presto.spi.Plugin is not on the root like Maven jar, so my EMR presto cluster can't load this jar. Is there a way to put resources to root path? Thanks for your help.

Hi we are using pants on a scala repo and also using the pants fmt.scalafmt target using our .scalafmt.conf however we noticed that the scalafmt cli that pants is using is locked at version 1.5.1 which does not respect many of our scalafmt conf options which is on version 2.3.2. Is there any way to override this jar dependency of the scalafmt CLI to use the latest in 2.3.2? Thanks everyone

Hi folks, I have a services entry in BUILD file:

java_library(
    name='prestoUdfs-lib',
    dependencies=[provided('3rdparty/java/com/facebook:presto-spi'),
                  provided('3rdparty/java/com/google:guava')
                 ],
    sources=globs('*.java'),
)

jvm_binary(
    name='prestoUdfs-artifacts',
    dependencies=[':prestoUdfs-lib'],
    services={'com.facebook.presto.spi.Plugin': 'com.roku.dea.presto.udfs.UdfPlugin'}
)

I extracted the jar file, somehow com.facebook.presto.spi.Plugin is broken. Do you guys have any thoughts, thanks a lot.

cat META-INF/services/com.facebook.presto.spi.Plugin 
# Generated from pants target src/main/java/com/roku/dea/presto/udfs:prestoUdfs-artifacts
c
o
m
.
r
o
k
u
.
d
e
a
.
p
r
e
s
t
o
.
u
d
f
s
.
U
d
f
P
l
u
g
i
n
# Generated from pants target src/main/java/com/roku/dea/presto/udfs:prestoUdfs-artifacts
c
o
m
.
r
o
k
u
.
d
e
a
.
p
r
e
s
t
o
.
u
d
f
s
.
U
d
f
P
l
u
g
i
n
# Generated from pants target src/main/java/com/roku/dea/presto/udfs:prestoUdfs-artifacts
c
o
m
.
r
o
k
u
.
d
e
a
.
p
r
e
s
t
o
.
u
d
f
s
.
U
d
f
P
l
u
g
i
n

Looks like a classic case of listy string. I imagine the services dict is a map from service interface name to list of implementation names. Try wrapping your 1 impl in a list.

Hello everyone! I know that scala compiler plugins can be used with Pants, but what about sbt plugins? Posted in #scala

Good morning everyone! I have been upgrading dependencies in my project, and it looks like one of the Google dependencies I'm importing (

google-cloud-logging

) puts duplicate classes/resources on the class path. Is there an elegant way to flag this as "OK" in my

3rdparty

build file for this library, or do I have to do

Duplicate.SKIP

in all of my bundle files for those duplicate classes/resources?

===== For target JvmBinary(BuildFileAddress(src/jvm/com/.../graphql/bin/BUILD, bin)):
                   Duplicate classes and/or resources detected in artifacts: ('com.google.api.grpc-proto-google-cloud-logging-v2-0.82.0.jar', 'com.google.api.grpc-proto-google-common-protos-1.17.0.jar')
                        com/google/logging/type/HttpRequest$Builder.class
                        com/google/logging/type/LogSeverityProto.class
                        com/google/logging/type/HttpRequestProto$1.class
                        com/google/logging/type/HttpRequestProto.class
                        google/logging/type/http_request.proto
                        com/google/logging/type/HttpRequest$1.class
                        com/google/logging/type/LogSeverity.class
                        com/google/logging/type/HttpRequestOrBuilder.class
                        google/logging/type/log_severity.proto
                        com/google/logging/type/LogSeverityProto$1.class
                        ... 2 more ...

Both of those artifacts seem to be compile dependencies of https://mvnrepository.com/artifact/com.google.cloud/google-cloud-logging/1.99.0

Hi all, quick bikeshed about where v2 plugins should live in the pants repo. See for context: https://docs.google.com/document/d/1E6a7nKq5Lf1G-VRS987ZT77KYkR5DT8JNxb9x4I4xEA/edit?usp=sharing

OK, how do people feel about v2 backends being named

component

, e.g.,

pants.component.python

, or

unit

, e.g.,

pants.unit.python

? This helps avoid code confusion with existing v1 backends. Note that all these words are somewhat overloaded.

Alternatively, we could do

pants_backend.python

Hey y’all, we’d love your input on this PR to make

./pants help

more useful and accessible to read! No need to look at the code, only the pictures in the PR description and give input on what you like and think we could further improve. https://github.com/pantsbuild/pants/pull/8820

Hey all, is there any way to run a jupyter notebook in a binary pex with correct dependencies ? something like:

python_binary(
    name='jupyter-notebook',
    dependencies = [
        ':package1',
        ':package2',
        'packages-python/third_party:jupyter',
    ],
    source='notebook.py',
)