Ergonomic, hermetic, user-friendly software build system for Python, Go, Java, Kotlin, Protobuf, Scala, Shell. Pants lets you fearlessly scale up your codebase!

Pants

<@UBV8BCQPJ>: re 6660: you should have push access on twitter/pants

```
 ~  workspace  otherpants  dwagnerhall/v2/urlfetch  4⬆  git push twitter dwagnerhall/v2/urlfetchERROR: Permission to twitter/pants.git denied to blorente.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
```

<@UBV8BCQPJ>: it looks like it will need a rebase. while i work on getting you access to that org, i'm going to rebase your branch from <https://github.com/blorente/pants/tree/dwagnerhall/v2/urlfetch> and push it to the PR. you'll want to pick that up before resuming

<@UBV8BCQPJ>: i'll continue to look at the access issue, but you might consider opening a new PR to iterate on <https://github.com/pantsbuild/pants/pull/6660> with travis

Makes sense, I’ll do that before the day ends.

Done. I also addressed the 400 and 500 codes yesterday, but I think I didn’t push.

would a review be helpful now, or are you mostly focused on getting it green?

Probably not worth doing now, unless travis comes out green (which I doubt will happen) I should have it by Monday

from the run on the original PR, it looks like we might need openssl installed in travis, which should be an apt-get away i think...?

(just a superficial assessment... not certain.)

That would make sense, we do fetch external stuff as part of the tests, albeit from public places.

well, i mean that it failed with a bunch of errors about loading openssl

(...but only on the non-docker linux shards)

This is the failing shard that uses docker: <https://travis-ci.org/pantsbuild/pants/jobs/447747293>

hey. so i took a look at this, but not for long enough to actually fix it: sorry!

i was optimistic about this change: ```$ git diff -- src/rust/engine/Cargo.toml
diff --git a/src/rust/engine/Cargo.toml b/src/rust/engine/Cargo.toml
index 40dd82c..c2e2478 100644
--- a/src/rust/engine/Cargo.toml
+++ b/src/rust/engine/Cargo.toml
@@ -89,3 +89,7 @@ tokio = "0.1"
 tempfile = "3"
 ui = { path = "ui" }
 url = "1.7.1"
+
+[target.'cfg(not(any(target_os = "windows", target_os = "macos", target_os = "ios")))'.dependencies]
+# TODO: We experience linker errors on Linux without vendored openssl.
+openssl-sys = { version = "= 0.9.39", features = ["vendored"] }``` based on what it mentions here: <https://docs.rs/openssl/0.10.15/openssl/#vendored>

but it fails in the docker image with a collision with the openssl we use for gRPC

the gRPC thing is interesting, and has me wondering whether that is related in the non-vendored case as well

sorry i wasn't able to make more progress =(

if you'd like to pick up any of the other tickets from <https://github.com/pantsbuild/pants/projects/6> , the assignments should currently be accurate

to run docker, can do: ```docker run -v "$(pwd):/pantsbuild" -w '/pantsbuild' --rm -it pantsbuild/centos6:latest /bin/bash```

Thanks! My problem was with sharing the `.cache` folder and permissions, to not have to download everything again. In any case, I was fiddlign with the travis_ci image, not the centos one. But this one is better, thanks!

BUT you could in theory mount a brand new "linux only HOME directory" somewhere under your OSX home dir, and reuse that

-.js

That would be my latest change, which makes me think my tests were non-deterministic because they passed Travis. But yeah, that was definitely me, apologies for the inconvenience: <https://github.com/pantsbuild/pants/pull/6516/files#diff-96019bf81bd494185ef01f01d9e76be8>

Opened <https://github.com/pantsbuild/pants/issues/6687> to track it.

i have not. a few things landed today, but master is green, afaik

Thanks! That looks great, except it has no option for async requests that I can see. The actual reading and storing of the file can be async, because it returns a stream, but nobody can save us from the network latency. That said, even if we were to make the request async, we do wait for it to complete before returning from the node execution, so we’d probably not be gaining much?

Yea, I think the API in the ticket was currently blocking anyway. This would be a stopgap

<@UBV8BCQPJ>: There is a CpuPool that we can/should run this kind of work on to avoid blocking the tokio threads... I forgot to mention that in the review.

Okay, cool :slightly_smiling_face: thanks.

<@U06A03HV1> You might want to check out the comment I just left regarding the travis failures

Would you like me to keep trying to figure out the SSL error with `reqwest`? If we eventually need to switch it would be useful, but I don’t have a good intuition of how much of a stopgap `chttp` is.

i'm going to look into the segfault i think.

<@UBV8BCQPJ>: are there any other issues that look interesting for you?

the "improve engine UI" one is relatively small, without dependencies

the "Consume logger/console" one would interact fairly heavily with python code, but is otherwise fairly small

I thought that was going to be a bigger one

yea, it would be. i take that back. "medium to large"

Assuming I will be ooo for most of next week, I think the improve engine UI is better

(re: #6660) I can finally repro the SSL error (in my personal laptop), and I’m 100% sure a version of libssl-dev is installed, because I got a different error until I installed it. My current hypothesis is that it’s a missing linker flag, because it actually builds the native-engine library without issues. Could be that, or the wrong version of libssl-dev

<@U6ZRNH0TC>: do you have time to look at <https://github.com/pantsbuild/pants/pull/6716> ? the fix is legit for osx, but breaks on linux

yes, i'm about to finish up another issue in about 5 minutes so perfect timing

yeah that issue is 100% legit and my fault, didn't realize that it of course breaks on linux vs osx, looking into it

if anyone has feedback on <https://github.com/pantsbuild/pants/issues/6004#issuecomment-435712270> , it would be appreciated. will start it sometime this week

<@U06A03HV1> <@UBV8BCQPJ> I think I have the openssl thing working

(Literally as a reached the “I will give up on this and just defer until we switch to tower” point)

Ok, I have seen the openssl thing working on both linux and mac now

But it requires the person building pants to have an env var specified which points at their openssl install… Does anyone know of any tips for locating openssl [i.e. its include and lib dirs]? There are a few standard-ish directories AFAICT…

<@U6ZRNH0TC> <@U04S45AHA> may have some ideas…?

<@U6ZRNH0TC> Any better ideas for “Where is openssl” than `for dir in /usr /usr/local /usr/local/Cellar ...; do if [[ -e "${dir}/openssl/include" ]]; then echo "Found openssl: ${dir}"; done`?

hey sorry they moved my desk three floors and it’s screwing with my back

is pkg-config not available or not working?

Doesn’t appear to work for brew-installed openssl

<@U6YPB4SJX>: could you set the env var conditionally on linux, and let OSX continue to do what it was doing?

(assuming that it is the openssl-sys crate consuming this...?)

Sadly not; we’re changing how grpcio is consuming libssl - before it was statically linking from bundled sources, and now it’s dynamically linking from the system (which means it needs to find the system install)

I don’t think this is going to end up looking too horrible… But something broke between the version I saw working on linux and the version I PR’d from a mac, so I’m back to Linux :slightly_smiling_face:

so, the `openssl-sys` crate uses the `pkgconfig` crate to find things

pkg-config the crate works on osx but it may be that pkg-config the utility is borked or can’t see brew installed openssl

it does talk about its strategies though: <https://docs.rs/openssl/0.10.15/openssl/>

So <@U6YPB4SJX> using `pkg-config libssl --variable=includedir` and `pkg-config libssl --variable=libdir` manually for osx fails to reveal brew openssl paths?

```
$ pkg-config --list-all | grep -i ssl
grpc_unsecure             gRPC unsecure - high performance general RPC framework without SSL
grpc++_unsecure           gRPC++ unsecure - C++ wrapper for gRPC without SSL
libevent_openssl          libevent_openssl - libevent_openssl adds openssl-based TLS support to libevent
```

But `/usr/local/Cellar/openssl/1.0.2n` exists and is good

And what about the brew version of pkg-config? Are there two and you're using OSX's?

I have two `pkg-config`s. The other one shows the same result except without `libevent_openssl`

So… Different, but not in a useful way :smile:

my link above to <https://docs.rs/openssl/0.10.15/openssl/#automatic> was to suggest that some copy-pasta from whatever they are doing might be a good idea

because they reference `brew` and pkgconfig awareness

where are the two pkg-configs? if one is non homebrew and one is from homebrew that might make sense (or not at all) — having more than one is confusing to me but i can’t say whether it’s strange or wrong 

```
$ which -a pkg-config
/opt/twitter_mde/bin/pkg-config
/usr/local/bin/pkg-config
```

I’ll have a look at the openssl link, hopefully it will make everything clear :slightly_smiling_face: Thanks for your help!

hey, i had to go home because of the desk thing but i am back now. i can't determine whether there is something i can do to assist here, but since it is whatever o clock in london right now, i have time to pick it up today if that would unblock you

No worries :slightly_smiling_face: I’ve got it - thanks!

ok, awesome! i am _deeply_ aware of the fact that `pkg-config` can be screwy and _especially_ with homebrew (and i should really turn my feelings about homebrew into pull requests). i'm going to figure out and fix the GOROOT issue you identified a few days ago right now (stu mentioned there could be an interaction with an existing go install, which i hadn't considered, so the problem may be way simpler)

I commented on <https://github.com/pantsbuild/pants/issues/6666>. Didn’t get enough time to finish it, but it’s close in case someone wants to pick it up.

<@U06A03HV1> Are the `pants.engine.legacy.structs.BaseGlobs` subclasses still used?

...in a very roundabout way, perhaps? just for the initial conversion?