pantsbuild #general

For the record: https://github.com/pantsbuild/pants/issues/4800 I'm experimenting with this and will update that issue so watch there if you're interested.

enough-analyst-54434

08/09/2017, 5:06 AM

K - got S3 working, PR soon to unblock CI and use S3.

enough-analyst-54434

08/09/2017, 5:18 AM

I'll TBR https://github.com/pantsbuild/pants/pull/4801 if it goes green.

enough-analyst-54434

08/09/2017, 5:19 AM

External users can make a similar change too to work around the bintray outage if that PR goes green. I'll send an email to pants-devel@ if so spelling out the issues / workaround / path forward longer term.

rough-minister-58256

08/09/2017, 5:21 AM

🚢

rough-minister-58256

08/09/2017, 8:38 AM

thanks for fixing that up!

enough-analyst-54434

08/09/2017, 9:06 AM

You're welcome. Hopefully I left enough context on https://github.com/pantsbuild/pants/issues/4800 and https://github.com/pantsbuild/binaries/issues/29 for anyone else that needs to work with this. Should be smooth though.

astonishing-jelly-60479

08/09/2017, 2:53 PM

is it cost-effective to put cloudfront in front of S3 for this too?

astonishing-jelly-60479

08/09/2017, 2:59 PM

also, my pants is trying to fetch

bin/native-engine/linux/x86_64/af2e5b09cb7eee743c8d4e065258ac046c8a19ac/native_engine.so

from bintray which doesn't exist in the S3 bucket or in the binaries repo 😞

enough-analyst-54434

08/09/2017, 3:11 PM

Not sure about CDN cost @astonishing-jelly-60479. I realized the native engine publishing hole just this am. Getting breakfast now and will see how far I can get towards fixing that hole here after breakfast. I'll pass the torch if I can't get it done before I need to bolt in a few hours.

astonishing-jelly-60479

08/09/2017, 3:12 PM

thanks @enough-analyst-54434!

astonishing-jelly-60479

08/09/2017, 3:13 PM

maybe fastly would be interested in providing some bandwidth: https://www.fastly.com/open-source/

enough-analyst-54434

08/09/2017, 3:14 PM

Yeah - worth checking. This really spiralled and may still be a stop gap. Wider committer team will need to weigh in on path forward.

famous-salesclerk-49406

08/09/2017, 3:31 PM

<!here|@here> - our build pipeline is down because of the permission issue on bintray. Is anyone looking into this?

enough-analyst-54434

08/09/2017, 3:32 PM

Please read above, but yes.

enough-analyst-54434

08/09/2017, 3:52 PM

For folks hitting failures to fetch native engine version XYZ, if you could please add the version of pants you're using to https://github.com/pantsbuild/pants/issues/4800, that would be helpful. I should have publishing set up here shortly to our .travis.yml setup but then there will be the matter of back-populating native engine versions folks need nowish, then doing a full backfill after. Unlike all other binaries, which are checked into https://github.com/pantsbuild/binaries, native engine blobs get uploaded straight to bintray ... and we can't pull them down of course.

enough-analyst-54434

08/09/2017, 4:49 PM

K, this should do it for native-engine publishes going forward: https://github.com/pantsbuild/pants/pull/4804

enough-analyst-54434

08/09/2017, 4:50 PM

It'll only run on a tag though so I'll try to hack up something to pre-emptively test w/o a tag

enough-analyst-54434

08/09/2017, 4:54 PM

I've got no love listing of pants versions used by affected folks on https://github.com/pantsbuild/pants/issues/4800 - I'll start manually creating / uploading major releases since native-engine intro then work backwards on dev releases

fast-nail-55400

08/09/2017, 4:58 PM

are there notes/writeups from the summit?

fast-nail-55400

08/09/2017, 4:58 PM

just curious what went on 🙂

mysterious-farmer-45668

08/09/2017, 5:03 PM

so we need to manually apply https://github.com/pantsbuild/pants/pull/4801?

mysterious-farmer-45668

08/09/2017, 5:03 PM

to work around the bintray issue?

enough-analyst-54434

08/09/2017, 5:08 PM

Yes @mysterious-farmer-45668 but just the pants.ini part