https://pantsbuild.org/ logo
#general
Title
# general
h

hundreds-father-404

12/12/2018, 5:25 PM
I’m trying to file a flaky test issue for failing contrib shard, but am unclear what test is actually failing. Someone mind helping me to parse the logs? https://api.travis-ci.org/v3/job/467047088/log.txt
Maybe it’s this line:
contrib/avro/tests/java/org/pantsbuild/contrib/avro:avro[
? That’s the most recent Pants line I see before stderr and stack trace
a

average-vr-56795

12/12/2018, 5:26 PM
Looks like it was a network connectivity issue downloading the errorprone jar from maven?
r

red-balloon-89377

12/12/2018, 5:27 PM
You can Ctrl-F for
:: problems summary ::
👌 1
It failed to fetch a jar
h

hundreds-father-404

12/12/2018, 5:28 PM
Okay that’s helpful. Thank you both! And it looks like it failed to fetch the JAR during the
contrib/avro/tests/java/org/pantsbuild/contrib/avro:avro
tests, right?
Do we open
flaky-test
issues for networking problems?
r

red-balloon-89377

12/12/2018, 5:28 PM
Please do! It’s great to have them documented 🙂
👌 1
e

enough-analyst-54434

12/12/2018, 5:44 PM
I'm not sure I agree with that! There is flaky expected - which is any test that uses network, and there is flaky unexpected - we wrote a bad test or are using non-deterministic data structures. The former are probably never going away, the latter we should always be able to squash.
So from a label perspective, it seems to me network flakes just add noise to cleanup efforts. I advocate for a separate label or just not doing this.
👍 1
h

hundreds-father-404

12/12/2018, 5:46 PM
@enough-analyst-54434 thoughts on a new tag
expected-flaky
? I’d be happy to make right now!
a

aloof-angle-91616

12/12/2018, 5:46 PM
what are your thoughts on adding retries to particularly flaky network requests?
👍 1
e

enough-analyst-54434

12/12/2018, 5:47 PM
I don't believe there is such a thing as particularly flaky network requests. I would guess maven central, pypi and cargo central are all about as stable.
a

aloof-angle-91616

12/12/2018, 5:47 PM
there was that go download with the meta import tag that kept flaking? i need to page back into that
h

hundreds-father-404

12/12/2018, 5:48 PM
I like retries. Could increase the number of timeouts we get, but better that than flaky imo. Either way, I like the idea of still somehow cataloging in GitHub issues so that when you’re debugging the failure you don’t have to reproduce the effort some already put in to seeing if the failure is a flake
👍 1
a

aloof-angle-91616

12/12/2018, 5:48 PM
in that case the answer might be to change what we are making requests for
e

enough-analyst-54434

12/12/2018, 5:48 PM
Let me put it this way. Over the last few days I've fought flakes with a vengeance and the incidence of network to our bugs was maybe 1/10
I think our buggy tests are a much bigger problem
👍 3
a

aloof-angle-91616

12/12/2018, 5:48 PM
ok, that makes sense
i would feel more comfortable with a separate tag with much less priority? and tickets would only be made if there is relevant background or investigation, or an idea on how to fix them? i do 100% agree the buggy tests are the focus and this might help me to focus on that -- but i'm not wedded to the thought
e

enough-analyst-54434

12/12/2018, 5:50 PM
I'm fine with a separate tag.
👍 1
h

hundreds-father-404

12/12/2018, 5:50 PM
Okay, I’m thinking either a label
acceptable-flaky-test
or
expected-flaky-test
a

aloof-angle-91616

12/12/2018, 5:51 PM
ci-network-failure
or
low-priority-flaky-test
?
e

enough-analyst-54434

12/12/2018, 5:51 PM
network-flaky
👍 2
a

aloof-angle-91616

12/12/2018, 5:52 PM
@hundreds-father-404 are you planning to make an issue with that tag for the above issue? a description on that tag that says it's lower priority might be helpful, not sure how github surfaces those descriptions in the ui
h

hundreds-father-404

12/12/2018, 5:54 PM
Yes, making the label right now this is what I have for description:
Copy code
Flaky test resulting from network issues, implying the flake does not result from a bug in our code
❤️ 1
a

aloof-angle-91616

12/12/2018, 5:54 PM
thanks!
❤️ 1
e

enough-analyst-54434

12/12/2018, 5:55 PM
Thanks folks
💯 2
a

aloof-angle-91616

12/12/2018, 5:56 PM
thanks for being the adult in the room and acknowledging the flakiness needs to be addressed! this increases my sanity tenfold
❤️ 1
h

hundreds-father-404

12/12/2018, 5:56 PM
I’ll update the email thread to loop everyone in on this
👌 1
a

aloof-angle-91616

12/12/2018, 6:00 PM
we can update that label in the future when the sun begins to degrade and we have to guard against cosmic rays (pants will be around until then along with my brain in a jar (file))
😂 1
👌 1
3 Views