What's been going on with CI recently? Seeing some...
# development
h
What's been going on with CI recently? Seeing some flakiness.
h
Known sources: • timeouts of Python tools is likely from the changes to stabilize tool lockfiles, which added new tests like testing that every version of Python works w/ the tool. If it times out, please bump the timeout • JVM tests were restored recently but are still flaky. @fast-nail-55400 has been working on stabilizing. (Should we skip in meantime?) • macOS jobs sometimes time out, no idea why
f
I’ve been looking into the JVM test flakiness. Theory: The engine is running multiple copies of coursier concurrently, and unpacking the JDK download conflicts with other attempts to unpack the download by other coursier processes.
resulting in the “Directory not empty” errors that we are seeing
h
@fast-nail-55400 should we skip those in the meantime?
f
are they consistently breaking CI?
h
maybe 5% of runs?
h
I'm seeing
Failed to load Toolchain token from env var 'TOOLCHAIN_AUTH_TOKEN'
, that's just a warning, but still (cc @polite-garden-50641 when he gets back from vacation)
ImportError: No module named generate_github_workflows
when trying to build generate_github_workflows.pex
I don't see how that change could affect that
h
That's almost certainly not a flake
I'm seeing Failed to load Toolchain token from env var 'TOOLCHAIN_AUTH_TOKEN', that's just a warning,
Asher fixed this last week, only release hasn't been consumed yet
p
the token error is just a UX issue. I am following up on that https://github.com/toolchainlabs/toolchain/issues/10846
h
OK, that error is in fact due to my change interacting badly with something