#18669 `archive` is not reproducible Issue created by huonw Describe the bug The

archive

target's output can change at a byte level even when its inputs is identical. This appears to be due to timestamps changing, but there may be other factors too (e.g. file sort order, permissions, groups/owners). This reproducer runs two builds of .zip and .tar archives, using a fixed file on disk. This may be worse if generating inputs to

archive

too (e.g.

packages=[...]

or using the output of

shell_command

), but I haven't tested in detail. The

--no-pantsd --no-local-cache

args ensures that the archiving process definitely runs, as might happen when running on different machines (i.e. no shared cache).

cd $(mktemp -d)

cat > pants.toml <<EOF
[GLOBAL]
pants_version = "2.16.0rc0"

backend_packages = []

[anonymous-telemetry]
enabled = false
EOF

echo 1 > foo.txt

cat > BUILD <<EOF
file(name="file", source="foo.txt")
for format in ["zip", "tar"]:
    archive(name=format, format=format, files=[":file"])
EOF

pants --no-pantsd --no-local-cache package ::
cp -r dist first

sleep 3 # ensure the invocations are separated

pants --no-pantsd --no-local-cache package ::
cp -r dist second

# BUG: these are different
shasum first/* second/*

# BUG: for tar, it is specifically the (octal) timestamp that differs
diff -U3  <(hexdump -C first/tar.tar) <(hexdump -C second/tar.tar)

If I run that, I see (nondeterministic) output like:

9c0f0769e85cc4a6acc490f92ea32b79862a953a  first/tar.tar
669f4c618dac7c3590f392f9c5f23670731456d6  first/zip.zip
81b0b4dba3c99855642ce468500c89afa8c9f862  second/tar.tar
9be595741557454a409d243aa0bf84ac91f7b5e2  second/zip.zip

--- /dev/fd/11	2023-04-04 14:53:36.000000000 +1000
+++ /dev/fd/12	2023-04-04 14:53:36.000000000 +1000
@@ -4,7 +4,7 @@
 00000060  00 00 00 00 30 30 30 36  34 34 20 00 30 30 30 37  |....000644 .0007|
 00000070  36 35 20 00 30 30 30 30  32 34 20 00 30 30 30 30  |65 .000024 .0000|
 00000080  30 30 30 30 30 30 32 20  31 34 34 31 32 37 32 36  |0000002 14412726|
-00000090  35 30 30 20 30 31 32 35  30 37 00 20 30 00 00 00  |500 012507. 0...|
+00000090  35 31 30 20 30 31 32 35  31 30 00 20 30 00 00 00  |510 012510. 0...|
 000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 *
 00000100  00 75 73 74 61 72 00 30  30 68 75 6f 6e 00 00 00  |.ustar.00huon...|

Pants version 2.16.0rc0 OS macOS Additional info N/A pantsbuild/pants