#22200 `package` goal fails with `--changed-since=...` when including non UTF-8 files in `files` target Issue created by krylowicz Describe the bug I'm using files target to include text files that are used in unit tests. Some of these files intentionally contain non-UTF-8 characters, which are important for the test cases. However, when I run the pants package goal, it fails with an error related to character encoding. It appears only when using

--changed-since=...

, where the diff includes changes to non UTF-8 files. pants --print-stacktrace -ldebug --changed-since=a7b2003d --changed-dependents=transitive package 214454.56 [DEBUG] File handle limit is: 10000 214454.81 [INFO] Initializing Nailgun pool for 24 processes... 214454.81 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution. 214458.30 [DEBUG] Launching 1 roots (poll=false). 214458.34 [DEBUG] computed 1 nodes in 0.033700 seconds. there are 10 total nodes. 214458.34 [DEBUG] Launching 1 roots (poll=false). 214458.34 [DEBUG] computed 1 nodes in 0.000759 seconds. there are 14 total nodes. 214459.09 [DEBUG] File handle limit is: 10000 214459.14 [INFO] Initializing Nailgun pool for 24 processes... 214459.14 [DEBUG] Using [cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }, cache::CommandRunner { inner: bounded::CommandRunner { inner: SwitchedCommandRunner { .. }, .. }, .. }] for process execution. 214503.04 [DEBUG] specs are: Specs(includes=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=True, from_change_detection=False), ignores=RawSpecs(description_of_origin='CLI arguments', address_literals=(), file_literals=(), file_globs=(), dir_literals=(), dir_globs=(), recursive_globs=(), ancestor_globs=(), unmatched_glob_behavior=<GlobMatchErrorBehavior.error: 'error'>, filter_by_global_options=False, from_change_detection=False)) 214503.05 [DEBUG] changed_options are: ChangedOptions(since='a7b2003d', diffspec=None, dependents=<DependentsOption.TRANSITIVE: 'transitive'>) 214503.05 [DEBUG] Launching 1 roots (poll=false). 214503.09 [DEBUG] computed 1 nodes in 0.045949 seconds. there are 10 total nodes. 214503.09 [DEBUG] Launching 1 roots (poll=false). 214503.10 [DEBUG] Completed: acquire_command_runner_slot 214503.10 [DEBUG] Running Searching for

bash

on PATH=/usr/bin/bin/usr/local/bin:/opt/homebrew/bin under semaphore with concurrency id: 1, and concurrency: 1 214503.11 [DEBUG] /home/kkr/.cache/pants/lmdb_store/immutable/files -> /tmp hardlinkable: true 214503.11 [DEBUG] Completed: setup_sandbox 214503.12 [DEBUG] spawned local process as Some(695) for Process { argv: ["./find_binary.sh", "bash"], env: {"PATH": "/usr/bin/bin/usr/local/bin:/opt/homebrew/bin"}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, inputs: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: {} }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Searching for

bash

on PATH=/usr/bin/bin/usr/local/bin:/opt/homebrew/bin", level: Debug, append_only_caches: {}, jdk_home: None, cache_scope: PerRestartSuccessful, execution_environment: ProcessExecutionEnvironment { name: None, platform: Linux_x86_64, strategy: Local }, remote_cache_speculation_delay: 0ns, attempt: 0 } 214503.13 [DEBUG] Completed: Searching for

bash

on PATH=/usr/bin/bin/usr/local/bin:/opt/homebrew/bin 214503.13 [DEBUG] Completed: Scheduling: Searching for

bash

on PATH=/usr/bin/bin/usr/local/bin:/opt/homebrew/bin 214503.13 [DEBUG] Completed: acquire_command_runner_slot 214503.13 [DEBUG] Running Test binary /bin/bash. under semaphore with concurrency id: 2, and concurrency: 1 214503.13 [DEBUG] Completed: setup_sandbox 214503.14 [DEBUG] Completed: acquire_command_runner_slot 214503.14 [DEBUG] Running Test binary /usr/bin/bash. under semaphore with concurrency id: 3, and concurrency: 1 214503.14 [DEBUG] Completed: setup_sandbox 214503.14 [DEBUG] spawned local process as Some(696) for Process { argv: ["/bin/bash", "--version"], env: {}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, inputs: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: {} }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Test binary /bin/bash.", level: Debug, append_only_caches: {}, jdk_home: None, cache_scope: PerRestartAlways, execution_environment: ProcessExecutionEnvironment { name: None, platform: Linux_x86_64, strategy: Local }, remote_cache_speculation_delay: 0ns, attempt: 0 } 214503.15 [DEBUG] spawned local process as Some(697) for Process { argv: ["/usr/bin/bash", "--version"], env: {}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, inputs: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: {} }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Test binary /usr/bin/bash.", level: Debug, append_only_caches: {}, jdk_home: None, cache_scope: PerRestartAlways, execution_environment: ProcessExecutionEnvironment { name: None, platform: Linux_x86_64, strategy: Local }, remote_cache_speculation_delay: 0ns, attempt: 0 } 214503.15 [DEBUG] Completed: Test binary /usr/bin/bash. 214503.15 [DEBUG] Completed: Test binary /bin/bash. 214503.15 [DEBUG] Completed: Scheduling: Test binary /usr/bin/bash. 214503.15 [DEBUG] Completed: Scheduling: Test binary /bin/bash. 214503.15 [DEBUG] Completed: Finding the

bash

binary 214503.16 [DEBUG] Completed: acquire_command_runner_slot 214503.16 [DEBUG] Running Searching for

git

on PATH=/usr/bin/bin/usr/local/bin:/opt/homebrew/bin under semaphore with concurrency id: 4, and concurrency: 1 214503.16 [DEBUG] Completed: setup_sandbox 214503.17 [DEBUG] spawned local process as Some(698) for Process { argv: ["./find_binary.sh", "git"], env: {"PATH": "/usr/bin/bin/usr/local/bin:/opt/homebrew/bin"}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<74daa8164942a954dfa7b0e771e4a38e58b3dd98c3f2daffa9d54fcd98b6626a>, size_bytes: 91 }, tree: "Some(..)" }, nailgu… pantsbuild/pants