I've been down the

system_binary

rabbit hole ... Trying to run

dbt compile

across a few projects.

dbt

is installed in globally in an pyenv python dist -

which dbt

gives

/Users/rhysmadigan/.pyenv/shims/dbt

, which is

#!/usr/bin/env bash
set -e
[ -n "$PYENV_DEBUG" ] && set -x

program="${0##*/}"

export PYENV_ROOT="/Users/rhysmadigan/.pyenv"
exec "/opt/homebrew/opt/pyenv/bin/pyenv" exec "$program" "$@"

This in turn depends on a pyenv shim, which then needs other tools. This is what I've come up with:

system_binary(
    name="bash",
    binary_name="bash",
    fingerprint_args=["--version"],
)

system_binary(
    name="grep",
    binary_name="grep",
    fingerprint_args=["--version"],
)

system_binary(
    name="sed",
    binary_name="sed",
    fingerprint=r".*?",
    fingerprint_args=["q"],
)

system_binary(
    name="awk",
    binary_name="awk",
    fingerprint_args=["--version"],
)

system_binary(
    name="cut",
    binary_name="cut",
    # Cut needs a file - any file
    fingerprint_args=[
        "-f",
        "-4",
        env("HOME") + "/.pyenv/shims/dbt",
    ],
)

system_binary(
    name="sort",
    binary_name="sort",
    fingerprint_args=["--version"],
)

system_binary(
    name="basename",
    binary_name="basename",
    fingerprint_args=["'test'"],
)

system_binary(
    name="dbt",
    binary_name="dbt",
    fingerprint_args=["--version"],
    fingerprint_dependencies=[
        ":bash",
        ":grep",
        ":sed",
        ":awk",
        ":cut",
        ":sort",
        ":basename",
    ],
    extra_search_paths=[
        "/usr/bin",
        "/usr/local/bin",
        "/opt/homebrew/bin",
        env("HOME") + "/.pyenv/shims",
    ],
)

files(
    name="dbt-sources",
    sources=["**/*.yml", "**/*.sql"],
)

adhoc_tool(
    name="dbt-compile",
    runnable=":dbt",
    runnable_dependencies=[
        ":bash",
        ":grep",
        ":sed",
        ":awk",
        ":cut",
        ":sort",
        ":basename",
    ],
    args=["compile"],
    execution_dependencies=[
        ":dbt",
        ":dbt-sources",
    ],
    output_files=[
        "dbt_project.yml",
    ],
    root_output_directory=".",
)

Which is run with

pants export export-codegen ::

Please let me know if there's a better way of doing this! 🤯

Update: I was able to get it working with a pex_binary which is probably the better approach.

That's the path I've been going down for

dbt