Is it possible to avoid this constraint?

absolutely! and we are actually planning to phase out v1

./pants filedeps

in favor of a faster version at some point, which is currently available with

./pants fast-filedeps