But I would like to run the tests for our custom pants targets in our CI as well

If your pants plugin(s) are isolated in their own directory tree(s) you could use

./pants --changed... filter | xargs ./pants test

this filters the list of changed targets and passes that filtered list into pants. For example, the 1st half of that pipeline to just run changed contrib/ tests in the pants repo might be:

$ ./pants --changed-parent=HEAD~5 filter --regex=^contrib/ -q
contrib/mypy/src/python/pants/contrib/mypy/tasks:tasks
contrib/python/src/python/pants/contrib/python/checks/tasks/checkstyle:all
contrib/node/src/python/pants/contrib/node/subsystems/resolvers:npm_resolver

If you don't have clean directory tree isolation, you could add tags to your plugin targets and filter on tags.