We need a new name for our

pantsbuild.pants.testinfra

wheel that we release to PyPI because we’re moving the code to

src/python/pants

instead of

tests/python/pants_test

. Thoughts or suggestions? Possible candidates:

pantsbuild.pants.tests

pantsbuild.pants.testutils

pantsbuild.pants.testsupport

pantsbuild.pants.testing

Why does a file move necessitate a new wheel?

Tricky issue with setup-py logic that the exported targets must be “owned” by their own wheel. We must release the tests/ wheel for backwards compatibility for the next two major releases. Those tests/ depend on src/ targets that they simply re-export. If those src/ targets don’t have their own pants_setup_py they belong to, we get an error that they aren’t owned. So, we have to introduce a new wheel for src/ Installing the traditional pantsbuild.pants.testinfra will pull in this new wheel as a prerequisite

So we’ll be releasing two wheels where we currently release one? What will be in the wheel that will remain in

tests/

?

Yes. That wheel simply re-exports the

src/

wheel with a deprecation cycle with a warning to migrate. So, you can still use

from pants_tests.base_tests

but you should use

from pants.testutils.base_test

Aha, so it’s just about import paths?

Yep! And the name of the wheel. The name of the wheel should correspond to the source path. So the wheel

pantsbuild.pants.testutils

implies

src/python/pants/testutils

I simply don’t know what to call the thing

We could just keep it being one wheel, and have both symbols be available in it for two releases?

We could just keep it being one wheel, and have both symbols be available in it for two releases?

That’s what I really wanted to do, but it’s impossible :/ see my comment about “owned” files. I spent a lot of time wrestling with it to try to get one wheel to work

Ugh, how does ownership work? Does it require a shared source-root or something?

Yes exactly. It traverses ancestors and siblings

That’s a pretty frustrating API limitation :S Do you know why?

I don’t. Cc @enough-analyst-54434 who I was DMing this weekend to figure this all out

Well, it's all here: https://github.com/pantsbuild/pants/blob/33a6d51db91ea1fe69c39117c8878abcb824cdfb/src/python/pants/backend/python/tasks/setup_py.py#L247-L265 Maybe my assertion starting 'It's the latter restriction...' was incorrect? It's been a long time since I wrote this...

It sounds like it’s a good general rule which we want to side-step because we now view that ever having had a

pants_test

top-level package was a mistake (and the logic there is in fact codifying that it was a mistake)?

I'm not sure. There is decent test coverage for working on a change though: https://github.com/pantsbuild/pants/blob/33a6d51db91ea1fe69c39117c8878abcb824cdfb/tests/python/pants_test/backend/python/tasks/test_setup_py.py#L54

So if we wanted to make a change, it would be something like

allowed_extra_roots

which we’d recommend people only used for migration?

I'm not sure if we need a flag. I'm never fully confident about when we should babysit vs outright deny vs let folks do what they do. If an existing python user exports a dist with two or more top-level packages they currently can't use pants. With

allowed_extra_roots

we scold them. Perhaps we should just let them be and allowed_extra_roots. Someone else can make that call.

shrug I certainly don’t care enough to do anything about it - I leave it to how much @hundreds-father-404 cares 🙂

My initial thought would allow it without any extra machinery (a flag is too difficult to discover) but log a warning

I meant an attribute on the setup py target, rather than a flag, but just allowing it definitely works for me!

To implement this, the attack will have to be a full repo scan. The benefit of the current restriction is you can walk the tree containing the provides target and be done. To ensure no dual ownership in this new world you have to examine the complete graph. The logic gets simpler, the scan gets bigger. The latter will only currently affect Twitter fapp. Not sure how long a full source scan takes these days. Probably not a big deal since its an infrequent operation hopefully ... although Twitter used to publish thrift jars frequenetly - hopefully that disease has not spread to python land.

you can have a directory called

unittesting

, then put all testing libraries there.

Does that mean all module directories will have `tests/`directory with the unittests in them?

For the internal Pants repo, we’re moving in the direction of having all Pants tests live at the same level as src code in the

src

folder. So, we would have

src/python/pants/util/strutil.py

and

src/python/pants/util/test_strutil.py

This discussion is about where the test util code for Pants should live. Right now, for example, it’s

tests/python/pants_test/test_base.py

and we want to move it to

src/python/pants/testutils/test_base.py

The tricky part is that we release our test utils as a wheel for Pants plugin authors to use in their own testing. We need to (for two release cycles) still release the old stuff.