https://pantsbuild.org/ logo
Join Slack
Powered by
Question about `code_quality_tool` I'm trying ``...
# general
s
Question about 
code_quality_tool
I'm trying
Copy code
code_quality_tool(
    name="doctest",
    runnable="//:3rdparty/python:reqs-dev#pytest",
    args=["--doctest-modules"],
    file_glob_include=["**/*.py"],
)
Copy code
def add(a, b):
    """

    >>> add(1, 2)
    4
    """
    return a + b
and expecting 
pants lint demo/:doctest
to fail but it succeeds. What am I doing wrong here?
fwiw 
pytest --doctest-modules demo/
works as expected
doctest
is part of the python standard library so I don't think I need to include any other dependencies
s
Some ideas: 1) 
:doctest
is the target of the tool. You don't need to lint the tool itself 2) 
lint
usually runs a linter per file, but doctest might import other modules, so you might need 
check
, but I'm not sure
s
Copy code
pants check demo/::
20:22:10.18 [ERROR] No relevant backends activate the `check` goal, so the goal would do nothing.
pants lint demo/::
also doesn't return anything
Just to confirm
Copy code
pants run 3rdparty/python:reqs-dev#pytest -- --doctest-modules demo/
works as expected
If I use made up values in 
code_quality_tool
runnable or args it doesn't complain. Just returns 0 silently
This is embarrassing since I've been using pants for >1 year now but why does 
--keep-sandboxes=always
sometimes not do anything? Maybe it's not creating a sandbox? In which case how do I inspect what pants is actually doing
s
have you setup the code_quality_tool properly? with register.py and stuff
s
Don't I just add 
pants.backend.experimental.adhoc
to my backend_packages in pants.toml?
Screen Shot 2024-05-02 at 1.52.01 PM.png
I copied all of the build-support files over and bad_file.py expecting 
pants lint ::
to fail but it doesn't.
s
> Don't I just add 
pants.backend.experimental.adhoc
to my backend_packages in pants.toml? No, this will only add the target 
code_quality_tool
, but then you need to actually configure and register a new mini plugin
here is an example
s
I see. That's the
Copy code
cfg = CodeQualityToolRuleBuilder(
        goal="lint", target="build-support:no_badcode_tool", name="No Bad Code", scope="nobadcode"
    )
in the test
Thanks for all of the help @square-psychiatrist-19087, it's much appreciated.
Just curious why this needs to be registered? When I've used 
adhoc_tool
I didn't need to do anything like that. (I suppose that was runnable though)
s
because 
adhoc_tool
is a single target with concrete graph rules, inputs and outputs, but 
code_quality_tool
is supposed to be run over other targets and it needs to be added to 
lint
goal, which means adding a new rule to the computation graph and pants doesn't support adding rules to computation graph from 
BUILD
files. At least, this is how I understand it
s
That makes sense. That example was really helpful. The docs for https://www.pantsbuild.org/2.20/reference/targets/code_quality_tool make it looks like a regular ol target
Copy code
╰❱ pants lint ::
14:41:26.53 [INFO] Completed: Running code quality tool
14:41:26.53 [ERROR] Completed: pants.backend.adhoc.code_quality_tool.CodeQualityToolRuleBuilder._build_lint_rules.run_code_quality_doctest - doctest failed (exit code 1).
============================= test session starts ==============================
platform darwin -- Python 3.10.13, pytest-8.2.0, pluggy-1.5.0
rootdir: /private/var/folders/m8/cmsvrd417n39w5xjzjhmnznm0000gq/T/pants-sandbox-9DIFNd
collected 1 item

example.py F                                                             [100%]

=================================== FAILURES ===================================
____________________________ [doctest] example.add _____________________________
002 
003     Add two numbers together
004 
005     >>> add(1, 2)
Expected:
    4
Got:
    3

example.py:5: DocTestFailure
=========================== short test summary info ============================
FAILED example.py::example.add
============================== 1 failed in 0.01s ===============================
yay
I'm not sure if doctests is an appropriate use of this tool. Once I made the example a little more complicated I got
Copy code
╰❱ pants lint ::                   
14:55:43.83 [INFO] Initializing scheduler...
14:55:47.30 [INFO] Scheduler initialized.
14:55:48.82 [INFO] Completed: Running code quality tool
14:55:48.82 [ERROR] Completed: pants.backend.adhoc.code_quality_tool.CodeQualityToolRuleBuilder._build_lint_rules.run_code_quality_doctest - doctest failed (exit code 2).
============================= test session starts ==============================
platform darwin -- Python 3.10.13, pytest-8.2.0, pluggy-1.5.0
rootdir: /private/var/folders/m8/cmsvrd417n39w5xjzjhmnznm0000gq/T/pants-sandbox-WhRI4d
collected 0 items / 2 errors

==================================== ERRORS ====================================
___________________ ERROR collecting proj1/src/proj1/foo.py ____________________
ImportError while importing test module 'proj1/src/proj1/foo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
proj1/src/proj1/foo.py:1: in <module>
    from proj2.bar import message
E   ModuleNotFoundError: No module named 'proj2'
___________________ ERROR collecting proj1/src/proj1/foo.py ____________________
ImportError while importing test module 'proj1/src/proj1/foo.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Cellar/python@3.10/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
proj1/src/proj1/foo.py:1: in <module>
    from proj2.bar import message
E   ModuleNotFoundError: No module named 'proj2'
=========================== short test summary info ============================
ERROR proj1/src/proj1/foo.py
ERROR proj1/src/proj1/foo.py
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 2 errors in 0.05s ===============================
I guess this is only meant to be used with static code analysis tools?
s
this is what I was talking about
2) 
lint
usually runs a linter per file, but doctest might import other modules, so you might need 
check
, but I'm not sure
The docs for https://www.pantsbuild.org/2.20/reference/targets/code_quality_tool make it looks like a regular ol target
yeah, that's a hot new feature in 2.20 and it doesn't have a dedicated docs page yet
s
Thanks again for all of the context. I wonder if this tool will be extended to support additional goals like 
check
and 
test
.
s
I personally run doctest via 
python_test
like this:
Copy code
def test_docs() -> None:
    failure_count, test_count = doctest.testmod(my_module)
    assert test_count > 0
    assert failure_count == 0
if you have a lot of modules you need to test, you might want to look into adding new checker https://www.pantsbuild.org/2.20/docs/writing-plugins/common-plugin-tasks/add-a-typechecker I think the checker fits better than a test runner https://www.pantsbuild.org/2.20/docs/writing-plugins/common-plugin-tasks/run-tests even though it technically is a test runner
s
That's a very useful snippet, I'm going to just adopt that for doctests.
4 Views
Open in Slack
PreviousNext
Powered by