Hi all hopefully this is a dumb question I m having trouble Pants #general

Hi all, hopefully this is a dumb question! I'm hav...

quiet-evening-25363

11/30/2021, 2:33 AM

Hi all, hopefully this is a dumb question! I'm having trouble with a simple example of importing a JSON file defined in a

resource

rule into a test. I'll post the code and test output in the thread, but

pkgutil.get_data("util", "test.json")

always returns

None

. A related question... • Is there a better way to debug whether the file is included in the package other than inferring from the output of

pants peek

? Thanks!

quiet-evening-25363

11/30/2021, 2:34 AM

Copy code

☁  python [main] ⚡  tree
.
├── BUILD
├── Dockerfile
├── pants
├── pants.toml
├── requirements.txt
└── util
    ├── BUILD
    ├── convert.py
    ├── convert_test.py
    └── test.json

Copy code

☁  python [main] ⚡  bat util/BUILD     
───────┬─────────────────────────────────────────────────────────────────────────────────────────
       │ File: util/BUILD
───────┼─────────────────────────────────────────────────────────────────────────────────────────
   1   │ python_sources()
   2   │ 
   3   │ python_test(
   4   │     name="convert_test",
   5   │     source="convert_test.py",
   6   │     dependencies=[
   7   │         ":test_data",
   8   │     ],
   9   │ )
  10   │ 
  11   │ resource(
  12   │     name="test_data",
  13   │     source="test.json",
  14   │ )
───────┴─────────────────────────────────────────────────────────────────────────────────────────

Copy code

☁  python [main] ⚡  bat util/convert_test.py 
───────┬─────────────────────────────────────────────────────────────────────────────────────────
       │ File: util/convert_test.py
───────┼─────────────────────────────────────────────────────────────────────────────────────────
   1   │ import json
   2   │ import pytest
   3   │ import pkgutil
   4   │ 
   5   │ from util.convert import core_to_paper
   6   │ 
   7   │ def test_core_to_paper_succeeds() -> None:
   8   │     data = pkgutil.get_data("util", "test.json")
   9   │     assert data is not None
  10   │     paper = core_to_paper(json.loads(data.decode("utf-8")))
  11   │     assert True
───────┴─────────────────────────────────────────────────────────────────────────────────────────

Copy code

=================================== FAILURES ===================================
_________________________ test_core_to_paper_succeeds __________________________

    def test_core_to_paper_succeeds() -> None:
        data = pkgutil.get_data("util", "test.json")
>       assert data is not None
E       assert None is not None

util/convert_test.py:9: AssertionError
---- generated xml file: /tmp/process-executionyBiOiy/util.convert_test.xml ----
=========================== short test summary info ============================
FAILED util/convert_test.py::test_core_to_paper_succeeds - assert None is not...
============================== 1 failed in 0.09s ===============================

Copy code

☁  python [main] ⚡ ./pants peek //util:convert_test
02:08:08.01 [INFO] Initializing scheduler...
02:08:08.13 [INFO] Scheduler initialized.
[
  {
    "address": "util:convert_test",
    "target_type": "python_test",
    "dependencies": [
      "util:test_data",
      "util/convert.py"
    ],
    "dependencies_raw": [
      ":test_data"
    ],
    "description": null,
    "experimental_resolve": null,
    "extra_env_vars": null,
    "interpreter_constraints": null,
    "runtime_package_dependencies": null,
    "skip_black": false,
    "skip_tests": false,
    "source_raw": "convert_test.py",
    "sources": [
      "util/convert_test.py"
    ],
    "tags": null,
    "timeout": null
  }
]
☁  python [main] ⚡ ./pants peek //util:test_data  
02:08:25.86 [INFO] Initializing scheduler...
02:08:25.99 [INFO] Scheduler initialized.
[
  {
    "address": "util:test_data",
    "target_type": "resource",
    "dependencies": [],
    "dependencies_raw": null,
    "description": null,
    "source_raw": "test.json",
    "sources": [
      "util/test.json"
    ],
    "tags": null
  }
]

enough-analyst-54434

11/30/2021, 2:39 AM

I think you need to add an empty

__init__.py

for

pkgutil

to work here (see last sentences): https://docs.python.org/3/library/pkgutil.html#pkgutil.get_data

enough-analyst-54434

11/30/2021, 2:43 AM

On the related question, try `./pants filedeps //util:convert_test`: https://www.pantsbuild.org/docs/reference-filedeps

quiet-evening-25363

11/30/2021, 3:32 AM

yes adding

__init__.py

fixed it, thank you! and great I'll take a look at filedeps

quiet-evening-25363

11/30/2021, 3:36 AM

not a huge deal, but when I run

filedeps

the .json file is not included, even though it seems like it might be based on the wording in the docs. Is that the expected behavior?

Copy code

☁  python [main] ⚡ './pants filedeps //util:convert_test'
03:34:13.84 [INFO] Initializing scheduler...
03:34:13.98 [INFO] Scheduler initialized.
util/BUILD
util/convert_test.py

enough-analyst-54434

11/30/2021, 3:39 AM

Did you try

--transitive

- it is one level of indirection away in the

resource

target.

quiet-evening-25363

11/30/2021, 3:44 AM

nice thank you transitive shows what I'd expect 🙏

5 Views

Open in Slack

Previous Next