Is there anyway to force pants to rebuild everything when packaging? I need to increment the version number of my libraries before deployment. I created a plugin as per the documentation that does the trick. For now it reads the version from a file in the repo root. The plugin adds the version to the PKG-INFO and setup.py of all my wheels. However, when I increment the version number and re-run package, it seems to just use a cached version. I tried making the file containing the version number a dependency in the python_distribution target thinking it would see a change and rebuild, but no luck.

Hm, I'm afk but try --no-pantsd to isolate if it's being cached to disk vs memoized in memory

Yup that works

Huh. I would have expected the file watcher to pick up the change 🤔 if you have a moment, it would be helpful to add log statements to your plugin https://www.pantsbuild.org/docs/rules-api-logging If the rule is memoized, it won't log what's inside because the rule won't execute. So, you could add a log to before where you read the file with PathGlobs for example, then after, to isolate where the memorization is happening when you don't expect it. That will help us figure out if there's a legit Pants bug

I already did have logging in the plugin. After the first run no more logs. With -no-pantsd I see the logging everytime.

which API are you using to read the file in the plugin?

Right now in the plugin I'm just using os.read, which I'll eventually change. But the thing is is that the plugin doesn't even get called again after the initial package.

that’s the source of the bug

kk - I did however make the version file a dependency in the python_library target. Thought that would cause it to detect the change. Guess not. Ok I'll read the version file the proper way and see if that works

Pants uses filesystem watching to know when to re-run

@rules

, but in order to know what to watch, specific APIs need to be used to access the filesystem

Makes sense. Is there a way to just read a loose file or do I have to make it a dependency in the distribution target?

in this case you probably want to do something like:

contents = await Get(DigestContents, PathGlobs(["my/version/file.txt"]))
version_content = contents[0].content.decode()

Trying now

sorry for the trouble… we’d eventually like to make it harder to accidentally use

os.read

/ etc with better sandboxing

No worries. I knew I was supposed to use the file API but took a shortcut. My bad.

Would it be useful to cite os.read in the docs, or elaborate on what other calls are implied in the docs by "work with the filesystem"? https://www.pantsbuild.org/docs/rules-api-file-system

It is not safe to use os.path or pathlib.Path like you normally would because this can break caching. Instead, Pants has several mechanisms to work with the file system in a safe and concurrent way.

yea, will mention. thanks!