<@U02KAN6061E> Does this asserted assumption mean ...
# development
@bitter-ability-32190 Does this asserted assumption mean that fmt plugins cannot add files? https://github.com/pantsbuild/pants/blob/2.15.x/src/python/pants/core/goals/fix.py#L392-L394 That conflicts with a feature (prob an accidental feature) I tried to expose as an official feature that was released in 2.14: https://github.com/pantsbuild/pants/pull/15846 Also, I think that's what is preventing me from changing elements in my fmt Batch to be FieldSets instead of files.
As of today yes. Thanks for the link, that sparked my memory. The code is very hierarchical, which helps and hurts. But ultimately is to reduce mistakes. So linters are opaque in the handoff between partitioner and runner. Fixers (which include formatters) don't get the luxury because we have to run the tools serially so we have to reduce partitions into a common base.
That being said, I wouldn't be surprised if there's a way to hack it to work. I think what you'd need is a field set type whose Source isn't hydrated? Or is hydrated via your codegen?
But yeah, I think you're shoving a square peg in a round hole. The hole used to be bigger than the square, but it got narrower.
Ok. I think I've got things sorted so I don't need the field set any more. But what about Fmt/fix plugins adding files?
YMMV on that.
We can try and set up some time to discuss it in a call. Maybe figure where best something like that belongs
Yes please.
I've beaten my head on this for awhile and would love some help. When might be best for you?
Let's figure that out next week? Ive been OoO all week so I have no clue what next week looks like 😬
LOL. The joys of after-vaca-catch-up. Let me know when you know 😄
Yes. That assertion prevents adding files.
Copy code
15:35:09.47 [WARN] Completed: Update contrib/schemas/*.json with st2-generate-schemas - st2-generate-schemas made changes.
15:35:09.47 [ERROR] 1 Exception encountered:

Engine traceback:
  in `fmt` goal

AssertionError: Expected ('contrib/schemas/action.json', 'contrib/schemas/alias.json', 'contrib/schemas/pack.json', 'contrib/schemas/policy.json', 'contrib/schemas/rule.json', 'contrib/schemas/sensor.json') to match ('contrib/schemas/action.json', 'contrib/schemas/alias.json', 'contrib/schemas/pack.json', 'contrib/schemas/rule.json', 'contrib/schemas/sensor.json')
Yeah I'm not surprised. Fmt/Fox wasn't designed to add files
Remember that the core code has to serialize the calls and partitions as well. We need to know what we're operating on ahead of time 😟
Now I'm thinking about adding a way for a subsystem to opt-in to being able to add files during fmt. (essentially something that switches that assert from
IMHO That seems like it's best handled through other means. Codegen or a new goak
The whole point is it has to fit into the normal flow of what people do all the time
No new goal. No
./pants run
. No special script. Because we already have that (an ugly makefile target and a script) and people don't run it.
Well in this case you aren't formatting or fixing files. Sooooooo
Yeah - logically it fits into codegen... But codegen is so isolated it's not useful.
I disagree. I actually use it a lot in our repo 🙂
Restricting the export-codegen to only touch
makes it pretty worthless (for this usecase)
I think there's a ticket for that 😉
It would have to be some kind of codegen that runs during lint (just like fmt) and fails if codegen would make changes to the in-repo generated files. I suppose if we had something that prompted people to run
./pants export-codegen ...
similar to the prompt to run
./pants fmt
then that would work. I'm starting to burn out on this migration.
I'll attempt to be patient and wait till we can chat next week. Sorry to bother you.
In this case id just wrap the pants invocation in a shell script. We gobble those like candy
ick. no
No wrapping anything in shell scripts
I'm trying to delete all of those
"No more Makefiles!" is my mantra
I guess I still don't understand how these are codegenerated but not in repo but also not codegenerated-the-pants-way
The migration to pants is still in-progress (ouch. this is taking awhile) So, until that migration is complete, all the old infra has to continue working. Once I can show that all tests are passing both under a pants exported venv and under the old stuff, then I'll be able to start dropping the old Makefiles and actually lean into using pants more directly. As is, things have to work both under pants and under the Makefile.
We have codegenerated files in repo. We check them with a unit test. If it doesn't match expected we give a nice error message about what to run
It will be awhile before I can migrate the tests so that they can actually run under pytest and therefore be run by pants.
But I still don't understand how fmt is supposed to format a file that doesn't exist before it's run
It's more like "populate this directory" than "format this file". No matter what gets passed in, the correct files get spit out. If someone deletes a file, (or if they add code for a new type where there is no existing file), then it should just show up. The developer experience has been so bad, I'm trying to minimize the number of things people have to learn.
are the only bits I'm planning to document, and then point interested folks at the pants docs for anything beyond that. Yeah. I think I'm just being a Karen at this point. I should get off my soap box -- the world does not revolve around my use cases.
There's value and knowledge in every use case
@bitter-ability-32190 when you get a chance, would you check out my proposal: https://github.com/pantsbuild/pants/discussions/18235 It builds on some of the insights from this thread.
👀 1