https://pantsbuild.org/ logo
Join Slack
Powered by
daft question on the new option types: is there a ...
# development
w
daft question on the new option types: is there a way to validate that the values of two (or more options) are valid among themselves? shall I do it in a 
@rule
? I’m giving it a go to https://github.com/pantsbuild/pants/issues/13067 which requires the addition of two new options to manage the total memory for child processes and default memory assigned each child process. These two options are bound together and therefore I would like to validate that 
total process mem > single process mem
and error out to the end user when the given values do not meet that criteria.
h
Hi, not a daft question, but there is no such way that I can recall. Probably best to do it where you consume the option values?
w
thought about that, it’s just that these values will be sent to pantsd and consumed inside the engine.
I assume in that case if I put the error in the 
Result<…>
type used in the Rust code, the error will get back to the client…?
mmm, in fact if I add the validation in the Rust code and make it return an 
Err(…)
, it seems that 
pantsd
panics with error message I put inside the 
Err
but leaves the client side of pants in a loop printing 
waiting for pantsd to start…
🤨
I think it’d be better if I make a draft PR so you guys have more context to see what I’m trying to do …
b
I'm pretty sure whatever subsystem you're adding the options to will be able to hold your validation code. You could always make the option attributes private and expose their values together as a tuple or something in a property, and validate there
h
You could create a ProcessMemory type and a rule that takes the relevant options subsystem and produces an object of that type for downstream code to consume?
w
right, think I got a bit of tunnel vision
I’ll try making the attributes private and then declaring a custom type, will need to pass that type through the FFI into the rust code but I don’t think that is going to be hard
✅ 1
b
You can still use raw types if you want, just have them be returned as-is (perhaps a tuple of values, which PyOx understands IIRC) from your property post-validation
w
mmm, I’m even considering whether this is something worth addressing in this fashion… the two options that are being added are bootstrap options and any other example of those kind of options being passed to 
pantsd
and validated is the following one: https://github.com/pantsbuild/pants/blob/8cbcac89f5ea8f92cbfff41e7145e65e02c531cc/src/rust/engine/src/externs/interface.rs#L331
so, if I mess around with the value of shards in 
pants.toml
I get a similar behaviour: 
pantsd
panics (or basically exits) and the client goes in a loop waiting for it to start.
the way the error looks like to the end user is as follows (this is my code using same validation):
Copy code
Traceback (most recent call last):
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/bin/pants_loader.py", line 119, in <module>
    main()
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/bin/pants_loader.py", line 115, in main
    PantsLoader.main()
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/bin/pants_loader.py", line 109, in main
    cls.run_alternate_entrypoint(entrypoint)
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/bin/pants_loader.py", line 85, in run_alternate_entrypoint
    entrypoint_fn()
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/pantsd/pants_daemon.py", line 211, in launch_new_pantsd_instance
    daemon = PantsDaemon.create(options_bootstrapper)
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/pantsd/pants_daemon.py", line 57, in create
    core = PantsDaemonCore(options_bootstrapper, executor, cls._setup_services)
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/pantsd/pants_daemon_core.py", line 51, in __init__
    self._options_initializer = OptionsInitializer(options_bootstrapper, executor)
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/init/options_initializer.py", line 101, in __init__
    self._bootstrap_scheduler = create_bootstrap_scheduler(options_bootstrapper, executor)
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/init/options_initializer.py", line 74, in create_bootstrap_scheduler
    EngineInitializer.setup_graph(
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/init/engine_initializer.py", line 186, in setup_graph
    return EngineInitializer.setup_graph_extended(
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/init/engine_initializer.py", line 308, in setup_graph_extended
    scheduler = Scheduler(
  File "/Users/antonioalonsodominguez/Workspace/Personal/pants/src/python/pants/engine/internals/scheduler.py", line 202, in __init__
    exec_stategy_opts = PyExecutionStrategyOptions(
ValueError: Nailgun pool can not be initialised as the total amount of memory allowed is smaller than the memory allocation for a single child process.
        
        - total child process memory allowed (bytes): 268435456
        - default child process memory (bytes): 536870912
        
16:59:26.41 [INFO] waiting for pantsd to start...
16:59:31.43 [INFO] waiting for pantsd to start...
^CInterrupted by user:
so this makes me wonder if this could be solved by having some error handling in the scheduler.py 
___init___
method such that the error is shown to the user in a more graceful manner…?
or if not in the scheduler, it could be put in the 
pants_loader.py
Open in Slack
PreviousNext
Powered by