https://pantsbuild.org/ logo
Join Slack
Powered by
I'm getting a cryptic error on the rust side (with...
# development
b
I'm getting a cryptic error on the rust side (with some new changes), I think it's coming from trying to read a file on disk with perms 
600
, and my user. When running with the daemon I see the error, without I see a different error (later in the code). What perms/user does the daemon run as?
e
ps -ef | grep pantsd
or your favorite variant to see the user.
b
Running as my user 🤔
Copy code
$ ps -ef | grep pantsd
josh       31864   18485  0 09:31 pts/0    00:00:00 grep --color=auto pantsd
The file in question:
Copy code
$ ls -l ~/.aws/credentials 
-rw------- 1 josh josh 116 Nov 15 08:56 /home/josh/.aws/credentials
e
The ps has 0 hits though, that's just a hit for your grep
b
I don't think so, see 
--color=auto
? Makes me think it matched the daemon
e
Maybe widen the scope to python or just read the pid file to find the pid.
b
oh thats the arg to grep, lol
e
Then muck about in /proc/<pid>/... - 18 ways
b
better:
Copy code
josh       36453   12042  4 09:46 ?        00:00:07 pantsd [/home/josh/work/pants]
still me
e
Depending what / where the error is, this could be a good thing. If a X00 file makes it in the lmdb, you've likely just been owned, by yourself. Anything in lmdb can be just as easily remoted and the visible to everyone hooked up to your remoting.
b
This is a file being read directly on the rust side. For transparency I'm looking into maybe supporting s3 as a URI, and that requires AWS credentials
e
Ok, watch that lmdb!
b
So I'm leveraging a library that reads the correct config(s) directly. With the daemon I get some vague error (I blame the library) without the daemon I have no affliction
e
Ok, I'll let you have the pleasure of finding things out there. Just wanted to raise that dire prospect in case you were drifting there. The number of times Twitter leaked credentials to OSS was embarrassing, left mental scars.
b
Well it's still a slight cry for help. I'm admittedly out of my element as I've only used Unix for a year now 😛 If the daemon is running as 
josh
, should it not be able to read a file whose owner is 
josh
? Perhaps I'll test this with an empty file in my source tree with those perms
e
Please provide all details if you want help. What's the backtrace / error and where is the code. Same as I'd ask of a general question.
b
Anyway to get a traceback from native code?
e
RUST_BACKTRACE=1 for rust controlled panics. If it's segfaulty, you need to attach gdb to the process and then get thread backtraces from within the gdb session.
b
Something has gone very sideways...
Copy code
11:06:50.38 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
11:06:53.89 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
11:06:57.41 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
11:07:00.92 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
11:07:04.44 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
11:07:07.95 [INFO] Filesystem changed during run: retrying `ExportCodegen` in 500ms...
Turning on 
RUST_BACKTRACE
seems to kick the filewatcher continusously
Oddly the log in 
.pants.d/run-tracker
shows:
Copy code
11:15:25.14 [31m[ERROR][0m panic at 'called `Result::unwrap()` on an `Err` value: CredentialsNotLoaded(CredentialsNotLoaded { source: "no providers in chain provided credentials" })', src/downloads.rs:119
11:15:25.14 [31m[ERROR][0m Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
This is with:
Copy code
$ RUST_BACKTRACE=1 ./pants export-codegen //:dumpit
e
So you have a panic identified now. Presumably you're off and running.
b
No, since it distinctly isn't logging the backtrace
e
Well you have file and line.
b
I knew that previously, it's in the code I'm authoring. That line just calls into a 3rdparty library
e
Ok. Well, get creative! You need pantsd to launch with that env var, make sure it is getting launched with that.
Catch the panic yourself. I've done that to debug hard to debug. The rust code for that is tricky but we'll documented.
Hopefully you don't need to do that, but it's an option.
b
I might just have to go manually start following the 3rdparty's source code 😕
Although I think I have a new theorey. The code likely uses 
~
as a path and the home for the daemon is not my home
Yup thats it. It's trying to read 
~/...
and 
HOME
isnt set on the daemon
So, now we understand why it's failing, time to think about fixes. Should the daemon have 
HOME
set? Seems like this certainly would match the behavior of 
--no-pantsd
https://github.com/pantsbuild/pants/pull/11641
Finally, we empty the environment for 
pantsd
runs to enforce explicit usage of `CompleteEnvironment`/`Environment`.
Yeah that's not a bad goal. This particular use comes from the intrinsic though 🤔
It looks like the env vars I might need here are 
HOME
and any 
AWS_*
env vars...
e
Can the intrinsic not take (portions of) the environment as an explicit parameter?
Basically, just like rules do for Processes
b
Yeah, I was trying to tease that out from the existing code. It might, but so far I haven't cracked it
MMmm thats a good trail of crumbs to follow
e
I've got to say I find this chat-whiteboarding befuddling, but I'm glad it works for you.
b
So looks like normal rules use 
EnvironmentVarsRequest
which I don't think the intrinsics can use. I would assume I could use 
CompleteEnvironmentVars
as an input, but wiring that up gives a GraphError (it also means each download node would depend on all env vars. Not fatal but maybe unfortunate)
Copy code
intrinsics.insert(
      Intrinsic {
        product: types.directory_digest,
        inputs: vec![
          DependencyKey::new(types.download_file),
          DependencyKey::new(types.complete_env_vars),
        ],
      },
      Box::new(download_file_to_digest),
    );
# Keep in sync with <http://intrinsics.rs|intrinsics.rs>
oh booooo
I just happened to stumble on that at random
Open in Slack
PreviousNext
Powered by