https://pantsbuild.org/ logo
Powered by
n

nutritious-monkey-78516

11/27/2018, 5:02 PM
So I am getting a “works fine on my machine” error. Our CI/CD is getting the following error while all of our workstations are working fine.
Copy code
pants.bin.pants_loader.InvalidLocaleError: System preferred encoding is `ANSI_X3.4-1968`, but `UTF-8` is required.
h

hundreds-father-404

11/27/2018, 5:13 PM
We added this check about two months ago to fix a Unicode issue someone reported getting, which we found was due to their environment not using UTF-8. I’m on my phone atm but can try to find the relevant env var once I get home It’s something external you’ll have to set, not a bug with Pants or Pants configuration
n

nutritious-monkey-78516

11/27/2018, 5:14 PM
Okay I’ll try to look as well thanks
a

aloof-angle-91616

11/27/2018, 6:12 PM
i'm fixing a bug in the intellij plugin related to this now -- i think checking the locale like this is probably the right idea because it turns up other places we aren't validating the encoding
PANTS_IGNORE_UNRECOGNIZED_ENCODING=1
should allow you to bypass this for now, otherwise setting 
LC_ALL
and 
LANG
env vars to 
en_US.UTF-8
is what the error message usually says
@nutritious-monkey-78516 which vars did you set? some shells will stop you from setting those env vars, i'm not sure if you ran into that here
h

hundreds-father-404

11/27/2018, 6:13 PM
We fix in our Dockerfile with 
ENV LC_ALL="en_US.UTF-8"
a

aloof-angle-91616

11/27/2018, 6:14 PM
the default bash on ubuntu won't let you set 
LC_ALL
and 
LANG
because it runs 
setlocale
under the hood and you have to do some ridiculous ubuntu nonsense to get it to just let you set the env var
so in docker testing i've been setting the unrecognized encoding var but i need to get more familiar with that -- orthogonal to pants though
h

hundreds-father-404

11/27/2018, 6:15 PM
Hm, @nutritious-monkey-78516 what are you using for CI?
n

nutritious-monkey-78516

11/27/2018, 6:16 PM
I am currently using Bitbucket Pipelines
h

hundreds-father-404

11/27/2018, 6:17 PM
n

nutritious-monkey-78516

11/27/2018, 6:18 PM
Let me take a look
I tried to input those variables into the environmental variables in the CI and it still didn’t work. I’m going to take a look at the document here and see if it gives me any other insights.
👍 2
Yeah I I don’t know how this is going to work. How do I pass the language encoding into pantsbuild? Also, I’m doing a Python Build
h

hundreds-father-404

11/27/2018, 6:23 PM
@chilly-painter-67686 these are the lines getting tripped up
Copy code
import locale

encoding = locale.getpreferredencoding()
if encoding.lower() != 'utf-8' and os.environ.get(cls.ENCODING_IGNORE_ENV_VAR, None) is None:
a

aloof-angle-91616

11/27/2018, 6:25 PM
(it might be a more effective error message to also mention that we specifically need the python 
locale
module to return 
utf-8
, so people can search the docs themselves to see how to satisfy that. the current message is great though)
👍 1
h

hundreds-father-404

11/27/2018, 6:26 PM
So, you can set the env variable 
PANTS_IGNORE_UNRECOGNIZED_ENCODING=true
to bypass this error, although we cannot guarantee things will work correctly because we expect UTF-8, or we have to figure out how to get Python std lib’s 
locale.getpreferredencoding()
to return 
utf-8
a

aloof-angle-91616

11/27/2018, 6:29 PM
i only jumped in because i happened to be responding to another instance of this error -- i think requiring the encoding is probably the right move and i think just some docs might help -- but i'll let you know after i fix that error
h

hundreds-father-404

11/27/2018, 6:31 PM
Fix the orthogonal IntelliJ error, right? This behavior Jonathon is reporting is 100% expected, although agreed perhaps we could have better docs
a

aloof-angle-91616

11/27/2018, 6:31 PM
yes! i was trying to explain how my concerns were confusing to have interjected into this thread, thanks
👌 1
n

nutritious-monkey-78516

11/27/2018, 6:36 PM
Yeah the error message was good
👍 1
and informative 🤯
🔥 2
a

aloof-angle-91616

11/27/2018, 6:37 PM
good error messages save lives imo
n

nutritious-monkey-78516

11/27/2018, 6:37 PM
So I think I’m going to have the pants ignore it
a

aloof-angle-91616

11/27/2018, 6:38 PM
eric can say for sure but i think that is fine for now if you accept that you might have mysterious encoding errors at some point (and if you just keep that in mind, you should be fine, because you can always try removing the env var if things break for no reason)
and if it's difficult to make things like bitbucket pipelines spit out utf-8 then there may be some tooling that would help -- that's what i was thinking more than docs
but not needed right now probably
h

hundreds-father-404

11/27/2018, 6:41 PM
Yeah if it was me, I would 100% ignore for now and only fix once it becomes an issue. Things might just work™ as is, we just can’t guarantee that. If it doesn’t work, then you can spend the time figuring out how to get Bitbucket to use UTF-8.
n

nutritious-monkey-78516

11/27/2018, 7:08 PM
Well, its now working thank goodness.
h

hundreds-father-404

11/27/2018, 7:17 PM
Working via the ignore, you mean?
n

nutritious-monkey-78516

11/27/2018, 7:34 PM
Yeah, I just did the ignore, and its working now. Poor colleagues had to see my building spam for quite a bit.
😂 1
🎉 1
h

hundreds-father-404

11/27/2018, 7:55 PM
Opened https://github.com/pantsbuild/pants/pull/6821 per this conversation. @nutritious-monkey-78516 let me know if there are further ways that we could improve the wording / make the error less confusing! I’d love for Pants error messages to become more human readable a la Elm’s error messages
Powered by