Ergonomic, hermetic, user-friendly software build system for Python, Go, Java, Kotlin, Protobuf, Scala, Shell. Pants lets you fearlessly scale up your codebase!

Pants

yea, can do it in `Service.setup` i think.

surprisingly pickling performs the fastest, time taken pickle &lt; msgpack &lt; json, each is about 20% slower for the same amount data

wonder if it's to do our data is many small objects

i am comparing just the time spent in those libraries, excluding to prepare them so should be fair

doing compression takes about the same time to serialize, compression rate is about 50%, no much effect on time to compute digest though

which means for in memory storage (i.e, not storing serialized objects, only for digest purpose), compression is not useful

to validate that hypothesis of why pickling takes less time i will do another experiment on larger objects

doesn’t pickle do some form of memoization

wonder if thats true for the other serializers

fwiw, we turned off pickle memorization because it's based on `id` see <https://github.com/pantsbuild/pants/commit/e6eac05f83be71ece5d89b7630c7cc02d95bdf94>

(...because peiyu spent hours and hours trying to figure out why two objects weren't equal)

```Nice find! Please fix the spelling everywhere though: this is "memoization"```

<@U098ZNTCN> : ok, not to throw a monkey wrench. but. one other way to think about `Keys` (based on your discovery that we only turn a few different things into Subjects,  and thus Keys) would be to formalize them in some other way

...but. i don't know. i guess the best that could possibly do would be explicitly hashing the individual fields of the objects

and pickling is probably going to do that faster

are we talking about just for performance?

and just the field values in order should be sufficient, + class itself,  field names are implied by class, but that's probably a small portion

<https://mail.python.org/pipermail/speed/2016-November/000472.html> pants should be on the side that will benefit from python3.6 in performance