pantsbuild #development

Channels

welcome

github-notifications

development

rough-minister-58256

10/20/2016, 1:37 AM

https://blog.sentry.io/2016/10/19/fixing-python-performance-with-rust.html

👍 1

rough-minister-58256

10/20/2016, 1:40 AM

Handling a 30MB source map makes a single Python process expand to ~800MB in memory, executing millions of memory allocations and keeping the garbage collector very busy with tokens’ short-lived nature.

rough-minister-58256

10/20/2016, 1:42 AM

fairly sure these guys are nearby, maybe we should invite them over for lunch

rough-minister-58256

10/20/2016, 1:42 AM

met a couple of them at pybay

witty-crayon-22786

10/20/2016, 5:36 PM

HN darling

witty-crayon-22786

10/20/2016, 5:36 PM

even more of a selling point perhaps: http://blog.faraday.io/saved-by-the-compiler-parallelizing-a-loop-with-rust-and-rayon/

fancy-queen-20734

10/20/2016, 8:36 PM

Figured out the mystifying bug—ProductGraph.Entry’s dependencies are Entry’s, but Entry.cyclic_dependencies are Nodes. So you can’t compare

fancy-queen-20734

10/20/2016, 8:37 PM

I’d read the comment on cyclic_deps, but didn’t realize the implications else where

fancy-queen-20734

10/20/2016, 8:37 PM

🍠

witty-crayon-22786

10/20/2016, 9:03 PM

witty-crayon-22786

10/27/2016, 8:23 PM

@fancy-queen-20734 : hey: so given yesterday's discussion, it seems clear that we should try to land https://rbcommons.com/s/twitter/r/4270/ asap

witty-crayon-22786

10/27/2016, 8:24 PM

i think that that means focusing very heavily on polish for the graph generation patch might not be worthwhile at this juncture. i'm going to do my best to get it in soon

witty-crayon-22786

10/27/2016, 8:24 PM

ie, this weekend if i can

witty-crayon-22786

10/27/2016, 8:24 PM

but... yea. sorry.

witty-crayon-22786

10/27/2016, 8:24 PM

that'll be quite a rebase

fancy-queen-20734

10/27/2016, 8:24 PM

Cool. I’ve done further looking over it, but at this point I think it makes more sense to build on top anyway

witty-crayon-22786

10/27/2016, 8:25 PM

@fancy-queen-20734 : i'm going to post the rebase to master today

fancy-queen-20734

10/27/2016, 8:25 PM

nice!

witty-crayon-22786

10/27/2016, 8:25 PM

that will likely be the place to start

witty-crayon-22786

10/27/2016, 8:35 PM

@fancy-queen-20734 : hey, so we currently index a task by both the outer constraint and the inner types: https://github.com/pantsbuild/pants/blob/master/src/python/pants/engine/rules.py#L205-L209

witty-crayon-22786

10/27/2016, 8:37 PM

i'd like to drop the indexing by constraint, and just do a lookup for each inner type (rather than expecting an index by constraint)... the index by constraint is weird, because it seems to act like its own unique type, which ideally it should not

witty-crayon-22786

10/27/2016, 8:37 PM

(focused on this because i'd like to remove heterogenity from these indexes to port them)

witty-crayon-22786

10/27/2016, 8:37 PM

does that sound feasible? or am i missing something

fancy-queen-20734

10/27/2016, 8:43 PM

I think that’s reasonable

witty-crayon-22786

10/27/2016, 8:44 PM

ok, thanks

fancy-queen-20734

10/27/2016, 8:45 PM

I think I did it that way to avoid weird branching before lookups. But if we normalized all output types to unions, then we get the same effect

witty-crayon-22786

10/27/2016, 8:46 PM

i don't think so, because you can't do a hash lookup for a union

witty-crayon-22786

10/27/2016, 8:46 PM

(without doing an exact match of the union)

fancy-queen-20734

10/27/2016, 8:47 PM

oh, the other bit was that I wanted to be able to have (<types-from-parsing>, ….) and (…, (Select(<types-from-parsing>),...)

fancy-queen-20734

10/27/2016, 8:47 PM

line up