#general
- b
bored-art-40741
02/26/2016, 12:34 AMIssues with our artifact cache - b
bored-art-40741
02/26/2016, 12:34 AMWorth noting that @happy-morning-1599 is working on a much more robust generic artifact cache solution using HDFS as the backing store - g
gentle-potato-32845
02/26/2016, 12:35 AMwoooah - g
gentle-potato-32845
02/26/2016, 12:35 AMthat sounds đŸ’£ - b
bored-art-40741
02/26/2016, 12:38 AMIndeed - g
gentle-potato-32845
02/26/2016, 12:38 AMare there docs for how to set up a remote artifact cache? my first pass through it was hard to tell. - b
bored-art-40741
02/26/2016, 12:40 AMI think there was a doc floating around that we failed to find last time we looked - b
bored-art-40741
02/26/2016, 12:42 AMThe gist of it is that it's a simple REST interface, and we liked Varnish in front of nginx. But we've outgrown that now, and we're working on something that's a hash-sharded HAProxy or nginx reverse proxying to HAProxy in front of varnish in front of HAProxy in front of HttpFS in front of HDFS. And also there's another nginx between varnish and the backend HAProxy to handle rewriting pants' REST requests to HttpFS requests - b
bored-art-40741
02/26/2016, 12:43 AMI think we're pretty close to finished now--we'll share the doc and node configs when we're confident it works - w
witty-crayon-22786
02/26/2016, 12:43 AM@bored-art-40741: you had "just a varnish config" at some point, right? - w
witty-crayon-22786
02/26/2016, 12:43 AMor was it always varnish+nginx - b
bored-art-40741
02/26/2016, 12:44 AMIt was always varnish as the cache and nginx as the "backing store" (both on the same machine) - w
witty-crayon-22786
02/26/2016, 12:45 AMmm - b
bored-art-40741
02/26/2016, 12:45 AMNow we're finding ourselves NIC-bound on a single-machine setup rather frequently, so we're scaling it out so that we can just add more nodes if we're not keeping up with capacity - w
witty-crayon-22786
02/26/2016, 12:45 AMours is a finagle app - b
bored-art-40741
02/26/2016, 12:45 AMAnd we're nowhere near the throughput of HDFS/HttpFS, so it's a natural candidate for backing store - w
witty-crayon-22786
02/26/2016, 12:46 AMbut it's backed by our internal blobstore, which isn't generic =/ - w
witty-crayon-22786
02/26/2016, 12:46 AM@bored-art-40741: hm, i don't really think so... - w
witty-crayon-22786
02/26/2016, 12:46 AMHDFS hates small files? - r
rough-minister-58256
02/26/2016, 12:46 AMyeh that’d be my concern too - w
witty-crayon-22786
02/26/2016, 12:46 AMand lots of files in particular - b
bored-art-40741
02/26/2016, 12:46 AMThat's true, but we don't have enough of those for it to be a large concern for me - w
witty-crayon-22786
02/26/2016, 12:47 AMhm. yea, i'd still caution against that. - b
bored-art-40741
02/26/2016, 12:47 AMAnd remember it's fronted by varnish, so the working set is almost entirely in memory - w
witty-crayon-22786
02/26/2016, 12:47 AMthe namenode needs the entire filelist in memory - w
witty-crayon-22786
02/26/2016, 12:47 AM...still. 10 years later. - b
bored-art-40741
02/26/2016, 12:47 AMYou'll hit these files twice in practice from HttpFs at most - b
bored-art-40741
02/26/2016, 12:47 AMHm - b
bored-art-40741
02/26/2016, 12:47 AMI'll keep that in mind, thanks for the heads up - b
bored-art-40741
02/26/2016, 12:48 AMI'd much prefer to be using the blob store that I dream about every night, but I'm not so lucky