downstream we have a bunch of HDFS thrift stuff that has the prior structure baked in, tho we can probably just parse upstream and continue using that in the backend