#21163 query paramters ignored in http_source cache key Issue created by cburroughs The context here was trying to use S3 object versioning which looks something like a

versionId=Tx4Yxz3_aALevSHA5iesgg8vnZPK7Rkb

parameter on the url. Something like

?versionId=FAKE_TESTING

appeared to do nothing. With this handy test program running:

from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse, parse_qs
import hashlib

class RequestHandler(BaseHTTPRequestHandler):
    
    def do_GET(self):
        parsed_url = urlparse(self.path)
        query_params = parse_qs(parsed_url.query)
        
        self.send_response(200)

        if 'badger' in query_params:
            resp = '''
            Badger, badger, badger, badger
            Badger, badger, badger, badger
            Mushroom, mushroom
            '''.encode()
        else:
            resp = query_params['val'][0].encode()
        self.send_header('X-pants-sha256', hashlib.sha256(resp).hexdigest())
        self.send_header('X-pants-len', len(resp))
        self.end_headers()        
        print(query_params)

        self.wfile.write(resp)


host_name = 'localhost'
host_port = 8080
httpd = HTTPServer((host_name, host_port), RequestHandler)
print(f'Started HTTP server on {host_name}:{host_port}')
httpd.serve_forever()

And this BUILD file

file(
    name="hello",
    source=http_source(
        url='<http://localhost:8080/hello?val=world>',
        #url='<http://localhost:8080/hello?val=galaxy>',
        #url='<http://localhost:8080/hello?val=world&badger=true>'                
        
        sha256="486ea46224d1bb4fb680f34f7c9ad96a8f24ec88be73ea8e5a6c65260e9cb8a7",
        len=5,
    ),
)

run_shell_command(
    name="meow",
    execution_dependencies=[":hello"],
    command="cat {chroot}/hello",
)

Running:

$ pants --local-store-dir=/tmp/gh-issue  --no-pantsd run :meow
world

Now change the BUILD file to use

url='<http://localhost:8080/hello?val=galaxy>

$ pants --local-store-dir=/tmp/gh-issue  --no-pantsd run :meow
world

When instead the url should be fetched again and cause a digest error for the content/sha256 mismatch. I don't know Rust well but in my debugging I was suspicious of this line: pants/src/rust/engine/src/nodes/downloaded_file.rs Line 30 in </pantsbuild/pants/commit/7b2dcad1b9649b9005ba9683d1a0e09e29ef5127|7b2dcad>

fn url_key(url: &Url, digest: Digest) -> CacheKey {
        let observed_url = ObservedUrl {
            url: url.path().to_owned(),  // <---------
            observed_digest: Some(digest.into()),
        };
        CacheKey {
            key_type: CacheKeyType::Url.into(),
            digest: Some(Digest::of_bytes(&observed_url.to_bytes()).into()),
        }
    }

Versions

pants_version = "2.21.0"

While debugging this I also tripped over #21162 but I think they are distinct issues. pantsbuild/pants