fs.comet.libhdfs.schemes is matched after the s3a alias rewrite, so listing s3 also routes s3a through libhdfs

[andygrove]

Describe the bug

fs.comet.libhdfs.schemes is compared against the scheme after the s3-alias rewrite, so listing s3 also routes s3a:// reads through libhdfs. Before #5314 the two were independent: the libhdfs decision was taken from the URL the user typed, and s3a was normalized to s3 only when it was not libhdfs-routed.

prepare_object_store_with_configs in native/core/src/parquet/parquet_support.rs now reads:

let url = normalize_object_store_url(url.as_str(), object_store_configs)?;
let is_hdfs_scheme = is_hdfs_scheme(&url, object_store_configs);

normalize_object_store_url guards its own early return with is_hdfs_scheme, so an s3a:// URL whose scheme is not in the list falls through to rewrite_alias_to_s3 and comes back as s3://. The classification on the next line then sees s3, matches the list, and selects the Hadoop backend.

Before #5314 the order was the other way round:

let is_hdfs_scheme = is_hdfs_scheme(&url, object_store_configs);
let mut scheme = url.scheme();
if !is_hdfs_scheme && scheme == "s3a" { scheme = "s3"; url.set_scheme("s3")?; }

Steps to reproduce

With fs.comet.libhdfs.schemes=s3 set and no s3a entry, read s3a://bucket/file.parquet. The scan is served by create_hdfs_object_store instead of the native S3 store.

Adding this to native/core/src/parquet/objectstore/s3_blob_fs_support.rs on 424c31aa7 fails:

#[test]
fn s3a_is_not_libhdfs_routed_when_only_s3_is_listed() {
    let configs = HashMap::from([(
        "fs.comet.libhdfs.schemes".to_string(),
        "s3".to_string(),
    )]);
    let typed = Url::parse("s3a://bucket/f.parquet").unwrap();
    assert!(!is_hdfs_scheme(&typed, &configs));
    let normalized = normalize_object_store_url("s3a://bucket/f.parquet", &configs).unwrap();
    assert!(!is_hdfs_scheme(&normalized, &configs));
}

Output:

PROBE typed scheme      = s3a
PROBE typed  is_hdfs    = false
PROBE normalized scheme = s3
PROBE final  is_hdfs    = true

The converse case is fine: with fs.comet.libhdfs.schemes=s3a, normalize_object_store_url returns early and the scheme keeps its spelling, so s3a stays libhdfs-routed as asked.

Expected behavior

The libhdfs decision should be taken once, from the scheme the user wrote, and carried through the alias rewrite rather than being recomputed from the rewritten URL.

Additional context

Found while reviewing #5503, whose isolates_backends_even_when_s3_alias_and_configs_match test covers exactly this pair (s3a native versus s3 libhdfs under one config) and fails in CI on the merge with main while passing on the branch alone. That test is a good regression guard for this fix.

Introduced by #5314. cc @comphead @sunchao

[comphead]

Thanks @andygrove I'm on it