Skip to content

Can we trim the storage scheme after parsing? #1411

Closed as not planned
Closed as not planned

Description

@Xuanwo

One question I have is whether we really need to keep the original abfss scheme. I don't recall any situation where we need to use it, perhaps only when creating a table? I wonder if we could simply remove it after parsing the input path.

Originally posted by @Xuanwo in #1368 (comment)

Activity

  1. DerGut commented on Jun 5, 2025

    @DerGut
    Contributor

    Right now, it's only used to validate the passed path's scheme against the scheme of the endpoint that might have been used to configure the FileIO.
    We don't really distinguish between wasb[s] and abfs[s], only between their TLS and non-TLS versions.

    If we say that a user should also be able use a TLS endpoint via abfss:// even though they might have passed a plain text http://account.dfs.core.windows.net to the FileIO, then we can drop it!

  2. DerGut commented on Jun 5, 2025

    @DerGut
    Contributor

    I see a little bit of value in forcing all requests to use TLS even if some path may specify the plain text variant. Especially when users use SAS token-based auth.

  3. Xuanwo commented on Jun 5, 2025

    @Xuanwo
    MemberAuthor

    I see a little bit of value in forcing all requests to use TLS even if some path may specify the plain text variant. Especially when users use SAS token-based auth.

    Aha 😆 , the biggest blocker are from users of minio and azurite.

  4. DerGut commented on Jun 5, 2025

    @DerGut
    Contributor

    the biggest blocker are from users of minio and azurite

    Yeah, as far as I can see, that's the main value of allowing plain text at all. But IIUC neither Azurite, nor Minio support the ADLS APIs 🤔

    Browsing the Azure console, I can only force TLS, but not configure the storage account to only allow plain text. So local testing maybe is the only reason to support plain text.

    I guess it still makes sense to support it in case someone builds their own local emulation...

  5. mrcnc commented on Jun 5, 2025

    @mrcnc

    One question I have is whether we really need to keep the original abfss scheme

    Wouldn't we need to keep this to support existing tables that have files with this scheme?

  6. liurenjie1024 commented on Jun 6, 2025

    @liurenjie1024
    Contributor

    I think it's valuable to keep it to validate against input, for example when the FileIO was created with hdfs, while the user trying to create an "s3" file with it. It may not be a problem if we have finished #1314

  7. deleted a comment from liurenjie1024 on Sep 16, 2025
  8. github-actions commented on Mar 16, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

  9. github-actions commented on Sep 13, 2026

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

  10. github-actions commented on Sep 27, 2026

    @github-actions
    Contributor

    This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions