Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORE-2157: cloud_storage_clients: add support for path-style addressing #17806

Merged
merged 7 commits into from
Apr 24, 2024

Commits on Apr 17, 2024

  1. cloud_storage_clients: add s3_url_style

    Redpanda currently supports only virtual-hosted style URLs
    for use with S3.
    
    Add `s3_url_style` to `cloud_storage_clients/types.h" to allow for
    future work towards supporting path-style URLs.
    WillemKauf committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    9f5c3e8 View commit details
    Browse the repository at this point in the history
  2. cloud_storage_clients: use s3_url_style for requests

    To support path-style requests in addition to the currently supported
    virtual-host style, 's3_url_style' has been added to 's3_configuration',
    as well as 'request_creator'.
    
    New methods 'make_host()' and 'make_target()' in 'request_creator'
    will generate host and target strings according to the url style set
    for 's3_client' requests (currently defaulted to virtual-host).
    WillemKauf committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    7218f75 View commit details
    Browse the repository at this point in the history
  3. config: add s3_url_style to configuration options

    Allow user to set s3_url_style in their redpanda configuration
    file. New setting 'cloud_storage_url_style' can be set to
    'virtual_host' or 'path'.
    WillemKauf committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    df5b47f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7ad5b02 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. cloud_storage_clients: add url_style option to s3_test_client

    Added the option for `url_style` to `s3_test_client_main.cc`,
    allowing for testing of both virtual-style and path-style requests.
    
    Also make sure the `url_style` is initialized correctly everywhere
    the `s3_configuration` object is created.
    WillemKauf committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    080ae70 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2c8c094 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. rptest: add addressing style configuration to ducktape

    The addressing style used by `redpanda` for ducktape tests can now
    be set with the variable `cloud_storage_url_style` in `SISettings`,
    or injected into a test similar to `cloud_storage_type` with a decorator.
    
    e.g: `@matrix(cloud_storage_url_style=['path','virtual_host'])`
    WillemKauf committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    5e9ac31 View commit details
    Browse the repository at this point in the history