Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate Flash-Decoding into engine #181

Merged
merged 30 commits into from
Feb 12, 2024
Merged

Integrate Flash-Decoding into engine #181

merged 30 commits into from
Feb 12, 2024

Commits on Jan 29, 2024

  1. test stub

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    30e57a0 View commit details
    Browse the repository at this point in the history

  2. wip

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    6f3429a View commit details
    Browse the repository at this point in the history

  3. wip

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    97a4366 View commit details
    Browse the repository at this point in the history

  4. wip

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    7279cb6 View commit details
    Browse the repository at this point in the history

  5. compiled

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    7348f0e View commit details
    Browse the repository at this point in the history

  6. wip

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    b692376 View commit details
    Browse the repository at this point in the history

  7. fix

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    1df6cac View commit details
    Browse the repository at this point in the history

  8. fix

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    8c8872c View commit details
    Browse the repository at this point in the history

  9. wip, decode with flash decoding works

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    6a8272f View commit details
    Browse the repository at this point in the history

  10. all work

    @masahi
    masahi committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    487129c View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2024

  1. add paged_kv_cache_type option

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    8114197 View commit details
    Browse the repository at this point in the history

  2. read kv_type from artifact

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    2d6c81b View commit details
    Browse the repository at this point in the history

  3. black

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    67353b2 View commit details
    Browse the repository at this point in the history

  4. refactor attention backend

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    b9e41e1 View commit details
    Browse the repository at this point in the history

  5. minor clean up

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    910e31b View commit details
    Browse the repository at this point in the history

  6. Integrate flash-decoding into mlc-serve

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    ab910f2 View commit details
    Browse the repository at this point in the history

  7. remove --use-vllm-attention

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    4c8a75b View commit details
    Browse the repository at this point in the history

  8. wip decode_multi_query integration

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    00e1d09 View commit details
    Browse the repository at this point in the history

  9. temp handling for multi-query logits

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    5fbf671 View commit details
    Browse the repository at this point in the history

  10. remove tmp support for multi-query decode

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    2eff7b0 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    c51c2a4 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d7704e2 View commit details
    Browse the repository at this point in the history

  2. typo

    @masahi
    masahi committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    404b305 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    d87506c View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    99af3fb View commit details
    Browse the repository at this point in the history

  2. use block size 128 or 64 when possible

    @masahi
    masahi committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    a003965 View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2024

  1. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Feb 3, 2024
    Configuration menu
    Copy the full SHA
    780e244 View commit details
    Browse the repository at this point in the history

  2. remove unused var

    @masahi
    masahi committed Feb 3, 2024
    Configuration menu
    Copy the full SHA
    56d7a23 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. Merge branch 'batch-serving' into flash-decoding-engine

    @masahi
    masahi committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    1b976dc View commit details
    Browse the repository at this point in the history

  2. merge fix

    @masahi
    masahi committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a028c7d View commit details
    Browse the repository at this point in the history