Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨[Feature] Automatically remove output repacking for convert_method_to_trt_engine calls #1938

Closed
gs-olive opened this issue May 19, 2023 · 0 comments · Fixed by #1945
Closed
Assignees
Labels
feature request New feature or request

Comments

@gs-olive
Copy link
Collaborator

Context

For fully-supported models having multiple output Tensors, Torch often appends a Tuple/List packing function call to the scripted IR. This can cause convert_method_to_trt_engine, which would otherwise succeed due to converter coverage, to fail.

Details

This proposed feature would automatically detect and remove unnecessary output packing schemes, such as prim::TupleConstruct and prim::ListConstruct, which are applied to the outputs of existing TRT engines, thereby allowing more models to be fully converted to their TRT engine counterparts directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
1 participant