test: Client-side input shape/element validation #7427
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
11 tasks
yinggeh
changed the title
11 tasks
rmccorm4
reviewed
Jul 25, 2024
rmccorm4
reviewed
Jul 25, 2024
yinggeh
force-pushed
the
yinggeh-DLIS-6657-client-input-byte-size-check
branch
from
f432d41
to
3863c39
Compare
yinggeh
force-pushed
the
yinggeh-DLIS-6657-client-input-byte-size-check
branch
from
17053e9
to
482409e
Compare
rmccorm4
changed the title
rmccorm4
reviewed
Aug 5, 2024
GuanLuo
reviewed
Aug 6, 2024
| } | ||
| EOL | ||
|
|
||
| cp -r $DATADIR/qa_model_repository/graphdef_object_int32_int32 models/. |
There was a problem hiding this comment.
Use model "simple_identity" instead to test string inputs.
Comment on lines
206
to
227
| if client_type == "http": | ||
| triton_client = tritonhttpclient.InferenceServerClient("localhost:8000") | ||
| else: | ||
| triton_client = tritongrpcclient.InferenceServerClient("localhost:8001") | ||
|
|
||
| # Example using BYTES input tensor with utf-8 encoded string that | ||
| # has an embedded null character. | ||
| null_chars_array = np.array( | ||
| ["he\x00llo".encode("utf-8") for i in range(16)], dtype=np.object_ | ||
| ) | ||
| null_char_data = null_chars_array.reshape([1, 16]) | ||
| identity_inference(triton_client, null_char_data, True) # Using binary data | ||
| identity_inference(triton_client, null_char_data, False) # Using JSON data | ||
|
|
||
| # Example using BYTES input tensor with 16 elements, where each | ||
| # element is a 4-byte binary blob with value 0x00010203. Can use | ||
| # dtype=np.bytes_ in this case. | ||
| bytes_data = [b"\x00\x01\x02\x03" for i in range(16)] | ||
| np_bytes_data = np.array(bytes_data, dtype=np.bytes_) | ||
| np_bytes_data = np_bytes_data.reshape([1, 16]) | ||
| identity_inference(triton_client, np_bytes_data, True) # Using binary data | ||
| identity_inference(triton_client, np_bytes_data, False) # Using JSON data |
There was a problem hiding this comment.
Copied from client/src/python/examples/simple_http_string_infer_client.py. It looks like the example demonstrated two ways of preparing string input data. I'll remove one of them.
rmccorm4
reviewed
Aug 6, 2024
| inputs[0].set_shape([2, 8]) | ||
| inputs[1].set_shape([2, 8]) | ||
|
|
||
| with self.assertRaises(InferenceServerException) as e: |
There was a problem hiding this comment.
Suggested change
| with self.assertRaises(InferenceServerException) as e: | |
| # If number of elements (volume) is correct but shape is wrong, the core will return an error. | |
| with self.assertRaises(InferenceServerException) as e: |
yinggeh
added
PR: test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Add client input size check to make sure input shape byte size matches input data byte size.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
triton-inference-server/client#742
Where should the reviewer start?
Should look at triton-inference-server/client#742 first.
Test plan:
n/a
17202351
Caveats:
Shared memory byte size checks for string inputs is not implemented.
Background
Stop malformed input request at client side before sending to the server.
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Relates to #7171