Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge from go #287

Merged
merged 5 commits into from
Sep 24, 2024
Merged

Merge from go #287

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
54ae911
changes for custom sample id generation - SDXL
anandhu-eng Sep 24, 2024
dc9bf99
changes for custom sample id generation - SDXL
anandhu-eng Sep 24, 2024
d8a33bc
fixed bug
anandhu-eng Sep 24, 2024
6490673
Merge pull request #129 from anandhu-eng/checksum-branch-2
arjunsuresh Sep 24, 2024
a9bfa16
Merge branch 'mlperf-inference' into mlperf-inference
arjunsuresh Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions script/app-mlperf-inference-mlcommons-python/customize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,8 @@ def get_run_cmd_reference(os_info, env, scenario_extra_options, mode_extra_optio
scenario_extra_options + mode_extra_options + \
" --output " + env['CM_MLPERF_OUTPUT_DIR'] + \
" --model-path " + env['CM_ML_MODEL_PATH']
if env.get('CM_COCO2014_SAMPLE_ID_PATH','') != '':
cmd += " --ids-path " + env['CM_COCO2014_SAMPLE_ID_PATH']

elif "llama2-70b" in env['CM_MODEL']:
env['RUN_DIR'] = os.path.join(env['CM_MLPERF_INFERENCE_SOURCE'], "language", "llama2-70b")
Expand Down
1 change: 1 addition & 0 deletions script/get-dataset-coco2014/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ new_env_keys:
- CM_DATASET_ANNOTATIONS_DIR_PATH
- CM_DATASET_ANNOTATIONS_FILE_PATH
- CM_CALIBRATION_DATASET_PATH
- CM_COCO2014_SAMPLE_ID_PATH

posthook_deps:
- enable_if_env:
Expand Down
3 changes: 3 additions & 0 deletions script/get-dataset-coco2014/customize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ def preprocess(i):

def postprocess(i):
env = i['env']
if env.get('CM_GENERATE_SAMPLE_ID', '') == "yes":
env['CM_COCO2014_SAMPLE_ID_PATH'] = os.path.join(os.getcwd(), 'install', 'sample_ids.txt')
print(env['CM_COCO2014_SAMPLE_ID_PATH'])
if env.get('CM_DATASET_CALIBRATION','') == "no":
env['CM_DATASET_PATH_ROOT'] = os.path.join(os.getcwd(), 'install')
#env['CM_DATASET_PATH'] = os.path.join(os.getcwd(), 'install', 'validation', 'data')
Expand Down
6 changes: 6 additions & 0 deletions script/get-dataset-coco2014/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ else
eval $cmd
test $? -eq 0 || exit 1
fi
if [[ ${CM_GENERATE_SAMPLE_ID} == "yes" ]]; then
cmd="python3 sample_ids.py --tsv-path ${INSTALL_DIR}/captions/captions.tsv --output-path ${INSTALL_DIR}/sample_ids.txt"
echo $cmd
eval $cmd
test $? -eq 0 || exit 1
fi
cd ${INSTALL_DIR}

test $? -eq 0 || exit 1
8 changes: 8 additions & 0 deletions script/run-mlperf-inference-app/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,12 @@ variations:
adr:
coco2014-preprocessed:
tags: _size.50
env:
CM_GENERATE_SAMPLE_ID: 'yes'
coco2014-dataset:
tags: _size.50
env:
CM_GENERATE_SAMPLE_ID: 'yes'
nvidia-preprocess-data:
extra_cache_tags: "scc24-base"
deps:
Expand All @@ -264,8 +268,12 @@ variations:
adr:
coco2014-preprocessed:
tags: _size.500
env:
CM_GENERATE_SAMPLE_ID: 'yes'
coco2014-dataset:
tags: _size.500
env:
CM_GENERATE_SAMPLE_ID: 'yes'
nvidia-preprocess-data:
extra_cache_tags: "scc24-main"
env:
Expand Down
Loading