-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hide useless logs and add useful ones #31
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -41,36 +41,45 @@ then | |
echo "GCS directory was not specified as the second argument. Defaulting to ${GCS_DIR:?}." | ||
fi | ||
|
||
# Spin up the web server generating the report (and bg the process) | ||
PYTHONPATH=. /venv/bin/python3 report/web.py ${RESULTS_DIR:?} ${WEB_PORT:?} & | ||
mkdir results-report | ||
|
||
while true; do | ||
mkdir results-report | ||
cd results-report | ||
# Spin up the web server generating the report (and bg the process). | ||
PYTHONPATH=. /venv/bin/python3 report/web.py "${RESULTS_DIR:?}" "${WEB_PORT:?}" & | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can be for another PR: Can we replace the PYTHONPATH invocation here? Also one other thing, it's hard to use this script locally because the local venv path may be different. Can we instead just have this invoke There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yes, let me create another PR for this.
Yep, good point! |
||
pid_web=$! | ||
|
||
# Recursively get all the experiment results | ||
wget2 --inet4-only --no-host-directories --http2-request-window 10 --recursive localhost:${WEB_PORT:?}/ | ||
cd results-report || exit 1 | ||
|
||
# Also fetch the sorted line cov diff report | ||
wget2 --inet4-only localhost:${WEB_PORT:?}/sort -O sort.html | ||
# Recursively get all the experiment results. | ||
echo "Download results from localhost." | ||
wget2 --quiet --inet4-only --no-host-directories --http2-request-window 10 --recursive localhost:${WEB_PORT:?}/ 2>&1 | ||
|
||
# Upload the report to GCS | ||
gsutil -m -h "Content-Type:text/html" \ | ||
# Also fetch the sorted line cov diff report. | ||
wget2 --quiet --inet4-only localhost:${WEB_PORT:?}/sort -O sort.html 2>&1 | ||
|
||
# Stop the server. | ||
kill -9 "$pid_web" | ||
|
||
# Upload the report to GCS. | ||
echo "Uploading the report." | ||
gsutil -q -m -h "Content-Type:text/html" \ | ||
-h "Cache-Control:public, max-age=3600" \ | ||
cp -r . gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/${GCS_DIR:?} | ||
cp -r . "gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/${GCS_DIR:?}" | ||
|
||
cd .. | ||
|
||
# Upload the raw results into the same GCS directory | ||
gsutil -m cp -r ${RESULTS_DIR:?} \ | ||
gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/${GCS_DIR:?} | ||
echo "Uploading the raw results." | ||
gsutil -q -m cp -r "${RESULTS_DIR:?}" \ | ||
"gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/${GCS_DIR:?}" | ||
|
||
echo "See the published report at https://llm-exp.oss-fuzz.com/Result-reports/${GCS_DIR:?}/" | ||
if [[ -f /experiment_status ]] && [[ "$(cat /experiment_status)" == "1" ]]; then | ||
|
||
if [[ -f /experiment_ended ]]; then | ||
echo "Experiment finished." | ||
break | ||
else | ||
echo "Experiment is running..." | ||
sleep 600 | ||
exit | ||
fi | ||
|
||
echo "Experiment is running..." | ||
sleep 600 | ||
done |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we migrate to logging.info,debug etc in a future PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes!
Created #34.