Submission.md

[ Back to MLPerf inference benchmark index ]

Generate actual submission tree

You should use the master branch of MLCommons inference repo for the submission checker:

cmr "generate inference submission" \
--clean \
--preprocess_submission=yes \
--run-checker \
--submitter=CTuning \
--tar=yes \
--env.CM_TAR_OUTFILE=submission.tar.gz \
--division=open \
--category=edge \
--env.CM_DETERMINE_MEMORY_CONFIGURATION=yes \
--quiet

• Use --division=closed to generate a closed division result
• Use --category=datacenter to generate results for datacenter category
• Use --hw_notes_extra option to add your name to the notes like --hw_notes_extra="Result taken by NAME"
• Use --hw_name="My system name" to give a meaningful system name. Examples can be seen here
• Use --submitter=<Your name> if your organization is an official MLCommons member and would like to submit under your organization

The above command should generate "submission.tar.gz" if there are no submission checker issues and you can upload it to the

Upload your results

Push the results to GitHub repo (Only if you are submitting public results under cTuning)

1. First, create a fork of this repo.
2. If you have not set up GIT config already please do

   git config --global user.name "[YOUR NAME]"
   git config --global user.email "[YOUR EMAIL]"
   

3. Then run the following command after replacing --repo_url with your fork URL.

cmr "push github mlperf inference submission" \
--repo_url=https://github.com/ctuning/mlperf_inference_submissions_v4.0 \
--commit_message="Results on <HW name> added by <Name>" \
--quiet

Create a PR to the cTuning repo

Upload the tarball to MLPerf UI (Only if your organization is an official MLCommons member)

You can upload the submission.tar.gz file generated by the previous command to the submission UI.

Questions? Suggestions?

Check the MLCommons Task Force on Automation and Reproducibility and get in touch via public Discord server.