The 28th IEEE International Symposium on High-Performance Computer Architecture (HPCA-28) introduces for the first time in HPCA an artifact evaluation process for accepted papers. Artifact evaluation has become a common practice in the systems community (OSDI, PLDI, PACT, MLSys), and recently, computer architecture conferences such as ASPLOS (in 2020 and 2021) and MICRO (in 2021) have also successfully introduced an artifact evaluation process. We therefore invite the authors of accepted HPCA-28 papers to submit their artifacts to be assessed based on the Reproducibility Badging at IEEE.
Papers that successfully go through the Artifact Evaluation process will receive a set of badges of approval that will appear on the final version of the published manuscript. These papers can also include an Artifact Appendix to the final paper (up to 2 pages), which will be free of charge. Note that this submission is voluntary and will not influence the final decision regarding the papers.
Important Dates
Paper Decision Notification: Oct. 27, 2021
Artifact Registration Deadline: Nov. 11, 2021
Artifact Submission Deadline: Nov. 10, 2021 Extended to Nov. 14, 2021
Artifact Decision: Dec. 13, 2021
Submission Information
The authors of accepted papers at HPCA 2022 will be invited to submit their artifacts according to the established submission guidelines followed by previous conferences. The required fields for submission are: paper title, author list, artifact abstract, PDF of your paper with an appendix describing how to access and validate your artifacts, and conflicts of interests. The appendix should follow the format in the provided template. The appendix is expected to contain the following main sections: abstract, itemized meta-information list, access to the artifact, system requirements and dependencies, experiment workflow, steps for evaluation, and results. Your artifact MUST be available at a public archival repository. Note that the paper does not need to be the final (camera-ready) version, as the main goal of this submission is to let artifact reviewers reproduce your experiments.
Reviewing Process and Badges of Approval
Artifacts Submissions will be then reviewed according to the reviewing guidelines. Artifacts that that successfully pass the evaluation process will receive a some of the following badges of approval.
*** IMPORTANT ***
Papers must obtain the Open Research Objects (ORO) badge (i.e., be permanently archived in a public repository) in order to opt for the remaining badges.
Open Research Objects (ORO)
This badge signals that author-created digital objects used in the research (including data and code) are permanently archived in a public repository that assigns a global identifier and guarantees persistence, and are made available via standard open licenses that maximize artifact availability. For example, authors could use Code Ocean platform (official partner of IEEE Digital Library).
Research Objects Reviewed (ROR)
This badge signals that all relevant author-created digital objects used in the research (including data and code) were reviewed according to the criteria provided by the badge issuer.
Results Reproduced (ROR-R)
This badge signals that an additional step was taken or facilitated by the badge issuer (e.g., publisher, trusted third-party certifier) to certify that an independent party has regenerated computational results using the author-created research objects, methods, code, and conditions of analysis.
Feedback
Please do not hesitate to get in touch with the Artifact Evaluation co-chairs for any question or suggestion regarding the evaluation process.
Artifact Evaluation Committee
Artifact Evaluation Co-chairs
Tushar Krishna, Georgia Institute of Technology
Alberto Ros, University of Murcia
Artifact Evaluation Committee
Ajeya Naithani, Ghent University
Alejandro Nocua, ATOS
Anurag Kar, Georgia Institute of Technology
Chenhao Xie, Pacific Northwest Natl Labs
Eduardo José Gómez-Hernández, University of Murcia
Francisco Muñoz-Martínez, University of Murcia
Gokul Subramanian Ravi, University of Chicago
Gushu Li, University of California, Santa Barbara
Josue Feliu, Universidad de Murcia
Michael Resch, University of Minnesota
Raveesh Garg, Georgia Institute of Technology
Ricardo Fernández-Pascual, University of Murcia
Srikant Bharadwaj, AMD
Wei Tang, Princeton University
Xueyang Liu, Georgia Institute of Technology
Yuan Yao, Uppsala University