PAEA Assessment Highlights from the 2022 Forum
PAEA’s commitment to developing high-quality exams is evident in the impact that our assessment products have on educational outcomes for programs and students. PACKRAT® is administered over 22,000 times a year, the PAEA End of Rotation™ exams are administered over 70,000 times a year, and the PAEA End of Curriculum™ exam has now been administered 15,000 times by PAEA member programs since the exam’s launch in January 2020.
While at the Forum this year, PAEA Assessment team members staffed an information booth and held a session, “PAEA Assessment 411,” to address member’s exam-related questions. Below, we’ve gathered a few highlights and FAQs from these events for members who couldn’t make it to San Diego last month.
PAEA PACKRAT, End of Rotation and End of Curriculum Exams
The Assessment 411 presentation provided several important updates for program faculty and exam administrators, including a review of the intent behind each of PAEA’s exam offerings and a student self-assessment tool. Dr. Kim Cavanagh stressed the importance of administering exams only when students have reached the educational milestones at which the exams were designed to be administered:
Q: Can I use End of Rotation exam forms as practice exams for my students?
A: No. It is important that programs only give the exam in accordance with their purpose to maintain the validity of the assessment data we provide to all PA programs. They are not practice exam for students. Nor is it recommended to use the End of Rotation exams as part of the summative evaluation.
End of Rotation Proctoring Policies
PAEA temporarily waived our proctoring policies for the End of Rotation exams effective 3/23/2020 and those policies remain waived today. This interim decision to waive the requirement for high-security remote proctoring was made to support programs as they navigated exam administration during COVID-19.
Given this policy waiver, we have been extremely vigilant to monitor for any unexplained changes in student test taker scores. PAEA reviewed overall performance metrics for each of our exams and compared results from the last three years, including results from one pre-COVID year. PAEA can say with confidence that the End of Rotation scores have remained secure and valid throughout this period even as some programs have elected to administer End of Rotation exams at home and/or unproctored. PAEA will continue to monitor overall performance on each exam.
Q: When will PAEA resume their proctoring policies for End of Rotation exams?
A: PAEA is sensitive that programs need time to plan for and adjust their local exam administration policies and procedures. We will keep programs updated on proctoring policy changes and new requirements as they become available.
New ‘Surgery’ End of Rotation Exam Blueprint Update
PAEA is in the process of developing a new Surgery End of Rotation exam. Some of the key changes that members can expect to see include:
- A professional practice task area, replacing health maintenance.
- Three new content areas, including breast surgery, pain medicine/anesthesia, and trauma/acute care, with pre-, post-, and intraoperative settings added as cross-cutting targets.
- Adjustments to distributions by task area, with the biggest change being a reduction in the overall number of questions from the gastrointestinal/nutritional content area.
Q: Will the General Surgery End of Rotation exam still be available?
A: Yes, the General Surgery End of Rotation exam will remain available for a period after we publish the new Surgery End of Rotation exam. We are sensitive that programs need time to plan to transition from one exam to the other.
Assessment Transition Update
This July, PAEA Assessment announced a delay in the launch of our next-generation exam platform. PAEA believes it is important to get the new exam platform and the transition right. We are committed to introducing the new platform once it meets our high standards for a seamless transition and outstanding user experience. Once we have a firm updated timeline, we will notify members along with providing clear next steps for the transition. We are sensitive that programs need time to train on the new system and adjust local policy and procedure. We will keep you updated on the revised timeline as it becomes available. You can also check the Assessment Transition Center on our website for updates.
For now, programs can continue to schedule and administer exams through ExamDriver and all current features and capabilities will remain the same.
Q: Will there be any training sessions on how to use the new platform?
A: Yes. PAEA Assessment is committed to providing member education on the new platform through Networker articles, published resources such as videos and checklists, live and recorded webinars, and one-on-one member support.
Q: Will the exam content change?
A: No. PAEA’s exams will not change. Our exams will continue to be developed by PA educators for PA students, as they always have been. Through the transition and beyond, PAEA will continue to provide superior exams that are validated and follow rigorous development standards to continue developing the same high quality assessment tools that our members rely on.
Q: Will the score reports change?
A: No, the score report content will still be the same, although the score report appearance will change.
Q: Will our score reports from ExamDriver remain available?
A: Yes. Score reports from ExamDriver will be stored in a file in the new Assessment Hub to ensure faculty have ongoing access.
Thank you to all our members who attended “PAEA Assessment 411” and asked questions about new developments for our suite of assessment products at the Forum. Whether you come virtually or in-person, we look forward to meeting more members at the 2023 PAEA Forum in New Orleans! In the meantime, we hope you can join us for one of our upcoming virtual Assessment Showcase Webinars in 2023:
