Evaluation: Cognitive Skills Practice

A group interaction activity within new hire training

Audience

Current/experienced flight attendants, attending recurrent training in person.

Responsibilities

Project management
- SME coordination
- Stakeholder communication coordination
- Quality control
Instructional design & content development
- Training needs analysis
- Instructor-led training
- Script writing, audio narration, graphic design and video production
- Brand compliance and template creation

Toolkit

DJI Gimbal
Final Cut Pro
iPhone camera & tripod
Lighting kit
Microsoft Office 365 (PowerPoint and Word)
Mint—Training Management System
SmartSheets

The problem

Senior leadership requested a way to gauge a flight attendant's critical thinking and decision-making skills in an emergency. A cognitive skills validation (CSV) was in place for the past two years as a jeopardy event. I made the recommendation to convert the validation into a practice, as we did not mandate a specific procedure for employees to critically think and make decisions. Alongside the suggestion, I also proposed to audit the grading/standards procedures for the event. We still felt the data was important to help improve the overall training program around the subjects of critical thinking and decision-making.

The solution

The decision was to convert the event into a non-jeopardy practice, renaming the evaluation to cognitive skills practice (CSP). We also updated the grading policy from a pass/fail on each of the tasks to a 3 level rubric grading (e.g., (1) The employee's action in this task was below expectations—possibly leading to an incident, (2) The employee's action in the task met expectations—would not lead to an incident, (3) The employee's action in the task was above expectations—would not lead to an incident). The grading rubric provided the standards to the evaluator for each grade category.

The process

This event was part of the larger yearly continuous qualification training (CQT) for the workgroup. To ensure alignment, I followed the project's project plan based on the Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model in SmartSheets.

Analyze

Met with Project Sponsors and SMEs to understand the problem
Partnered with internal analysis team to review safety and training reports
Partnered with internal safety teams
Reviewed current Job Task Analysis (JTA) and position expectations (PE)
Reviewed current evaluation materials

Design

Partnered with leadership and stakeholders to develop the:

Project plan and task tracker in SmartSheets
Visual mockups, outlines, and storyboards for emergency scenarios
Instructor Guide and Presentation template (Word and PPT)

Partnered with internal analysis team to create and propose a grading rubric to the client

Then worked with the stakeholders to define the standards within the rubric

Develop

Instructor Guide, Presentation, Grading Rubric, and Online Form

I partnered with the client and our internal analysis team to redesign the non-jeopardy evaluation
The scenario was based on real events and vetted through SMEs to ensure fidelity
This included storytelling through video clips embedded in PowerPoint for the student to follow along and answer questions
The instructor guide contained scripts and alternative scripts for the evaluator to follow alongside the grading rubric with definitions
I partnered with the internal analysis and systems teams to create the online grading form that would be used on an iPad.

Video production

Once the scenario was decided, I created a storyboard that was presented to the client for review
After implementing their feedback, I created a script and a production checklist/timeline document that was reviewed as well
Collaborated with the client to schedule the use of a cabin simulator and employees to act in the video
- Also consulted with the team to ensure brand and uniform compliance
Filmed using an iPhone 15 Pro, tripod with iPhone attachment, lighting, and DJI Gimbal for close-up scenes
The clips were edited using Final Cut Pro and embedded into PowerPoint and were backed up to a private Vimeo account and MS SharePoint

Implement

The training materials created were included in the larger projects Alpha, Beta, Gamma, and Train-the-Trainer (TTT) sessions
Alpha, Beta, Gamma, and TTT sessions included online feedback forms for participants to submit to help enhance the program
Partnered with workgroup training, scheduling, and support systems teams to support the launch of the program

Evaluate

Collaborated with Stakeholders and the internal analysis team to create a plan to evaluate the program in the future by conducting periodic analysis of the completed evaluations from the classroom and OJT, as well as participant and instructor surveys
At the end of the project, we conducted a review of what worked well and opportunities for improvement

What I learned

This was a fantastic opportunity to expand my skills in cognitive testing and video production. I researched critical thinking and decision-making for emergency situations, learning there were few sources for front-line employees, so I had to synthesize the information. Finally, collaborating with experts in the organizational psychology field from the internal analysis team helped enhance my knowledge in structuring evaluations.

Explore my other projects

Boston-based, globally ready!

Parker@DesignedByParker.com

Page updated

Google Sites

Report abuse