Simulate, Test, Analyze: A Framework For Rigor


“Have Fun Storming The Castle!”

At the end of this fall semester, the second year students in the Academy rolled and carried their medieval mechanisms of mayhem to the SRHS track and we spent the afternoon watching the devices hurl lacrosse balls across the athletic field.This project was the final performance assessment of the semester and required that students design a gravitationally powered projectile launcher. This is an age old engineering/applied physics project.

Like many engineering projects done in high school, the physics principles governing the dynamics of the project are quite complicated, and ultimately the actual “application” of the science principles is often cursory. Students don’t have the background or mathematical abilities to to do the complex calculations needed to make an optimization adjustment to their mechanical device.This leads to the disconnection between the science content and engineering practice. Students don’t have the ability to make an informed decision about design choices. This is because it is difficult, very difficult.

Over the past few years I have been very interested in addressing this problem. This post discusses a framework that I have been working on to incorporate science into engineering projects. I think this framework allows high school students to engage in difficult scientific analysis without overwhelming them.

A Framework For Rigor

I won’t claim that this is a perfect solution, but so far I think we have experienced some success in creating a tighter relationship between science and engineering.  Last December I helped conduct a workshop at the NCCPA Professional Development Conference in Petaluma, CA. The name of the workshop was “NGSS, Prediction Reports and Your Science Class” and the point of this workshop was to give the attendees a framework for incorporating the Engineering standards into the science curriculum.  My co-presenter (Vipul Gupta) and I focused on the creation of prediction reports using computer simulations as a way to address two very important standards in the NGSS framework:

Using Simulations with Informed Input

Computer simulations are very popular in the educational space. They give teachers and students a virtual space where students can interact with virtual lab equipment or virtual objects that behave similarly to physical objects in the real world. With that said, they can fail to address students misconceptions because they do not always succeed in linking a conceptual model to the physical behavior. I also believe that the best simulations are ones that output data that can be analyzed with other scientific/mathematical tools. I also think that a good simulation requires that students provide meaningful input that gives them opportunities for analyzing the relationship between the input and the output.

Simulations used in engineering projects can be extremely helpful in addressing one of the main problems in engineering education. Students often design and build mechanical devices without understanding the physical principles that govern the design. The design process becomes an exercise in trial and error, or simply is reduced to copying a design from the internet.

To do a predictive analysis of a rocket’s flight, or a bridge’s structural performance is extremely difficult and often requires advanced mathematics and physics. Simulations can give the students the ability to analyze their designs and understand how changing the design inputs affects the output. Once again, it is important to find a simulation that requires students to understand the inputs and outputs.

Virtual Trebuchet

For example, in our project, students were introduced to an online Trebuchet simulation tool. This simulation tool is great because it requires that the student learn how to measure and calculate certain inputs. The students must have a working knowledge of rotational inertia, center of mass, and other concepts before they use the simulation. This was ideal for our project because it gave students a relevance and motivation . They had learn about these concepts in order to actually use the simulation. The students could then change certain inputs and see how that would change the efficiency of the design, or the range of the projectile. The point is that they needed physics knowledge in order to use the tool. They might not have the ability to know how the simulation eventually calculated the output, but they knew that the simulation required an understanding of the inputs.


Example Report

The Prediction Report

The next step is to ask the students to prepare a prediction report. This report is designed to get students to demonstrate their understanding of the inputs, display evidence of the required calculations or measurements needed to create the inputs and then analyze the simulation outputs. In the report for this project, I asked students to show a set of calculations and measurements for determining the center of mass of their throwing arm and the rotational inertia (moment of inertia). Students also had to provide similar information for the counterweight.  The students then had to run the simulation and document the outputs from the simulation.

The Test:  Data is Needed

The next step is to test the device. To make this step more rigorous and to be able to relate the scientific analytical process to the engineering process, it is crucial for the students to collect data that can be used to analyze the performance of their device/product and then reflect on how they would improve their design.

For this project, we decided to use high-speed video and Vernier’s LoggerPro video analysis software to plot the position of the projectile as it was launched from the device.

The Analysis

The analysis is actually broken into two parts. The first part requires a collection of calculations while the second part uses those calculations to make some qualitative assessments.

For example, in the above project, students had to use the collected position data from the video analysis tool to calculate the kinetic energy of the projectile and then the efficiency of the device. They had to be proficient at the analytical tool, which in itself requires physics content knowledge, providing once again an opportunity to apply scientific models in the analysis portion of this engineering project.

I have included the instructions for the analysis report here: Projectile Launcher Analysis Report.

Finally, students are given the opportunity to use the information gathered in the analysis report to reflect on their design, and more importantly use the information to inform how they would improve on a future design. I have included below the set of questions that I asked my students:

  1. Compare the efficiency calculation of the simulation to the efficiency rating that you calculated for your actual performance. Please describe why you think these values are not the same.
  2. Consider the design of your trigger. What design and fabrication decisions would you change in order to improve your trigger, AND explain WHY you would make those changes.
  3.  Consider the design of your sling. What design and fabrication decisions would you change in order to improve your sling, AND explain WHY you would make those changes.
  4. Consider the design of your release mechanism (called the nose). What design and fabrication decisions would you change in order to improve this mechanism, AND explain WHY you would make those changes.
  5. Consider the design of your arm. What design and fabrication decisions would you change in order to improve your arm, AND explain WHY you would make those changes.
  6. Consider the design of all other components and the overall design. What design and fabrication decisions would you change in order to improve your device (other than the trigger, sling and arm), AND explain WHY you would make those changes.


The overall design of this framework can be boiled down to this:

  • Engage students in a computer simulation that simplifies the process of modeling and analyzing a complex physical/chemical/biological process, but be sure that the simulation requires some conceptual and computational thinking.
  • When testing the performance of the design (bridge, rocket, etc.) make sure that the students are required to  collect data that can be analyzed and that once again demands that they apply their theoretical models.
  • Design an assessment that uses the analysis and gives the students an opportunity to make informed judgements of their designs for the purpose of redesign.