past Lilly Fellow of Engineering
Use of Active Learning to Address ABET Course Learning Objectives in a Large
Undergraduate Environmental Engineering Class
M. Cupples, Weimin Sun, Susan Masten
The overall aim was to determine if in class activities
based on active learning improve the achievement of Accreditation Board for
Engineering & Technology (ABET) course learning objectives (CLOs) and overall
student learning. The study was performed (Fall semester of 2011) in a large
(~80 student) undergraduate introductory environmental engineering course (CE
280, Principles of Environmental Engineering).
The study was initiated because previous ABET data collected from CE 280
(Fall 2009) indicated a low level of CLO achievement. The approach was simple
and involved between 2 and 4 in class exercises or short questions in every
lecture session. The students responded to the questions using a response pad
(resulting in 5% of their final grade). The questions were all multiple choice
and were either qualitative questions or short quantitative questions. The
questions were typically at the beginning, then after 15, 35 minutes and 50
minutes. Three sets of data were collected to determine the efficiency of this
approach. Of these, two data sets were compared between 2009 and 2011. The
first assessment method concerned a comparison of Course Learning Objective
Evaluation (or CLOSE) forms. The second data set involved pre- and post-
surveys in 2011. The third set involved a comparison of student performance in
questions relating to each CLO. The CLOSE forms illustrated that the students
believed they had a high level of understanding of each CLO in 2009 at the end
of the semester and this did not change in 2011. The pre- and post- surveys
indicated the students recognized the value of in class assessment as well as
the use of response pads. The post- survey showed a statistically significant
more positive response to considering the CLOs when studying. The final assessment
method concerned an analysis of student performance on questions relating to
each CLO. In 2009 over five CLOs were not achieved, however in 2011, only 1 CLO
was not achieved. These data indicate the introduced pedagogical approach was
highly successful at improving student learning of CLO content.
In Fall 2009, ABET data were collected to determine if the course
learning objectives (CLOs) in a large, required, undergraduate class were being
achieved. The class, Principles of Environmental Engineering and Science (or CE
280), meets three times per week for 50 minutes and typically contains ~80
students. From the data collected, it was clear that the students were having
difficulty with a significant number of the topics included in this class. To
address these shortcomings, a study was designed to include in-class, active
learning and assessment activities to improve the teaching and learning of CE 280
content. In 2009, CE 280 contained 19 CLOs, however, this number was reduced to
15 in 2011 (Table 1). The CLOs that were similar between 2009 and 2011 were
compared in this study (CLOs 1-14 in table 1). Dr. Cupples was the instructor
in both 2009 and 2011.
1. A list of the 2011 CLOs for CE 280
Define and describe the primary environmental
regulations, and discuss how regulations affect engineering practice.
material balances to determine the concentration of water contaminants.
Perform elementary chemical calculations.
the four step process of risk assessment.
Solve hydrologic mass balance problems.
examples of biological and chemical substances that cause water pollution.
Provide examples of the human health effects
simple BOD calculations.
Calculate dissolved oxygen concentrations in a
the basic engineering approaches to drinking water treatment.
Describe the basic engineering approaches to
the regulated air pollutants and their sources.
Describe the impact of regulated air
pollutants on the environment.
the key characteristics of a hazardous waste and provide examples of
hazardous waste treatment technologies.
key aspects of sustainability relating to environmental engineering and
* CLO 15 was not included in the study because it was not
used in 2009.
Starting in Fall 2011, between 2 and 4 in class exercises
were included in every class session. The students were asked a question and
were required to answer the question using their response pad (classroom
performance system or CPS by eInstructionTM). The first homework
assignment was to buy and activate their response pad. The answers they
provided using their response pad contributed a small percentage (5%) to their
final grade. The questions posed to the students were both qualitative
questions (to highlight important concepts) as well as short, quantitative
questions. The questions were both designed to target material relating to the
CLOs as well as additional concepts/problems discussed in class. The questions
were all in the multiple-choice format and required only 1-4 minutes to
complete the answer. When longer quantitative problems were targeted (e.g. mass
balance examples) only a portion of the question was given to the student, and
the instructor completed the rest. Questions were often included at the
beginning and end of a lecture period for review purposes. Also, in general,
questions were typically inserted after 15 and 35 minutes into the lecture
period. For some questions, the students performed collaborative learning by worked
in pairs and for others they worked alone. The instructor provided the students
with the correct answer following the submission of their answers. The majority
of the questions were set a level that most students could answer relatively
The effectiveness of the method introduced in 2011 was
assessed in three ways (shown in Table 2). Of these, two (approaches 1 and 3,
table 2) were compared between 2009 and 2011. Additional information regarding
each approach is provided below.
Table 2. The
three approaches used to test the success of the introduced pedagogical
1. Course Learning ObjectiveS
Evaluation (CLOSE) forms
2. Surveys at the beginning and end of the
in student opinion
3. Analysis of student performance relating to the
The CLOSE forms are designed to obtain information on
student opinions relating to each CLO. The
CLOSE forms were distributed and completed by the students in the last lecture
period. For each CLO, the students provide a rating for four questions. Question
1 relates to their opinion before they took the course and questions 2, 3 and 4
relate to their opinions at the end of the course. The four questions were as
your level of understanding of this CLO before you took this course
you level of understanding of this CLO at the conclusion of this course
how confident you would feel explaining what you have learned to others
how confident you would feel using
information based on this CLO in a job
The student rating system for the CLOSE forms changed from
2009 to 2011 from a 4 point scale to a 5 point scale. The higher the score, the
higher the confidence level (e.g. 0 = not confident to 5 = very confident, in
account for the change between years and to enable a comparison, the data from
both years were normalized.
The second assessment method involved a short survey, given
to the students at the beginning and the end of the class (only in 2011). The
survey included questions to obtain student opinions on three areas, as follows
1) in class assessment, 2) use of response pads, and 3) CLOs. The surveys were
collected and stored until grades were submitted.
The third assessment method involved an analysis of student
performance on homework and exam questions relating to each CLO. The Civil and
Environmental Engineering (CEE) department uses one guideline to relate student
performance to each CLO. Each CLO is considered to be achieved if more than 75%
of the students obtain more than 75% of the available points. Data collected in
2009 and 2011 were compared to determine if the introduced pedagogical approach
improved student performance in CLOs achievement. Between 1 and 5 exam or
homework questions were used to generate these data for each CLO.
The 2009 and 2011 CLOSE data were summarized and are
illustrated below (Figure 1 A and B). In 2009, for the majority of CLOs a clear
increase can be seen between the question 1 (understanding before the course)
and questions 2, 3 and 4 (understanding after the course) (Figure 1A). The same
trend can be seen for 2011(Figure 1B). The only CLO showing limited improvement
for before and after the course is CLO 3 (to perform elementary chemical
calculations). In 2009, there was only a slight increase and in 2011 there was
no increase. Interestingly, when the data are compared between years, no change
is seen, indicating the new pedagogical approach had no effect on student
opinion of their own achievement of each CLO. This trend could potentially be
explained by the already high rating of questions 2, 3 and 4 in 2009, resulting
in limited room for improvement (ratings were almost saturated) in 2011.
The pre- and post- surveys provided additional data on
student opinions. In 2011, CE 280 students completed a survey at the beginning
and at the end of the semester in 2011. From 77 students, 39 students completed
both surveys and also signed a consent form. As discussed above, the surveys
involved questions relating to three areas. The results are summarized in
final assessment approach to test the efficiency of the introduced pedagogical
method in CE 280 was an examination of student performance relating to material
targeted by the CLOs. As discussed above, the CLO was considered to achieved
(successful teaching and learning of that CLO), if 75% (or more) of the
students achieved > 75% of the available points on questions targeting each
CLO. The data collected from 2009 and 2011 for each CLO were compared and are
shown in Figure 3. For each CLO data from between 1 and 4 questions (exam or
homeworks) were included.
Both surveys indicated positive opinions concerning the use
of in class assessment for improved student learning. As expected, the students
indicated a strong disagreement with only taking notes and not practicing
examples. When the data were compared using two tailed t-tests, no significant
difference was found between the two surveys for the section focusing on in
class assessment. Both surveys indicated the students were not concerned about
using clickers in class (almost all had used clickers in a previous class).
Also, the surveys provided data suggesting that the students believed clickers
provided an educational benefit. In addition, both surveys indicated the
students enjoyed and were comfortable using clickers, and that the devices helped
in learning the material. The final section of the surveys provided data on
student opinions on CLOs. These data indicated the students were already aware
of the importance of CLOs.
The pre- and post- survey data were compared (two-tailed
t-test). For the three sections, there was no significant difference (α = 0.05) in student answers between the two
surveys, except for one question.
Indicating that, in general, many opinions did not change as a result of
the pedagogical approach used. A significant difference was found for the
question “when I study, I consider the CLOs” (p value = 6 X 10-6, α = 0.05). As would be expected, the responses in
2011 were higher than those in 2009. Overall, the surveys provided valuable
data on student opinions on in class assessment, the use of clickers and the
importance of CLOs. Perhaps the most important message to be derived from both
surveys is the positive student opinion on in class assessment and the use of
Using the 75% threshold, several CLOs were not achieved in
2009 (CLOs 1, 3, 4, 8, 9, 10, 11, 14). In contrast to this, in 2011, only 1
(CLO 14) was not achieved. These data indicate the pedagogical approach introduced
in 2011 was successful.
The percentage of students achieving more than 75% of the
available points for each question was higher in 2011 for all CLOs except for
one (CLO 13). Unfortunately, many CLOs involved only one data point, limiting
any statistical analysis. Standard deviations were calculated and t-tests were
performed on CLOs with enough data points. Only one (CLO 4) showed a
significant difference between years (p value = 0.045, α = 0.05, two-tail).
These data on student
performance provide compelling evidence that the approach was successful in
improving student learning relating to the defined CLOs.
In 2011, in- class exercises were designed and included in
CE 280 to improve student learning. The exercises were targeted to ABET CLOs as
well as additional material covered in this class. The comparison of the 2009 and
2011 CLOSE forms indicated limited increase in student opinion of CLO
achievement in 2011 (however, student ratings were already high in 2009). Survey
data at the beginning and end of the semester provided evidence of strong
positive opinions on using clickers and on using in class assessment. The pre-
and post- survey data suggested student opinions did not change a result of
taking the class. However, opinions did change over the semester on one issue,
that being they will be more likely to consider CLOs when studying for exams.
Finally, and most importantly, a comparison of 2009 and 2011 ABET evidence data
illustrated a higher percentage of the students achieved each CLO in 2011. In
other words, student learning improved as a result of the pedagogical approach
introduced in 2011. The instructor intends to continue to use this approach.
The authors thank participating CE 280 students, Dr. DeZure,
Cindi Leverich (the Office of Faculty and Organizational Development), Dr.
Neeraj Buch (instruction on clickers, support letter, agreement on release
time), Dr. Ron Harichandran and Dr. Thomas Voice (support letters) and the
2011-2012 Lilly Teaching Fellow cohort. This project was funded by a grant
awarded to Dr. Cupples as part of the Lilly Teaching Fellowship Program at
Michigan State University.