Annual Program Assessment Report
Assessment Report
AY: 2021-2022
College: College of Letters and Science
Department: Mathematical Sciences
Submitted by: Elizabeth Burroughs, Department Head
Program(s) Assessed: Indicate all majors, minors, certificates and /or options that are included in this assessment:
|
Majors/Minors/Certificate
|
Options
|
|---|---|
|
Mathematics (Major)
|
Applied Math, Math, Math Teaching, Statistics
|
|
Mathematics (Minor)
|
|
|
Statistics (Minor)
|
|
Annual Assessment Process (CHECK OFF LIST)
| 1. Data are collected as defined by Assessment Plan | YES _X_ | NO ___ | |
|
2. Population or unbiased sameples of collected assignments are scored by at least two faculty members using scoring rubrics to ensure inter-rater reliability. |
YES _X_ | NO ___ | |
| 3. Areas where the acceptable persormance threshold has not been met are highlighted | YES _X_ | NO ___ | |
| 4. Assessment scores were presented at a program/unit faculty meeting. | YES _X_ | NO ___ | |
| 5. The faculty reviewed the assessment results, and responded accordingly (Check all appropriate lines) | |||
| Gather additional data to verify or refute the result. __X___ | |||
| Identify potential curriculum changes to try to address the problem _____ | |||
| Change the acceptable performance threshold, reassess _____ | |||
| Choose a different assignment to assess the outcome _____ | |||
| Faculty may reconsider thresholds_____ | |||
| Evaluate the rubric to assure outcomes meet student skill level _____ | |||
| Use Bloom’s Taxonomy to consider stronger learning outcomes _____ | |||
| Choose a different assignment to assess the outcome__X___ | |||
| OTHER: | |||
| 6. Does your report demonstrate changes made because of previous assessment results (closing the loop)? | YES__X___ | NO_____ |
1. Assessment Plan, Schedule and Data Source
| PROGRAM LEARNING OUTCOME | 2020-2021 | 2021-2022 | 2022-2023 | 2023-2024 | Data Source* |
|---|---|---|---|---|---|
| 1. Students will demonstrate mathematical reasoning or statistical thinking | X | X | M 242 Signature Assignment | ||
| 2. Students will demonstrate effective mathematical or statistical communication | X | X | M 242 Signature Assignment | ||
| 3. Students will develop a range of appropriate mathematical or statistical methods for proving, problem solving, and modeling | X | X | M 384, M 329, and Stat 412 Signature Assignments |
The UPC members are Jack Dockery, Ryan Grady, Stacey Hancock, Jennie Luebeck, and Tianyu Zhang. The department head appointed a task force to assess individual courses: Tianyu Zhang and Ryan Grady to analyze data from M 384, Beth Burroughs and Jennie Luebeck to analyze data from M 329, and Katie Banner and Mark Greenwood to analyze data from STAT 412.
The task force submitted the results below to the UPC and the DH on September 27,
2022. It was compiled and reviewed by the UPC on September 28 and October 5. It was
circulated among the faculty and discussed at the October 19 faculty meeting.
b. What are your threshold values for which you demonstrate student achievement?
Threshold Values:
| PROGRAM LEARNING OUTCOME | Threshold Value | Data Source |
|---|---|---|
| 1. Students will demonstrate mathematical reasoning or statistical thinking. | The threshold value for this outcome is for 70% of assessed students to score acceptable or proficient on the scoring rubric. | Not assessed this cycle. |
| 2. Students will demonstrate effective mathematical or statistical communication. | The threshold value for this outcome is for 70% of assessed students to score acceptable or proficient on the scoring rubric. | Not assessed this cycle. |
| 3. Students will develop a range of appropriate mathematical or statistical methods for proving, problem solving, and modeling. | The threshold value for this outcome is for 70% of assessed students to score acceptable or proficient on the scoring rubric. | M 384, M 329, and Stat 412 Signature Assignments |
2. What Was Done
a) Was the completed assessment consistent with the plan provided? YES__X___ NO_____
b) Please provide a rubric that demonstrates how your data was evaluated.
M 384: Criteria for demonstrating understanding:
a. For problem 1, understand that the Fourier series for a function defined on [-π
, π] is 2π periodic.
b. For problem 1, correctly use the periodicity of the Fourier series to calculate
its value at any given point.
c. For problem 2, understand that the norm of a normed linear space is induced by
an inner product if and only if the norm satisfies the parallelogram law
d. For problem 2, correctly apply the parallelogram law in the discrete lp space to
show that it is a Hilbert space if and only if p = 2
|
M 384 Rubric
|
Unacceptable 1
|
Acceptable 2
|
Proficient 3
|
|---|---|---|---|
|
Outcome 3: |
Displays limited or inappropriate reasoning strategies in the statistical content focus.
|
Adequately displays reasoning strategies in the statistical content focus.
|
Displays thorough and appropriate reasoning strategies in the statistical content focus.
|
| M 329 Rubric | Unacceptable 1 | Acceptable 2 | Proficient 3 |
|---|---|---|---|
|
Outcome 3: |
Displays limited or inappropriate proof, problem solving, or modeling strategies in the mathematical content focus.
|
Adequately displays appropriate proof, problem solving, or modeling strategies in the mathematical content focus.
|
Displays thorough and appropriate proof, problem solving, or modeling strategies in the mathematical content focus.
|
STAT 412: Criteria for demonstrating understanding. In at least 2 of the four problems the student does the following:
a. Distribution of the response is appropriate given the scenario (including matching
component)
b. Link function matches choice of distribution (dependent on choice in (a), even
if (a) is incorrect)
c. Systematic component accurately reflects the research question (i.e., is additive
or interactive where appropriate)
d. All variables are defined completely
| STAT 412 Rubric | Unacceptable 1 | Acceptable 2 | Proficient 3 |
|---|---|---|---|
|
Outcome 3: |
Displays limited or inappropriate reasoning strategies in the statistical content focus.
|
Adequately displays reasoning strategies in the statistical content focus.
|
Displays thorough and appropriate reasoning strategies in the statistical content focus.
|
3. How Data Were Collected
a) How were data collected? (Please include method of collection and sample size).
M 384: The signature assignment chosen was the final portfolio. Out of 28 enrolled students, the instructor of the course randomly identified 10 students (5 in the math option, 5 in the applied math option). For each student, the instructor collected one problem from each of two quizzes.
M 329: The two most recent instructors identified two problems that would allow assessment
of this outcome. Task force members chose the problem that addressed both facets (problem
solving and proving) and had complete student submissions. Of the 10 students enrolled
in the course, 6 were math teaching majors; all 6 of these students were included
in the sample.
STAT 412: The program assessment questions were on the final exam for STAT 412. The
instructor of the course collected the exams and removed identifying information,
including majors and minors of the students associated with each exam. There were
9 exams from students either majoring or minoring in Mathematics or Statistics.
In all cases, identifying information was removed and data were stored in a secure
One Drive folder for the task force to access and assess.
b) Explain the assessment process, and who participated in the analysis of the data. Include the signature assignment (for faculty review; delete before posting to the web because signature assignments may be reused on future exams).
M 384: blinded student work was assessed on two problems, which are maintained in the database.
Tianyu Zhang independently applied the rubric and then discussed any borderline cases with Ryan Grady (the current instructor of M 384) until they reached consensus on student scores.
| M 384: Of the 10 student submissions assessed: overall for Outcome 3, 90% scored at acceptable level (20% at proficient level). |
M 329: blinded student work was assessed on two problems, which are maintained in the database.
Jennie Luebeck and Elizabeth Burroughs analyzed the data independently, then met to
discuss the 1 data point on which they did not initially agree, and they discussed
in order to reach complete agreement.
| M 329: Of the 6 student submissions assessed: 100% were acceptable (83% at proficient level) on Problem 1, 83% were acceptable (33% at proficient level) on Problem 2; overall for Outcome 3, 83% scored at acceptable level (33% at proficient level). |
STAT 412: blinded student work was assessed on a 4-part question with each part representing a different research scenario. For each part, students were asked to:
• read the research scenario and associated research question
• choose an appropriate distribution for modeling the response variable
• write a linear or generalized linear model using appropriate mathematical
notation that allowed the research question
to be addressed, and
• correctly define all variables in their chosen model.
All exams were assessed independently by Katie Banner and Mark Greenwood. Discrepancies
in assessment scores were discussed and resolved using the scoring rubric.
| STAT 412: Of the 9 student submissions assessed: overall for Outcome 3, 89% scored at acceptable level (33% at proficient level). |
4. What Was Learned
Based on the analysis of the data, and compared to the threshold values provided, what was learned from the assessment?
a) Areas of strength
M 384: The experience provided in M 384 during the Spring 2022 semester was sufficient to meet the threshold of at least 70% of students at acceptable or proficient. Overall, students had a solid grasp of understanding theorems and applying theorems to prove or solve particular problems and to prove or solve elementary statements.
M 329: The experiences provided in M 329 are sufficient to meet the threshold of at least 70% of students at acceptable or better. The course prepares more students to be proficient at mathematical problem solving than proving.
STAT 412: The experience provided in STAT 412 during the Spring 2022 semester was sufficient to meet the threshold of at least 70% of students at acceptable or proficient. Overall, students had a solid grasp of choosing appropriate distributions for the response variable in a statistical model (generalized linear model) or writing appropriate functions of explanatory variables (both continuous and categorical) to represent research questions of interest, but not always both.
b) Areas that need improvement
M 384: Continue to focus on mathematical methods for proving and problem solving, with an aim to ensuring more students move beyond acceptability and achieve proficiency in their senior- and graduate-level coursework. Problem 2 indicated that throughout the M383/384 sequence, special attention should be paid to more intricate arguments, e.g., combining several relevant results, to build proficiency in mathematical argument.
M 329: Increase the focus on mathematical knowledge for teaching about proof. The problems
chosen for assessment should provide the opportunity to examine how math-teaching
majors demonstrate an acceptable level of mathematical problem solving, proving, and
modeling as relate to teaching.
STAT 412: In STAT 412 and subsequent courses, continue to focus on statistical methods for
proving, problem solving, and modeling, with an aim to ensuring more students move
beyond acceptability and
achieve proficiency in their senior- level coursework. Specifically, only 33% of students
had a strong grasp of choosing an appropriate probability distribution for modeling
the response variable and also specifying the most appropriate linear model to address
research questions of interest. Areas that require more focus are determining when
interactions in models are needed and appropriately representing categorical variables
in linear models as a set of indicator variables. These concepts are part of the curriculum
in STAT 217 and STAT 411 and they should continue to be emphasized to prepare students
to solidify understanding in STAT 412.
5. How We Responded
a) Describe how “What Was Learned” was communicated to the department, or program faculty. Was there a forum for faculty to provide feedback and recommendations?
Reports from individual courses went through two rounds of discussion and synthesis within the task force. The report was then circulated among the faculty and discussed at the October 19 faculty meeting.
b) Based on the faculty responses, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
These data do not suggest that major changes are needed to the assessed curriculum or the assessment process. However, the evidence reminds us of the importance of maintaining commitment to the more advanced learning goals within these courses. Overall, we suggest rebalancing the attention given to proof, justification, and sense making. These mathematical practices should be emphasized in all three Mathematical Sciences degree programs.
YES______ NO___X___
If yes, when will these changes be implemented? Not applicable
Please include which outcome is targeted, and how changes will be measured for improvement.
If other criteria are used to recommend program changes (such as exit surveys, or
employer satisfaction surveys) please explain how the responses are driving department,
or program decisions.
At this time there are no additional criteria used for undergraduate program assessment. We are exploring the possibility of adding a student exit survey to the assessment process every other year. The Undergraduate Program Committee is exploring this option.
c) When will the changes be next assessed?
These courses and Outcome 3 will be assessed again two years from now.
6. Program Action
a) Based on assessment from previous years, can you demonstrate program level changes that have led to outcome improvements?
We recently refined our program outcomes and realigned our assessment process. This was our first time using this instrument to assess Outcome 3. We found it to be an effective tool for data gathering, reflection, and discussion.