Graduate Program Assessment Report
Program Assessment Report
Academic Year(s) Assessed: AY2022-23 and AY2023-24
College: College of Letters and Science
Department: Mathematical Sciences
Department Head: Elizabeth Burroughs
Submitted by: Elizabeth Burroughs
|
Program Information
|
Program
|
|---|---|
|
Degree/s Assessed
|
Ph.D. Mathematics
M.S. Mathematics M.S. Mathematics – Mathematics Education Option M.S. Statistics Certificate in Applied Statistics |
|
|
|
|
College or Administrative Division
|
College of Letters and Science
|
|
Department/School
|
Mathematical Sciences
|
|
Report Submitted By
|
Elizabeth Burroughs, Department Head
|
|
Date Submitted
|
October 15, 2022
|
|
Assessment Period:
|
May 16, 2020 - May 15, 2022
|
Graduate assessment reports are to be submitted biennially. The report deadline is October 15th.
Biennial Graduate Assessment Process:
Every graduate program assessment must have the following key components:
Program Description:
Ph.D. Mathematics: This program provides graduates with proficiency in mathematics and the opportunity
to carry out independent research in the mathematical sciences as demonstrated by
the completion of a doctoral dissertation.
Ph.D. Mathematics, Mathematics Education emphasis: This program provides the opportunity for research focused on mathematics teaching
and learning and includes the study of graduate-level mathematics.
Ph.D. Statistics: This program provides graduates with proficiency in statistics and the opportunity
to carry out independent research in statistics as demonstrated by the completion
of a doctoral dissertation.
M.S. Mathematics: This program provides fundamental knowledge in core areas of pure and applied mathematics.
It prepares graduates for careers in industry and for a PhD program in mathematics
or applied mathematics.
M.S. Mathematics, Mathematics Education Option: This program provides fundamental knowledge for secondary mathematics teaching. The
program deepens graduates’ understanding of school mathematics, increases their pedagogical
content knowledge, and provides opportunities for professional reflection growth.
M.S. Statistics: This program gives students a background in the theory of statistics and hands-on
practice in the application of statistics to real problems. Students in this program
prepare either for further graduate work or for academic, industrial, business, or
government employment.
M.S. Data Science: This program provides graduate students with foundational training in data analysis,
with equal emphasis on the principles of computer science, mathematics, and statistics,
and the ability to apply these principles to a range of data-driven problems.
Graduate Certificate in Statistics: This certificate program provides additional education in statistical thinking and
methodology over and above the basic coursework taken by the typical graduate student.
This certificate provides a clear record of additional training in statistics for
future graduate programs or employers.
Graduate Certificate for Dual Enrollment Mathematics Teachers: This certificate program provides a set of three courses in mathematics that provide
foundational knowledge and study in algebra, calculus, and statistics. This program
has been discontinued since AY2023-2024, due to non-enrollment. For that reason, it
is not included in this or any future assessment report and is only included on this
list for completeness.
1. Past Assessment Summary:
In our last Graduate Program Assessment report, we reported on successes in meeting our program learning outcomes. We largely met our targets and made notes of the following areas we intended to improve in our assessment process or in our programs.
- Our assessment relies on percentages, which is not informative when numbers of students at a particular stage of progress are low. We have changed that reporting this time.
- We did not have a mechanism for reporting reasons students leave the program. We have incorporated that this time.
- We recommended implementing exit interviews for MSDS students to improve assessment.
- We recommended implementing exit interviews for other graduate programs as well.
2. Action Research Question:
Do our students progress through our graduate programs with the rigor we expect, according to the timeline we expect? In what ways does our curriculum support or hinder this progress?
3. Assessment Plan, Schedule, and Data Scources:
a) Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data).
| PhD Mathematics PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate competence in graduate-level real analysis and linear algebra. | X | X | Completion rate of required written qualifying exam in linear algebra and real analysis |
| Demonstrate a solid understanding of core mathematical concepts in at least one area of specialty | X | X | Completion rate of required written comprehensive exam in an additional area |
| Formulate new research problems | X | X | Completion rate of dissertation proposal |
| Clearly communicate mathematical research both orally and in writing | X | X | Completion rate of dissertation defense |
| PhD Mathematics (Education) PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate a solid understanding of core graduate level mathematics | X | X | Completion rate of required written comprehensive exam in a mathematical topic area |
| Formulate questions and design studies to address contemporary issues in mathematics education | X | X | Completion rate of dissertation proposal |
| Clearly communicate mathematics education research both orally and in writing | X | X | Completion rate of dissertation defense |
| PhD Statistics PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate a solid understanding of probability and advanced mathematical statistics | X | X | Completion rate of required written comprehensive exam in probability and advanced mathematical statistics |
| Demonstrate a solid understanding of core statistical content in at least one research area of specialty | X | X | Completion rate of required written comprehensive exam in an additional area |
| Formulate new research problems | X | X | Completion rate of dissertation proposal |
| Clearly communicate original statistical research both orally and in writing | X | X | Completion rate of dissertation defense |
| MS Mathematics PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate solid understanding of graduate level real analysis and advanced linear algebra | X | X | Completion rate of required written comprehensive exam or MS Thesis defense |
| Demonstrate solid understanding of core mathematical concepts in at least one area of specialty | X | X | Completion rate of M 511 and M 504 |
| MS Mathematics (education) PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate solid understanding of graduate level mathematics relevant to secondary content in algebra, calculus, geometry and statistics | X | X | Completion rate of 4 required content area courses (M 518, 524, 525, 527) |
| Demonstrate solid understanding of teaching practices that give every student access to rigorous mathematics learning | X | X | Completion rate of 2 required pedagogy courses (M 520, 528, 529 or 577) |
| Clearly communicate connections between program coursework and local classroom practice | X | X | Completion rate of written portfolio and public presentation |
| MS DATA Science PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate knowledge of essential deterministic, randomized and approximation algorithms for data classification and clustering, dimensionality reduction, regression, optimization. | X | X | Completion rate of M 508 |
| Demonstrate knowledge in the principles and practice of statistical experimental design, statistical inference, and decision theory. | X | X | Completion rate of STAT 541 |
| Demonstrate the ability to take a real-world data analysis problem, formulate and conceptual approach to the problem, match aspects of the problem to previously learned theoretical and methodological tools, break down the solution into step-by-step approach, and implement a working solution in a modern software language. | X | X | Completion rate of CSCI 531 |
| Communicate data science problems, analyses, and solutions effectively to both specialists and non-specialists through the use of effective technical writing, presentations, and data visualizations and teamwork and collaboration. | *This could be assessed by a capstone requirement, which is not currently available. | ||
| MS Statistics PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Possess a solid understanding of core graduate level applied statistics, probability, and mathematical statistics | X | X | Completion rate of Comprehensive exam |
| Be prepared for career as an applied statistician or a doctoral program in statistics | X | X | Completion rate of Statistical Consulting Seminar (Stat 510) |
| Clearly communicate results from a statistical data analysis or research problem both orally and in writing | X | X | Completion of Writing Project or Thesis |
| Graduate Certificate in Applied Statistics PROGRAM LEARNING OUTCOMES |
2022-23 | 2023-24 | Data Source |
| Demonstrate advanced statistical thinking and data collection. | X | X | Completion rate of STAT 511 |
| Apply advanced statistical methodology. | X | X | Completion rate of STAT 512 |
b) What are the threshold values for which your program demonstrates student achievement?
The PhD programs in Mathematics, Mathematics with Education emphasis, and Statistics are assessed by measuring the completion rates of students who advance through the program using a sequence of Milestones. The number of students who complete a given milestone is measured. The time required for each student to advance between successive milestones is also measured. The threshold values and data sources in Table 1 below incorporate both quantities into the assessment.
The MS programs in Mathematics, Mathematics Education, Statistics and Data Science
are assessed by measuring the completion rates of students who advance through their
program by achieving satisfactory performance in coursework, by demonstrating core
competencies on a written comprehensive exam and by demonstrating the ability to communicate
knowledge relevant to the particular field of study. The nature of the program determines
the structure of the assessment for these various programs, and the threshold values
and data sources are described in Table 2 below.
|
|
Ph.D. Programs
|
|
|---|---|---|
|
PROGRAM LEARNING OUTCOME
|
Threshold Value
|
Data Source
|
|
Demonstrate a solid understanding of [PhD core content appropriate to each degree,
as listed in program outcomes]
|
75%* of students who begin the degree program will pass the [appropriate] written comprehensive exam within 2 years
|
Milestone 1 Written comprehensive exam in additional specialty area |
|
Formulate new research problems
|
Of those students who have achieved Milestone 1, 75%* will pass Milestone 2 within
2 years of the term in which
Milestone 1 was achieved. |
Milestone 2
Oral comprehensive exam |
|
Clearly communicate [original research appropriate to each PhD degree] both orally
and in writing
|
Of those students who have achieved Milestone 2, 75%* will pass Milestone 3 within
2 years of the term in which
Milestone 2 was achieved. |
Milestone 3
Defense of dissertation |
*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.
|
|
M.S. Programs
|
|
|---|---|---|
|
PROGRAM LEARNING OUTCOME
|
Threshold Value
|
Data Source
|
|
MS Math and MS Stat
Demonstrate a solid understanding of [MS core content appropriate to the degree] |
75%* of students who begin the degree program will pass the [appropriate] written
comprehensive exam within 2
years. |
Written comprehensive exam
|
|
MS Math and MSMME
Demonstrate a solid understanding of [MS core content appropriate to the degree] |
75%* of students who begin the MS degree will earn a B or better in
[appropriate] coursework |
Coursework
|
|
MSMME
Clearly communicate connections between program coursework and local classroom practice |
75%* of students who begin the MSMME will present a portfolio within 3 years
|
Portfolio and presentation
|
|
MS Stat
Be prepared for career as an applied statistician or a doctoral program in statistics |
75%* of students who begin the MS Statistics program will complete 2 credits of Stat
510 with a B or better in 3
years |
Coursework
|
|
MS Stat
Clearly communicate results from a statistical data analysis or research problem both orally and in writing (MS Stat only) |
75%* of students of begin the MS Statistics program will complete a writing project
or thesis within 3 years
|
Writing project or thesis and presentation
|
*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.
4. What Was Done.
a) Self-reporting Metric (required answer): Was the completed assessment consistent with the program’s assessment plan? If not, please explain the adjustments that were made.
X Yes
b) How were data collected and analyzed and by whom? Please include method of collection and sample size.
Data were collected from enrolled student records and summarized in tabular form.
We used a census of enrolled students. The data were analyzed by the department’s
graduate program committee. Additionally, the Graduate Program Committee meets at
least monthly throughout the academic year and engages in regular communication with
the entire department faculty to discuss the programs, curriculum, and adjustments.
We use enrollment summaries, as follows.
Enrollment Summaries by Program
PhD Mathematics
Twenty students were enrolled in the PhD Mathematics program at some point during
the assessment period; three left the program for personal or professional reasons,
so 17 students are included in this summary. Of those 17, 9 were eligible to complete
Milestone 1. Eight of the 9 (89%) met the milestone. During the review period, two students were
eligible to complete Milestone 2, and both (100%) met the milestone. During the review period, four students were eligible
to complete Milestone 3, but one student is working full time while completing the dissertation, so the normal
timelines do not apply. Of the three remaining, two (67%) passed their dissertation
within two years of the dissertation proposal.
Qualitative analysis of Milestones 2 and 3: All eligible students met Milestone 2,
so we consider that the program learning outcome is met. For Milestone 3, the student
who did not meet the timeline was still successful in producing a dissertation, so
while the timeline was not met, we consider the program learning outcome to be met.
PhD Mathematics – Mathematics Education Option
Seven students were enrolled in the PhD Mathematics – Mathematics Education emphasis program at some point during the assessment period. One student was eligible to complete Milestone 1 and was successful (100%). Three students were eligible to complete Milestone 2 and all were successful (100%). Two students were eligible to complete Milestone 3 and all were successful (100%).
PhD Statistics
Fourteen students were enrolled in the PhD Statistics program during the assessment
period. One student was eligible to complete Milestone 1 and did not meet the timeline (0%). Four students were eligible to complete Milestone 2, and 3 (75%) met the milestone. Two students were eligible to meet Milestone 3, and one (50%) met the milestone.
Qualitative analysis of Milestones 1 and 3. For Milestone 1, the student has attempted
and passed one component of the written exam, but has not yet attempted the second
component. While the timeline was not met, we consider the program learning outcome
to be met. For Milestone 3, the student who did not meet the timeline was still successful
in producing a dissertation, so while the timeline was not met, we consider the program
learning outcome to be met.
MS Mathematics
A total of twenty-eight students (not including 4 students who earned masters en route to PhD) were enrolled in the MS Mathematics program for some portion of the assessment period; one left the program for personal reasons, so 27 students are included in this summary. Of those 27, 21 were eligible to complete Milestone 1, and 21 (100%) met the milestone. Twenty-one were eligible to complete Milestone 2, and 20 (95%) met the milestone.
MSMME
Thirty-four students were enrolled in the Master of Science in Mathematics – Mathematics Education Option during the assessment period. Of those enrolled, 26 had progressed to a point in their program where we would expect them to have achieved our program learning outcomes and all 26 achieved Milestones 1 & 2 (100%).
MS Data Science
Thirteen students were enrolled in the MS Data Science program during the assessment period. Of those, ten were eligible to complete Milestone 1, and all (100%) were successful. Ten were eligible to complete Milestone 2, and all (100%) were successful. Eleven were eligible to complete Milestone 3, and 7 were successful (64%).
Graduate Certificate in Applied Statistics
Of the 2 students enrolled in the program within the assessment period, 1 completed the required coursework for the degree (100%) and 1 continues in the program.
c) Please provide a rubric that demonstrates how your data were evaluated.
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate competence in graduate-level real analysis and linear algebra.
|
Completion rate of Milestone 1 – written comprehensive exam in
Real and Complex Analysis |
Met (89%)
|
|
Demonstrate a solid understanding of core mathematical concepts in at least one area
of specialty
|
Completion rate of Milestone 1 - written comprehensive exam in an
additional specialty area |
Met (89%)
|
|
Formulate new research problems
|
Completion rate of Milestone 2
|
Met (100%)
|
|
Clearly communicate mathematical research both orally and in writing
|
Completion rate of Milestone 3
|
Met – qualitative analysis of success in clear communication
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate a solid understanding of core graduate level mathematics
|
Completion rate of Milestone 1 - written comprehensive exam in a mathematical topic
area
|
Met (100%)
|
|
Formulate questions and design studies to address contemporary issues in mathematics
education
|
Completion rate of dissertation proposal
|
Met (100%)
|
|
Clearly communicate mathematics education research both orally and in writing
|
Completion rate of dissertation defense
|
Met (100%)
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate a solid understanding of advanced mathematical statistics, Bayesian statistics,
and data analysis methods.
|
Completion rate of Milestone 1 - written comprehensive exam components in advanced
mathematical statistics, Bayesian statistics and data analysis
|
Met – qualitative analysis
|
|
Demonstrate a solid understanding of core statistical content in at least one research
area of specialty
|
Completion rate of Milestone 1 - written comprehensive exam component in a research
area of specialty
|
Met – qualitative analysis
|
|
Formulate new research problems
|
Completion rate of dissertation proposal
|
Met (75%)
|
|
Clearly communicate original statistical research both orally and in writing
|
Completion rate of dissertation defense
|
Met – qualitative analysis
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate solid understanding of graduate level real analysis and advanced linear
algebra
|
Completion rate of required written comprehensive exam or thesis
defense. |
Met (100%)
|
|
Demonstrate solid understanding of core mathematical concepts in at least one area
of specialty
|
Completion rate of M 511 and M 504
|
Met (95%)
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate solid understanding of graduate level mathematics relevant to secondary
content
|
Completion rate of content courses
|
Met (100%)
|
|
Demonstrate solid understanding of teaching practices that give every student access
to rigorous mathematics learning
|
Completion rate of 2 required pedagogy courses (M 520, 528, 529 or 577)
|
Met (100%)
|
|
Clearly communicate connections between program coursework and local
classroom practice |
Completion rate of written portfolio and public presentation
|
Met (100%)
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Possess a solid understanding of core graduate level applied statistics, probability,
and mathematical
statistics |
Completion rate of written comprehensive exam
|
Met (95%)
|
|
Be prepared for career as an applied statistician or a doctoral program in statistics
|
Completion rate of Statistical Consulting Seminar (Stat 510)
|
Met (100%)
|
|
Clearly communicate results from a statistical data analysis or research problem both
orally and in writing
|
Completion rate of Writing Project or Thesis
|
Met (100%)
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate knowledge of essential deterministic, randomized and approximation algorithms
for data classification and clustering, dimensionality reduction,
regression, optimization. |
Completion of M 508
|
Met (100%)
|
|
Demonstrate knowledge in the principles and practice of statistical experimental design,
statistical inference, and decision theory.
|
Completion of STAT 541
|
Met (100%)
|
|
Demonstrate the ability to take a real-world data analysis problem, formulate and
conceptual approach to the problem, match aspects of the problem to previously learned
theoretical and methodological tools, break down the solution into step-by-step approach,
and implement a working solution
in a modern software language. |
Completion of CSCI 531
|
Not Met (64%)
|
|
Communicate data science problems, analyses, and solutions effectively to both specialists
and non-specialists through the use of effective technical writing, presentations,
and data visualizations and
teamwork and collaboration. |
Program completion
|
Met (8 graduates of the program)
|
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
|---|---|---|
|
Demonstrate advanced statistical thinking and data collection.
|
Completion rate of STAT 511
|
Met (100%)
|
|
Apply advanced statistical methodology.
|
Completion rate of STAT 512
|
Met (100%)
|
5. What Was Learned.
a) Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment? What areas of strength in the program were identified from this assessment process?
Our PhD programs are successful at meeting program learning outcomes, though sometimes
the timeline takes longer than our Milestones. Still, we find that our recent focus
pairing timelines with PLOs has helped keep our attention on supporting students to
make timely progress through the degree.
Our MS Math, Math Ed, and Stat programs are successful at meeting program learning
outcomes and at meeting timelines.
Our MS Data Science program requires more attention. For the second consecutive biennial
assessment report, we have not met the learning outcome threshold for the computer
science Algorithms foundation. We will provide focused attention on how to address
this in the upcoming two years, with a particular look at admissions standards and
prerequisite fulfillment for that course.
b) Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment?
What areas of strength in the program were identified from this assessment process?
Our PhD programs are successful at meeting program learning outcomes, though sometimes
the timeline takes longer than our Milestones. Still, we find that our recent focus
pairing timelines with PLOs has helped keep our attention on supporting students to
make timely progress through the degree.
Our MS Math, Math Ed, and Stat programs are successful at meeting program learning
outcomes and at meeting timelines.
Our MS Data Science program requires more attention. For the second consecutive biennial
assessment report, we have not met the learning outcome threshold for the computer
science Algorithms foundation. We will provide focused attention on how to address
this in the upcoming two years, with a particular look at admissions standards and
prerequisite fulfillment for that course.
6. How We Responded.
a) Describe how “What Was Learned” was communicated to the department, or program faculty. How did faculty discussions re-imagine new ways program assessment might contribute to program growth/improvement/innovation beyond the bare minimum of achieving program learning objectives through assessment activities conducted at the course level?
The Graduate Program Committee met and discussed the assessment process and results and led a discussion among the department. The Department has engaged in various program updates over the past two years, adjusting requirements, updating learning program outcomes, adjusting admissions processes by involving more faculty, adjusting advising practices, adjusting comprehensive and qualifying exam requirements and practices, and maintaining a focus on our students as those who are part of our vision of using the tools of our disciplines to promote human flourishing. In particular, to help students meet milestones, we have focused our attention on supporting students as individuals and engaging in collective conversations to discuss milestones in relation to each student’s academic progress and overall wellbeing.
b) How are the results of this assessment informing changes to enhance student learning in the program?
Reflecting on these results gives us confidence that our practice of ongoing discussion and adjustment is a positive for our programs. It reminds us to continue to focus on the Data Science program.
C) If information outside of this assessment is informing programmatic change, please describe that.
See part a.
d) What support and resources (e.g. workshops, training, etc.) might you need to make these adjustments?
We took part in a discussion with the assistant provost responsible for assessment and found that informal discussion useful.
7. Closing the Loop(s).
Reflect on the program learning outcomes, how they were assessed in the previous cycle (refer to #1 of the report), and what was learned in this cycle. What action will be taken to improve student learning objectives going forward?
a) Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
X Yes
b) Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
We updated a program learning outcome in the PhD mathematics program: we replaced
“Demonstrate a solid understanding of core graduate level real and complex analysis”
with “Demonstrate competence in graduate-level real analysis and linear algebra”,
to reflect the shift we made in reflecting on our last set of PLOs and required coursework
for the degree.
We adjusted how we use threshold values in the case of low numbers of students, and
specifying a reliance on qualitative analysis has given us a useful mechanism for
meaningful reflection in these circumstances.
Going forward, we intend to update the course requirements of our Math MS program
for the following effects.
1. Better reflect existing faculty expertise.
2. Better prepare Math MS students for a PhD program in a mathematically-related field.
3. Align the experiences of typical 1st year Math MS and typical 1st year Math PhD
students.
In the M.S. Statistics program, we intend to examine the alignment between comprehensive exam practices and course content and learning outcomes.
In the PhD Math Ed program, we intend to examine coursework to align it with current
trends among peer programs and with faculty expertise.
c) Have you seen a change in student learning based on other program adjustments made in the past? Please describe the adjustments made and subsequent changes in student learning.
This assessment process also lets us reflect on how and which students move from the MS to the PhD (in non-published data tables). This is useful, because recruiting for graduate programs remains a priority for the department.