Graduate Program Assessment Report
Program Assessment Report
Academic Year(s) Assessed: AY2023-24
College: College of Letters and Science
Department: Mathematical Sciences
Department Head: Elizabeth Burroughs
Submitted by: Elizabeth Burroughs
Program Information
|
Program
|
---|---|
Degree/s Assessed
|
Ph.D. Mathematics
|
College or Administrative Division
|
College of Letters and Science
|
Report Submitted By
|
Mathematical Sciences
|
Date Submitted
|
Elizabeth Burroughs, Department Head
|
Assessment Period:
|
May 16th, 2023 - May 15th, 2024
|
Graduate assessment reports are to be submitted biennially. The report deadline is October 15th.
Biennial Graduate Assessment Process:
Every graduate program assessment must have the following key components:
Program Description:
Ph.D. Mathematics: This program provides graduates with proficiency in mathematics and the
opportunity to carry out independent research in the mathematical sciences as demonstrated
by the
completion of a doctoral dissertation.
Ph.D. Mathematics, Mathematics Education emphasis: This program provides the opportunity for
research focused on mathematics teaching and learning and includes the study of graduate-level
mathematics.
Ph.D. Statistics: This program provides graduates with proficiency in statistics and the opportunity
to
carry out independent research in statistics as demonstrated by the completion of
a doctoral
dissertation.
M.S. Mathematics: This program provides fundamental knowledge in core areas of pure and applied
mathematics. It prepares graduates for careers in industry and for a PhD program in
mathematics or
applied mathematics.
M.S. Mathematics, Mathematics Education Option: This program provides fundamental knowledge for
secondary mathematics teaching. The program deepens graduates’ understanding of school
mathematics, increases their pedagogical content knowledge, and provides opportunities
for
professional reflection growth.
M.S. Statistics: This program gives students a background in the theory of statistics and hands-on
practice in the application of statistics to real problems. Students in this program
prepare either for
further graduate work or for academic, industrial, business, or government employment.
M.S. Data Science: This program provides graduate students with foundational training in data analysis,
with equal emphasis on the principles of computer science, mathematics, and statistics,
and the ability
to apply these principles to a range of data-driven problems.
Graduate Certificate in Statistics: This certificate program provides additional education in statistical
thinking and methodology over and above the basic coursework taken by the typical
graduate student.
This certificate provides a clear record of additional training in statistics for
future graduate programs or
employers.
Graduate Certificate for Dual Enrollment Mathematics Teachers: This certificate program provides a set
of three courses in mathematics that provide foundational knowledge and study in algebra,
calculus,
and statistics. This program has been discontinued since AY2023-2024, due to non-enrollment. For that
reason, it is not included in this or any future assessment report and is only included
on this list for
completeness.
1. Past Assessment Summary:
In our last Graduate Program Assessment report, we reported on successes in meeting our program learning outcomes. We largely met our targets and made notes of the following areas we intended to improve in our assessment process or in our programs.
- Our assessment relies on percentages, which is not informative when numbers of students at a particular stage of progress are low. We have changed that reporting this time.
- We did not have a mechanism for reporting reasons students leave the program. We have incorporated that this time.
- We recommended implementing exit interviews for MSDS students to improve assessment.
- We recommended implementing exit interviews for other graduate programs as well.
2. Action Research Question:
Do our students progress through our graduate programs with the rigor we expect, according to the timeline we expect? In what ways does our curriculum support or hinder this progress?
3. Assessment Plan, Schedule, and Data Sources:
a) Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data).
PhD Mathematics PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source
|
---|---|---|---|
Demonstrate competence in graduate-level real analysis and linear algebra.
|
X
|
X
|
Completion rate of required written qualifying exam in linear algebra and real analysis
|
Demonstrate a solid understanding of core mathematical concepts in at least one area
of specialty
|
X
|
X
|
Completion rate of required written comprehensive exam in an additional area
|
Formulate new research problems
|
X
|
X
|
Completion rate of dissertation proposal
|
Clearly communicate mathematical research both orally and in writing
|
X
|
X
|
Completion rate of dissertation defense |
PhD Mathematics (Education) PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source |
Demonstrate a solid understanding of core graduate level mathematics |
X
|
X
|
Completion rate of required written comprehensive exam in a mathematical topic area |
Formulate questions and design studies to address contemporary issues in mathematics education |
X
|
X
|
Completion rate of dissertation proposal |
Clearly communicate mathematics education research both orally and in writing |
X
|
X
|
Completion rate of dissertation defense |
PhD Statistics |
2022-23
|
2023-24
|
Data Source |
Demonstrate a solid understanding of probability and advanced mathematical statistics |
X
|
X
|
Completion rate of required written comprehensive exam in probability and advanced mathematical statistics |
Demonstrate a solid understanding of core statistical content in at least one research area of specialty |
X
|
X
|
Completion rate of required written comprehensive exam in an additional area |
Formulate new research problems |
X
|
X
|
Completion rate of dissertation proposal |
Clearly communicate original statistical research both orally and in writing |
X
|
X
|
Completion rate of dissertation defense |
MS Mathematics
PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source
|
---|---|---|---|
Demonstrate solid understanding of graduate level real analysis and advanced linear algebra |
X
|
X
|
Completion rate of required written comprehensive exam or MS Thesis defense
|
Demonstrate solid understanding of core mathematical concepts in at least one area
of specialty
|
X
|
X
|
Completion rate of M 511 and M 504
|
MS Mathematics (education)
|
2022-23
|
2023-24
|
Data Source
|
Demonstrate solid understanding of graduate level mathematics relevant to secondary
content in algebra, calculus,
geometry and statistics |
X
|
X
|
Completion rate of 4 required content area courses
(M 518, 524, 525, 527) |
Demonstrate solid understanding of teaching practices that give every student access
to rigorous mathematics learning
|
X
|
X
|
Completion rate of 2 required pedagogy courses
(M 520, 528, 529 or 577) |
Clearly communicate connections between program coursework and local classroom practice
|
X
|
X
|
Completion rate of written portfolio and public
presentation |
MS Data Science
PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source
|
Demonstrate knowledge of essential deterministic, randomized and approximation algorithms
for data classification and clustering, dimensionality reduction, regression, optimization
|
X
|
X
|
Completion rate of M 508
|
Demonstrate knowledge in the principles and practice of statistical experimental design,
statistical inference, and decision theory
|
X
|
X
|
Completion rate of STAT 541
|
Demonstrate the ability to take a real world data analysis problem, formulate and
conceptual approach to the problem, match aspects of the problem to previously learned
theoretical and methodological tools, break down the solution into step-by- step approach,
and implement a working solution in a modern software language.
|
X
|
X
|
Completion rate of CSCI 531
|
Communicate data science problems, analyses, and solutions effectively to both specialists
and non-specialists through the use of effective technical writing, presentations,
and data visualizations and teamwork and collaboration.
|
X
|
X
|
*This could be assessed by a capstone requirement,
which is not currently available. |
MS Statistics
PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source
|
Possess a solid understanding of core graduate level applied statistics, probability,
and mathematical statistics
|
X
|
X
|
Completion rate of Comprehensive exam
|
Be prepared for career as an applied statistician or a doctoral program in statistics
|
X
|
X
|
Completion rate of Statistical Consulting Seminar
(Stat 510) |
Clearly communicate results from a statistical data analysis or research problem both
orally and in writing
|
X
|
X
|
Completion of Writing Project or Thesis
|
Graduate Certificate in Applied Statistics
PROGRAM LEARNING OUTCOMES |
2022-23
|
2023-24
|
Data Source
|
Demonstrate advanced statistical thinking and data collection
|
X
|
X
|
Completion rate of STAT 511
|
Apply advanced statistical methodology
|
X
|
X
|
Completion rate of STAT 512
|
b) What are the threshold values for which your program demonstrates student achievement?
The PhD programs in Mathematics, Mathematics with Education emphasis, and Statistics are assessed by measuring the completion rates of students who advance through the program using a sequence of Milestones. The number of students who complete a given milestone is measured. The time required for each student to advance between successive milestones is also measured. The threshold values and data sources in Table 1 below incorporate both quantities into the assessment.
The MS programs in Mathematics, Mathematics Education, Statistics and Data Science
are assessed by measuring the completion rates of students who advance through their
program by achieving satisfactory performance in coursework, by demonstrating core
competencies on a written comprehensive exam and by demonstrating the ability to communicate
knowledge relevant to the particular field of study. The nature of the program determines
the structure of the assessment for these various programs, and the threshold values
and data sources are described in Table 2 below.
|
Ph.D. Programs
|
|
---|---|---|
PROGRAM LEARNING OUTCOME
|
Threshold Value
|
Data Source
|
Demonstrate a solid understanding of [PhD core content appropriate to each degree,
as listed in program outcomes]
|
75%* of students who begin the degree program will pass the [appropriate] written comprehensive exam within 2 years
|
Milestone 1
Written comprehensive exam in additional specialty area |
Formulate new research problems
|
Of those students who have achieved Milestone 1, 75%* will pass Milestone 2 within
2 years of the term in which
Milestone 1 was achieved. |
Milestone 2
Oral comprehensive exam |
Clearly communicate [original research appropriate to each PhD degree] both orally
and in writing
|
Of those students who have achieved Milestone 2, 75%* will pass Milestone 3 within
2 years of the term in which
Milestone 2 was achieved. |
Milestone 3
Defense of dissertation |
*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.
|
M.S. Programs
|
|
---|---|---|
PROGRAM LEARNING OUTCOME
|
Threshold Value
|
Data Source
|
MS Math and MS Stat
Demonstrate a solid understanding of [MS core content appropriate to the degree] |
75%* of students who begin the degree program will pass the [appropriate] written
comprehensive exam within 2
years. |
Written comprehensive exam
|
MS Math and MSMME
Demonstrate a solid understanding of [MS core content appropriate to the degree] |
75%* of students who begin the MS degree will earn a B or better in
[appropriate] coursework |
Coursework
|
MSMME
Clearly communicate connections between program coursework and local classroom practice |
75%* of students who begin the MSMME will present a portfolio within 3 years
|
Portfolio and presentation
|
MS Stat
Be prepared for career as an applied statistician or a doctoral program in statistics |
75%* of students who begin the MS Statistics program will complete 2 credits of Stat
510 with a B or better in 3
years |
Coursework
|
MS Stat
Clearly communicate results from a statistical data analysis or research problem both orally and in writing (MS Stat only) |
75%* of students of begin the MS Statistics program will complete a writing project
or thesis within 3 years
|
Writing project or thesis and presentation
|
*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.
4. What Was Done.
a) Self-reporting Metric (required answer): Was the completed assessment consistent with the program’s assessment plan? If not, please explain the adjustments that were made.
X Yes
b) How were data collected and analyzed and by whom? Please include method of collection and sample size.
Data were collected from enrolled student records and summarized in tabular form. We used a census of enrolled students. The data were analyzed by the department’s graduate program committee. Additionally, the Graduate Program Committee meets at least monthly throughout the academic year and engages in regular communication with the entire department faculty to discuss the programs, curriculum, and adjustments.
We use enrollment summaries, as follows.
Enrollment Summaries by Program
PhD Mathematics
Twenty students were enrolled in the PhD Mathematics program at some point during
the assessment period; three left the program for personal or professional reasons,
so 17 students are included in this summary. Of those 17, 9 were eligible to complete Milestone 1. Eight of the 9 (89%) met the milestone. During the review period, two students were
eligible to complete Milestone 2, and both (100%) met the milestone. During the review period, four students were eligible
to complete Milestone 3, but one student is working full time while completing the dissertation, so the normal
timelines do not apply. Of the three remaining, two (67%) passed their dissertation
within two years of the dissertation proposal.
Qualitative analysis of Milestones 2 and 3: All eligible students met Milestone 2,
so we consider that the program learning outcome is met. For Milestone 3, the student
who did not meet the timeline was still successful in producing a dissertation, so
while the timeline was not met, we consider the program learning outcome to be met.
PhD Mathematics – Mathematics Education Option
Seven students were enrolled in the PhD Mathematics – Mathematics Education emphasis program at some point during the assessment period. One student was eligible to complete Milestone 1 and was successful (100%). Three students were eligible to complete Milestone 2 and all were successful (100%). Two students were eligible to complete Milestone 3 and all were successful (100%).
PhD Statistics
Fourteen students were enrolled in the PhD Statistics program during the assessment
period. One student was eligible to complete Milestone 1 and did not meet the timeline (0%). Four students were eligible to complete Milestone 2, and 3 (75%) met the milestone. Two students were eligible to meet Milestone 3, and one (50%) met the milestone.
Qualitative analysis of Milestones 1 and 3. For Milestone 1, the student has attempted
and passed one component of the written exam, but has not yet attempted the second
component. While the timeline was not met, we consider the program learning outcome
to be met. For Milestone 3, the student who did not meet the timeline was still successful
in producing a dissertation, so while the timeline was not met, we consider the program
learning outcome to be met.
MS Mathematics
A total of twenty-eight students (not including 4 students who earned masters en route to PhD) were enrolled in the MS Mathematics program for some portion of the assessment period; one left the program for personal reasons, so 27 students are included in this summary. Of those 27, 21 were eligible to complete Milestone 1, and 21 (100%) met the milestone. Twenty-one were eligible to complete Milestone 2, and 20 (95%) met the milestone.
MSMME
Thirty-four students were enrolled in the Master of Science in Mathematics – Mathematics Education Option during the assessment period. Of those enrolled, 26 had progressed to a point in their program where we would expect them to have achieved our program learning outcomes and all 26 achieved Milestones 1 & 2 (100%).
MS Statistics
Twenty-nine students were enrolled in the MS Statistics program during the assessment period. Of those 29, 19 were eligible to complete Milestone 1 and 18 (95%) were successful. Nineteen were eligible to complete Milestone 2 and Milestone 3, and 100% were successful.
MS Data Science
Thirteen students were enrolled in the MS Data Science program during the assessment period. Of those, ten were eligible to complete Milestone 1, and all (100%) were successful. Ten were eligible to complete Milestone 2, and all (100%) were successful. Eleven were eligible to complete Milestone 3, and 7 were successful (64%).
Graduate Certificate in Applied Statistics
Of the 2 students enrolled in the program within the assessment period, 1 completed the required coursework for the degree (100%) and 1 continues in the program.
c) Please provide a rubric that demonstrates how your data were evaluated.
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate competence in graduate-level real analysis and linear algebra.
|
Completion rate of Milestone 1 – written comprehensive exam in
Real and Complex Analysis |
Met (89%)
|
Demonstrate a solid understanding of core mathematical concepts in at least one area
of specialty
|
Completion rate of Milestone 1 - written comprehensive exam in an
additional specialty area |
Met (89%)
|
Formulate new research problems
|
Completion rate of Milestone 2
|
Met (100%)
|
Clearly communicate mathematical research both orally and in writing
|
Completion rate of Milestone 3
|
Met – qualitative analysis of success in clear communication
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate a solid understanding of core graduate level mathematics
|
Completion rate of Milestone 1 - written comprehensive exam in a mathematical topic
area
|
Met (100%)
|
Formulate questions and design studies to address contemporary issues in mathematics
education
|
Completion rate of dissertation proposal
|
Met (100%)
|
Clearly communicate mathematics education research both orally and in writing
|
Completion rate of dissertation defense
|
Met (100%)
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate a solid understanding of advanced mathematical statistics, Bayesian statistics,
and data analysis methods.
|
Completion rate of Milestone 1 - written comprehensive exam components in advanced
mathematical statistics, Bayesian statistics and data analysis
|
Met – qualitative analysis
|
Demonstrate a solid understanding of core statistical content in at least one research
area of specialty
|
Completion rate of Milestone 1 - written comprehensive exam component in a research
area of specialty
|
Met – qualitative analysis
|
Formulate new research problems
|
Completion rate of dissertation proposal
|
Met (75%)
|
Clearly communicate original statistical research both orally and in writing
|
Completion rate of dissertation defense
|
Met – qualitative analysis
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate solid understanding of graduate level real analysis and advanced linear
algebra
|
Completion rate of required written comprehensive exam or thesis
defense. |
Met (100%)
|
Demonstrate solid understanding of core mathematical concepts in at least one area
of specialty
|
Completion rate of M 511 and M 504
|
Met (95%)
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate solid understanding of graduate level mathematics relevant to secondary
content
|
Completion rate of content courses
|
Met (100%)
|
Demonstrate solid understanding of teaching practices that give every student access
to rigorous mathematics learning
|
Completion rate of 2 required pedagogy courses (M 520, 528, 529 or 577)
|
Met (100%)
|
Clearly communicate connections between program coursework and local
classroom practice |
Completion rate of written portfolio and public presentation
|
Met (100%)
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Possess a solid understanding of core graduate level applied statistics, probability,
and mathematical
statistics |
Completion rate of written comprehensive exam
|
Met (95%)
|
Be prepared for career as an applied statistician or a doctoral program in statistics
|
Completion rate of Statistical Consulting Seminar (Stat 510)
|
Met (100%)
|
Clearly communicate results from a statistical data analysis or research problem both
orally and in writing
|
Completion rate of Writing Project or Thesis
|
Met (100%)
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate knowledge of essential deterministic, randomized and approximation algorithms
for data classification and clustering, dimensionality reduction,
regression, optimization. |
Completion of M 508
|
Met (100%)
|
Demonstrate knowledge in the principles and practice of statistical experimental design,
statistical inference, and decision theory.
|
Completion of STAT 541
|
Met (100%)
|
Demonstrate the ability to take a real-world data analysis problem, formulate and
conceptual approach to the problem, match aspects of the problem to previously learned
theoretical and methodological tools, break down the solution into step-by-step approach,
and implement a working solution
in a modern software language. |
Completion of CSCI 531
|
Not Met (64%)
|
Communicate data science problems, analyses, and solutions effectively to both specialists
and non-specialists through the use of effective technical writing, presentations,
and data visualizations and
teamwork and collaboration. |
Program completion
|
Met (8 graduates of the program)
|
PROGRAM LEARNING OUTCOMES
|
Data Source*
|
Outcome
|
---|---|---|
Demonstrate advanced statistical thinking and data collection.
|
Completion rate of STAT 511
|
Met (100%)
|
Apply advanced statistical methodology.
|
Completion rate of STAT 512
|
Met (100%)
|
5. What Was Learned.
a) Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment? What areas of strength in the program were identified from this assessment process?
Our PhD programs are successful at meeting program learning outcomes, though sometimes
the timeline takes longer than our Milestones. Still, we find that our recent focus
pairing timelines with PLOs has helped keep our attention on supporting students to
make timely progress through the degree.
Our MS Math, Math Ed, and Stat programs are successful at meeting program learning
outcomes and at meeting timelines.
Our MS Data Science program requires more attention. For the second consecutive biennial
assessment report, we have not met the learning outcome threshold for the computer
science Algorithms foundation. We will provide focused attention on how to address
this in the upcoming two years, with a particular look at admissions standards and
prerequisite fulfillment for that course.
b) What areas were identified that either need improvement or could be improved in a different way from this assessment process?
We are continually aware that our core graduate programs in mathematics, mathematics education, and statistics receive focused attention in a way that our data science program, as the newest program, does not. As documented in our most recent program review, the dispersed nature of the coursework across three disciplines, and two departments & colleges, means that the program needs more focused attention.
6. How We Responded.
a) Describe how “What Was Learned” was communicated to the department, or program faculty. How did faculty discussions re-imagine new ways program assessment might contribute to program growth/improvement/innovation beyond the bare minimum of achieving program learning objectives through assessment activities conducted at the course level?
The Graduate Program Committee met and discussed the assessment process and results and led a discussion among the department. The Department has engaged in various program updates over the past two years, adjusting requirements, updating learning program outcomes, adjusting admissions processes by involving more faculty, adjusting advising practices, adjusting comprehensive and qualifying exam requirements and practices, and maintaining a focus on our students as those who are part of our vision of using the tools of our disciplines to promote human flourishing. In particular, to help students meet milestones, we have focused our attention on supporting students as individuals and engaging in collective conversations to discuss milestones in relation to each student’s academic progress and overall wellbeing.
b) How are the results of this assessment informing changes to enhance student learning in the program?
Reflecting on these results gives us confidence that our practice of ongoing discussion and adjustment is a positive for our programs. It reminds us to continue to focus on the Data Science program.
C) If information outside of this assessment is informing programmatic change, please describe that.
See part a.
d) What support and resources (e.g. workshops, training, etc.) might you need to make these adjustments?
We took part in a discussion with the assistant provost responsible for assessment and found that informal discussion useful.
7. Closing the Loop(s).
Reflect on the program learning outcomes, how they were assessed in the previous cycle (refer to #1 of the report), and what was learned in this cycle. What action will be taken to improve student learning objectives going forward?
a) Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?
X Yes
b) In reviewing the last report that assessed the PLO(s) in this assessment cycle, what changes proposed were implemented and will be measured in future assessment reports?
We updated a program learning outcome in the PhD mathematics program: we replaced “Demonstrate a solid understanding of core graduate level real and complex analysis” with “Demonstrate competence in graduate-level real analysis and linear algebra”, to reflect the shift we made in reflecting on our last set of PLOs and required coursework for the degree.
We adjusted how we use threshold values in the case of low numbers of students, and specifying a reliance on qualitative analysis has given us a useful mechanism for meaningful reflection in these circumstances.
Going forward, we intend to update the course requirements of our Math MS program for the following effects.
- Better reflect existing faculty expertise.
- Better prepare Math MS students for a PhD program in a mathematically-related
field. - Align the experiences of typical 1st year Math MS and typical 1st year Math PhD
students.
In the M.S. Statistics program, we intend to examine the alignment between comprehensive exam practices and course content and learning outcomes.
In the PhD Math Ed program, we intend to examine coursework to align it with current trends among peer programs and with faculty expertise.
c) Have you seen a change in student learning based on other program adjustments made in the past? Please describe the adjustments made and subsequent changes in student learning.
This assessment process also lets us reflect on how and which students move from the MS to the PhD (in non-published data tables). This is useful, because recruiting for graduate programs remains a priority for the department.