Program Assessment Report

Academic Year(s) Assessed: AY2023-24
College: College of Letters and Science
Department: Mathematical Sciences
Department Head: Elizabeth Burroughs
Submitted by: Elizabeth Burroughs

Program(s) Assessed

Program Information
Program
Degree/s Assessed 

Ph.D. Mathematics
Ph.D. Mathematics(education emphasis) 
Ph.D. Statistics


M.S. Data Science
M.S. Mathematics
M.S. Mathematics – Mathematics Education Option
M.S. Statistics


Certificate in Applied Statistics
Certificate for Dual Enrollment Mathematics Teachers

College or Administrative Division 
College of Letters and Science
Report Submitted By
Mathematical Sciences
Date Submitted
Elizabeth Burroughs, Department Head
Assessment Period:
May 16th, 2023 - May 15th, 2024

Graduate assessment reports are to be submitted biennially.  The report deadline is October 15th.

Biennial Graduate Assessment Process:

Every graduate program assessment must have the following key components:

Program Description:  

Ph.D. Mathematics: This program provides graduates with proficiency in mathematics and the 
opportunity to carry out independent research in the mathematical sciences as demonstrated by the 
completion of a doctoral dissertation. 
Ph.D. Mathematics, Mathematics Education emphasis: This program provides the opportunity for 
research focused on mathematics teaching and learning and includes the study of graduate-level 
mathematics. 
Ph.D. Statistics: This program provides graduates with proficiency in statistics and the opportunity to 
carry out independent research in statistics as demonstrated by the completion of a doctoral 
dissertation. 
M.S. Mathematics: This program provides fundamental knowledge in core areas of pure and applied 
mathematics. It prepares graduates for careers in industry and for a PhD program in mathematics or 
applied mathematics. 
M.S. Mathematics, Mathematics Education Option: This program provides fundamental knowledge for 
secondary mathematics teaching. The program deepens graduates’ understanding of school 
mathematics, increases their pedagogical content knowledge, and provides opportunities for 
professional reflection growth. 
M.S. Statistics: This program gives students a background in the theory of statistics and hands-on 
practice in the application of statistics to real problems. Students in this program prepare either for 
further graduate work or for academic, industrial, business, or government employment. 
M.S. Data Science: This program provides graduate students with foundational training in data analysis, 
with equal emphasis on the principles of computer science, mathematics, and statistics, and the ability 
to apply these principles to a range of data-driven problems. 
Graduate Certificate in Statistics: This certificate program provides additional education in statistical 
thinking and methodology over and above the basic coursework taken by the typical graduate student. 
This certificate provides a clear record of additional training in statistics for future graduate programs or 
employers. 
Graduate Certificate for Dual Enrollment Mathematics Teachers: This certificate program provides a set 
of three courses in mathematics that provide foundational knowledge and study in algebra, calculus, 
and statistics. This program has been discontinued since AY2023-2024, due to non-enrollment. For that 
reason, it is not included in this or any future assessment report and is only included on this list for 
completeness.

1. Past Assessment Summary:  

In our last Graduate Program Assessment report, we reported on successes in meeting our program learning outcomes. We largely met our targets and made notes of the following areas we intended to improve in our assessment process or in our programs.

  • Our assessment relies on percentages, which is not informative when numbers of students at a particular stage of progress are low. We have changed that reporting this time.
  • We did not have a mechanism for reporting reasons students leave the program. We have incorporated that this time.
  •  We recommended implementing exit interviews for MSDS students to improve assessment.
  •  We recommended implementing exit interviews for other graduate programs as well.

2. Action Research Question:  

Do our students progress through our graduate programs with the rigor we expect, according to the timeline we expect? In what ways does our curriculum support or hinder this progress?

3. Assessment Plan, Schedule, and Data Sources:  

a)  Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data).

PhD Programs

PhD Mathematics

PROGRAM LEARNING OUTCOMES

2022-23
2023-24
Data Source
Demonstrate competence in graduate-level real analysis and linear algebra.
X
X
Completion rate of required written qualifying exam in linear algebra and real analysis
Demonstrate a solid understanding of core mathematical concepts in at least one area of specialty
X
X
Completion rate of required written comprehensive exam in an additional area
Formulate new research problems
X
X
Completion rate of dissertation proposal
Clearly communicate mathematical research both orally and in writing 
X
X

Completion rate of dissertation defense 

PhD Mathematics (Education)

PROGRAM LEARNING OUTCOMES

2022-23
2023-24

Data Source

Demonstrate a solid understanding of core graduate level mathematics

X
X

Completion rate of required written comprehensive exam in a mathematical topic area

Formulate questions and design studies to address contemporary issues in mathematics education 

X
X

Completion rate of dissertation proposal 

Clearly communicate mathematics education research both orally and in writing 

X
X

Completion rate of dissertation defense 

PhD Statistics 
PROGRAM LEARNING OUTCOMES

2022-23
2023-24

Data Source

Demonstrate a solid understanding of probability and advanced mathematical statistics 

X
X

Completion rate of required written comprehensive exam in probability and advanced mathematical statistics

Demonstrate a solid understanding of core statistical content in at least one research area of specialty

X
X

Completion rate of required written comprehensive exam in an additional area

Formulate new research problems

X
X

Completion rate of dissertation proposal 

Clearly communicate original statistical research both orally and in writing

X
X

Completion rate of dissertation defense 

Masters Programs
MS Mathematics
PROGRAM LEARNING OUTCOMES
2022-23
2023-24
Data Source

Demonstrate solid understanding of graduate level real analysis and advanced linear algebra

X
X
Completion rate of required written comprehensive exam or MS Thesis defense
Demonstrate solid understanding of core mathematical concepts in at least one area of specialty
X
X
Completion rate of M 511 and M 504
MS Mathematics (education)
2022-23
2023-24
Data Source
Demonstrate solid understanding of graduate level mathematics relevant to secondary content in algebra, calculus,
geometry and statistics
X
X
Completion rate of 4 required content area courses 
(M 518, 524, 525, 527)
Demonstrate solid understanding of teaching practices that give every student access to rigorous mathematics learning
X
X
Completion rate of 2 required pedagogy courses 
(M 520, 528, 529 or 577)
Clearly communicate connections between program coursework and local classroom practice
X
X
Completion rate of written portfolio and public 
presentation
MS Data Science
PROGRAM LEARNING OUTCOMES
2022-23
2023-24
Data Source
Demonstrate knowledge of essential deterministic, randomized and approximation algorithms for data classification and clustering, dimensionality reduction, regression, optimization
X
X
Completion rate of M 508
Demonstrate knowledge in the principles and practice of statistical experimental design, statistical inference, and decision theory
X
X
Completion rate of STAT 541
Demonstrate the ability to take a real world data analysis problem, formulate and conceptual approach to the problem, match aspects of the problem to previously learned theoretical and methodological tools, break down the solution into step-by- step approach, and implement a working solution in a modern software language.
X
X
Completion rate of CSCI 531
Communicate data science problems, analyses, and solutions effectively to both specialists and non-specialists through the use of effective technical writing, presentations, and data visualizations and teamwork and collaboration.
X
X
*This could be assessed by a capstone requirement, 
which is not currently available.
MS Statistics
PROGRAM LEARNING OUTCOMES
2022-23
2023-24
Data Source
Possess a solid understanding of core graduate level applied statistics, probability, and mathematical statistics
X
X
Completion rate of Comprehensive exam
Be prepared for career as an applied statistician or a doctoral program in statistics
X
X
Completion rate of Statistical Consulting Seminar 
(Stat 510)
Clearly communicate results from a statistical data analysis or research problem both orally and in writing
X
X
Completion of Writing Project or Thesis
Graduate Certificate in Applied Statistics
PROGRAM LEARNING OUTCOMES
2022-23
2023-24
Data Source
Demonstrate advanced statistical thinking and data collection
X
X
Completion rate of STAT 511
Apply advanced statistical methodology
X
X
Completion rate of STAT 512

b)  What are the threshold values for which your program demonstrates student achievement?

The PhD programs in Mathematics, Mathematics with Education emphasis, and Statistics are assessed by measuring the completion rates of students who advance through the program using a sequence of Milestones. The number of students who complete a given milestone is measured. The time required for each student to advance between successive milestones is also measured. The threshold values and data sources in Table 1 below incorporate both quantities into the assessment.


The MS programs in Mathematics, Mathematics Education, Statistics and Data Science are assessed by measuring the completion rates of students who advance through their program by achieving satisfactory performance in coursework, by demonstrating core competencies on a written comprehensive exam and by demonstrating the ability to communicate knowledge relevant to the particular field of study. The nature of the program determines the structure of the assessment for these various programs, and the threshold values and data sources are described in Table 2 below.

Table 1: Threshold Values and Data Sources, Ph.D.
 
Ph.D. Programs
 
PROGRAM LEARNING OUTCOME
Threshold Value
Data Source
Demonstrate a solid understanding of [PhD core content appropriate to each degree, as listed in program outcomes]

75%* of students who begin the degree program will pass the [appropriate] written comprehensive exam within 2 years


75%* of students who begin the degree will pass the written comprehensive exam in an additional specialty area
within 2 years

Milestone 1
Written qualifying exam in core area

 

Written comprehensive exam in additional specialty area

Formulate new research problems
Of those students who have achieved Milestone 1, 75%* will pass Milestone 2 within 2 years of the term in which
Milestone 1 was achieved.
Milestone 2
Oral comprehensive exam
Clearly communicate [original research appropriate to each PhD degree] both orally and in writing
Of those students who have achieved Milestone 2, 75%* will pass Milestone 3 within 2 years of the term in which
Milestone 2 was achieved.
Milestone 3
Defense of dissertation

*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.

Table 2:  Threshold Values and Data Sources, M.S.
 
M.S. Programs
 
PROGRAM LEARNING OUTCOME
Threshold Value
Data Source
MS Math and MS Stat
Demonstrate a solid understanding of [MS core content appropriate to the degree]
75%* of students who begin the degree program will pass the [appropriate] written comprehensive exam within 2
years.
Written comprehensive exam
MS Math and MSMME
Demonstrate a solid understanding of [MS core content appropriate to the degree]
75%* of students who begin the MS degree will earn a B or better in
[appropriate] coursework
Coursework
MSMME
Clearly communicate connections between program coursework and local classroom practice
75%* of students who begin the MSMME will present a portfolio within 3 years
Portfolio and presentation
MS Stat
Be prepared for career as an applied statistician or a doctoral program in statistics
75%* of students who begin the MS Statistics program will complete 2 credits of Stat 510 with a B or better in 3
years
Coursework
MS Stat
Clearly communicate results from a statistical data analysis or research problem both orally and in writing (MS Stat only)
75%* of students of begin the MS Statistics program will complete a writing project or thesis within 3 years
Writing project or thesis and presentation

*If the number of students eligible to complete a Milestone is less than 4, then qualitative rather than quantitative analysis will be used to examine the program learning outcome.

4. What Was Done.  

a)  Self-reporting Metric (required answer): Was the completed assessment consistent with the program’s assessment plan? If not, please explain the adjustments that were made.


                                                                        X Yes

b)  How were data collected and analyzed and by whom? Please include method of collection and sample size.

Data were collected from enrolled student records and summarized in tabular form. We used a census of enrolled students. The data were analyzed by the department’s graduate program committee. Additionally, the Graduate Program Committee meets at least monthly throughout the academic year and engages in regular communication with the entire department faculty to discuss the programs, curriculum, and adjustments.

We use enrollment summaries, as follows.

Enrollment Summaries by Program

PhD Mathematics

Twenty students were enrolled in the PhD Mathematics program at some point during the assessment period; three left the program for personal or professional reasons, so 17 students are included in this summary. Of those 17, 9 were eligible to complete Milestone 1. Eight of the 9 (89%) met the milestone. During the review period, two students were eligible to complete Milestone 2, and both (100%) met the milestone. During the review period, four students were eligible to complete Milestone 3, but one student is working full time while completing the dissertation, so the normal timelines do not apply. Of the three remaining, two (67%) passed their dissertation within two years of the dissertation proposal.
Qualitative analysis of Milestones 2 and 3: All eligible students met Milestone 2, so we consider that the program learning outcome is met. For Milestone 3, the student who did not meet the timeline was still successful in producing a dissertation, so while the timeline was not met, we consider the program learning outcome to be met.

 

PhD Mathematics – Mathematics Education Option

Seven students were enrolled in the PhD Mathematics – Mathematics Education emphasis program at some point during the assessment period. One student was eligible to complete Milestone 1 and was successful (100%). Three students were eligible to complete Milestone 2 and all were successful (100%). Two students were eligible to complete Milestone 3 and all were successful (100%).

 

PhD Statistics

Fourteen students were enrolled in the PhD Statistics program during the assessment period. One student was eligible to complete Milestone 1 and did not meet the timeline (0%). Four students were eligible to complete Milestone 2, and 3 (75%) met the milestone. Two students were eligible to meet Milestone 3, and one (50%) met the milestone.
Qualitative analysis of Milestones 1 and 3. For Milestone 1, the student has attempted and passed one component of the written exam, but has not yet attempted the second component. While the timeline was not met, we consider the program learning outcome to be met. For Milestone 3, the student who did not meet the timeline was still successful in producing a dissertation, so while the timeline was not met, we consider the program learning outcome to be met.

 

MS Mathematics

A total of twenty-eight students (not including 4 students who earned masters en route to PhD) were enrolled in the MS Mathematics program for some portion of the assessment period; one left the program for personal reasons, so 27 students are included in this summary. Of those 27, 21 were eligible to complete Milestone 1, and 21 (100%) met the milestone. Twenty-one were eligible to complete Milestone 2, and 20 (95%) met the milestone.

 

MSMME

Thirty-four students were enrolled in the Master of Science in Mathematics – Mathematics Education Option during the assessment period. Of those enrolled, 26 had progressed to a point in their program where we would expect them to have achieved our program learning outcomes and all 26 achieved Milestones 1 & 2 (100%).

 

MS Statistics

Twenty-nine students were enrolled in the MS Statistics program during the assessment period. Of those 29, 19 were eligible to complete Milestone 1 and 18 (95%) were successful. Nineteen were eligible to complete Milestone 2 and Milestone 3, and 100% were successful.

 

MS Data Science

Thirteen students were enrolled in the MS Data Science program during the assessment period. Of those, ten were eligible to complete Milestone 1, and all (100%) were successful. Ten were eligible to complete Milestone 2, and all (100%) were successful. Eleven were eligible to complete Milestone 3, and 7 were successful (64%).

 

Graduate Certificate in Applied Statistics

Of the 2 students enrolled in the program within the assessment period, 1 completed the required coursework for the degree (100%) and 1 continues in the program.

c)  Please provide a rubric that demonstrates how your data were evaluated.

PhD Mathematics
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate competence in graduate-level real analysis and linear algebra.
Completion rate of Milestone 1 – written comprehensive exam in
Real and Complex Analysis
Met (89%)
Demonstrate a solid understanding of core mathematical concepts in at least one area of specialty
Completion rate of Milestone 1 - written comprehensive exam in an
additional specialty area
Met (89%)
Formulate new research problems
Completion rate of Milestone 2
Met (100%)
Clearly communicate mathematical research both orally and in writing
Completion rate of Milestone 3
Met – qualitative analysis of success in clear communication

 

PhD Mathematics (Education)
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate a solid understanding of core graduate level mathematics
Completion rate of Milestone 1 - written comprehensive exam in a mathematical topic area
Met (100%)
Formulate questions and design studies to address contemporary issues in mathematics education
Completion rate of dissertation proposal
Met (100%)
Clearly communicate mathematics education research both orally and in writing
Completion rate of dissertation defense
Met (100%)

 

PhD Statistics
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate a solid understanding of advanced mathematical statistics, Bayesian statistics, and data analysis methods.
Completion rate of Milestone 1 - written comprehensive exam components in advanced mathematical statistics, Bayesian statistics and data analysis
Met – qualitative analysis
Demonstrate a solid understanding of core statistical content in at least one research area of specialty
Completion rate of Milestone 1 - written comprehensive exam component in a research area of specialty
Met – qualitative analysis
Formulate new research problems
Completion rate of dissertation proposal
Met (75%)
Clearly communicate original statistical research both orally and in writing
Completion rate of dissertation defense
Met – qualitative analysis

 

MS Mathematics
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate solid understanding of graduate level real analysis and advanced linear algebra
Completion rate of required written comprehensive exam or thesis
defense.
Met (100%)
Demonstrate solid understanding of core mathematical concepts in at least one area of specialty
Completion rate of M 511 and M 504
Met (95%)

 

MSMME
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate solid understanding of graduate level mathematics relevant to secondary content
Completion rate of content courses
Met (100%)
Demonstrate solid understanding of teaching practices that give every student access to rigorous mathematics learning
Completion rate of 2 required pedagogy courses (M 520, 528, 529 or 577)
Met (100%)
Clearly communicate connections between program coursework and local
classroom practice
Completion rate of written portfolio and public presentation
Met (100%)

 

MS Statistics
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Possess a solid understanding of core graduate level applied statistics, probability, and mathematical
statistics
Completion rate of written comprehensive exam
Met (95%)
Be prepared for career as an applied statistician or a doctoral program in statistics
Completion rate of Statistical Consulting Seminar (Stat 510)
Met (100%)
Clearly communicate results from a statistical data analysis or research problem both orally and in writing
Completion rate of Writing Project or Thesis
Met (100%)

 

MS Data Science
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate knowledge of essential deterministic, randomized and approximation algorithms for data classification and clustering, dimensionality reduction,
regression, optimization.
Completion of M 508
Met (100%)
Demonstrate knowledge in the principles and practice of statistical experimental design, statistical inference, and decision theory.
Completion of STAT 541
Met (100%)
Demonstrate the ability to take a real-world data analysis problem, formulate and conceptual approach to the problem, match aspects of the problem to previously learned theoretical and methodological tools, break down the solution into step-by-step approach, and implement a working solution
in a modern software language.
Completion of CSCI 531
Not Met (64%)
Communicate data science problems, analyses, and solutions effectively to both specialists and non-specialists through the use of effective technical writing, presentations, and data visualizations and
teamwork and collaboration.
Program completion
Met (8 graduates of the program)

 

Graduate Certificate in Applied Statistics
PROGRAM LEARNING OUTCOMES
Data Source*
Outcome
Demonstrate advanced statistical thinking and data collection.
Completion rate of STAT 511
Met (100%)
Apply advanced statistical methodology.
Completion rate of STAT 512
Met (100%)

5. What Was Learned.

a)  Based on the analysis of the data, and compared to the threshold values established, what was learned from the assessment? What areas of strength in the program were identified from this assessment process?


Our PhD programs are successful at meeting program learning outcomes, though sometimes the timeline takes longer than our Milestones. Still, we find that our recent focus pairing timelines with PLOs has helped keep our attention on supporting students to make timely progress through the degree.


Our MS Math, Math Ed, and Stat programs are successful at meeting program learning outcomes and at meeting timelines.


Our MS Data Science program requires more attention. For the second consecutive biennial assessment report, we have not met the learning outcome threshold for the computer science Algorithms foundation. We will provide focused attention on how to address this in the upcoming two years, with a particular look at admissions standards and prerequisite fulfillment for that course.

 

b) What areas were identified that either need improvement or could be improved in a different way from this assessment process?

We are continually aware that our core graduate programs in mathematics, mathematics education, and statistics receive focused attention in a way that our data science program, as the newest program, does not. As documented in our most recent program review, the dispersed nature of the coursework across three disciplines, and two departments & colleges, means that the program needs more focused attention. 

6. How We Responded.

a)  Describe how “What Was Learned” was communicated to the department, or program faculty. How did faculty discussions re-imagine new ways program assessment might contribute to program growth/improvement/innovation beyond the bare minimum of achieving program learning objectives through assessment activities conducted at the course level?

The Graduate Program Committee met and discussed the assessment process and results and led a discussion among the department. The Department has engaged in various program updates over the past two years, adjusting requirements, updating learning program outcomes, adjusting admissions processes by involving more faculty, adjusting advising practices, adjusting comprehensive and qualifying exam requirements and practices, and maintaining a focus on our students as those who are part of our vision of using the tools of our disciplines to promote human flourishing. In particular, to help students meet milestones, we have focused our attention on supporting students as individuals and engaging in collective conversations to discuss milestones in relation to each student’s academic progress and overall wellbeing. 

 

b)  How are the results of this assessment informing changes to enhance student learning in the program?

Reflecting on these results gives us confidence that our practice of ongoing discussion and adjustment is a positive for our programs. It reminds us to continue to focus on the Data Science program.

 

C)  If information outside of this assessment is informing programmatic change, please describe that.

See part a.

 

d)  What support and resources (e.g. workshops, training, etc.) might you need to make these adjustments?

We took part in a discussion with the assistant provost responsible for assessment and found that informal discussion useful.

7. Closing the Loop(s).

Reflect on the program learning outcomes, how they were assessed in the previous cycle (refer to #1 of the report), and what was learned in this cycle. What action will be taken to improve student learning objectives going forward?

 

a)  Self-Reporting Metric (required answer): Based on the findings and/or faculty input, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?

X Yes

b) In reviewing the last report that assessed the PLO(s) in this assessment cycle, what changes proposed were implemented and will be measured in future assessment reports?

We updated a program learning outcome in the PhD mathematics program: we replaced “Demonstrate a solid understanding of core graduate level real and complex analysis” with “Demonstrate competence in graduate-level real analysis and linear algebra”, to reflect the shift we made in reflecting on our last set of PLOs and required coursework for the degree. 

We adjusted how we use threshold values in the case of low numbers of students, and specifying a reliance on qualitative analysis has given us a useful mechanism for meaningful reflection in these circumstances. 

Going forward, we intend to update the course requirements of our Math MS program for the following effects.

  1. Better reflect existing faculty expertise.
  2. Better prepare Math MS students for a PhD program in a mathematically-related 
    field.
  3. Align the experiences of typical 1st year Math MS and typical 1st year Math PhD 
    students.

In the M.S. Statistics program, we intend to examine the alignment between comprehensive exam practices and course content and learning outcomes. 

In the PhD Math Ed program, we intend to examine coursework to align it with current trends among peer programs and with faculty expertise.

c)  Have you seen a change in student learning based on other program adjustments made in the past? Please describe the adjustments made and subsequent changes in student learning.

This assessment process also lets us reflect on how and which students move from the MS to the PhD (in non-published data tables). This is useful, because recruiting for graduate programs remains a priority for the department.

PDF of Graduate Program Assessment Report