Graduate Program Assessment Report
Assessment Report
AY: 2018-2020
College: College of Letters and Science
Department: Mathematical Sciences
Submitted by: Elizabeth Burroughs, Department Head
1 Program Description
Programs Assessed
|
PhD Programs
|
MS Programs
|
|---|---|
|
Ph.D. Mathematics
|
M.S. Mathematics
|
|
Ph.D. Mathematics (education emphasis)
|
M.S. Mathematics (Math Education option)
|
|
Ph.D. Statistics
|
M.S. Statistics
|
Ph.D. Mathematics:
This program provides graduates proficiency in mathematics and the opportunity to carry out independent research in the mathematical sciences as demonstrated by the completion of a doctoral dissertation.
Ph.D. Mathematics, Mathematics Education emphasis:
This program is aimed at students with research interests focused on mathematics teaching and learning and includes the study of graduate level mathematics.
Ph.D. Statistics:
This program provides graduates proficiency in statistics and the opportunity to carry out independent research in statistics as demonstrated by the completion of a doctoral dissertation.
M.S. Mathematics:
This program provides fundamental knowledge in core areas of pure and applied mathematics. It prepares graduates for careers in industry and for a PhD program in mathematics or applied mathematics.
M.S. Mathematics, Mathematics Education Option:
This program provides fundamental knowledge for secondary mathematics teaching. The program deepens graduates’ understanding of school mathematics, increases their pedagogical content knowledge, and provides opportunities for personal reflection and professional growth.
M.S. Statistics:
This program gives students a solid background in the theory of statistics and hands-on practice in the application of statistics to real problems. Students in this program prepare either for further graduate work or for academic, industrial, business, or government employment.
2 Program Learning Outcomes, Assessment Schedule, and Methods of Assessment
PhD Mathematics
| # | PhD Mathematics PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Demonstrate a solid understanding of core graduate level real and complex analysis | X | X | Completion rate of required written comprehensive exam in real and complex analysis |
| 2 | Demonstrate a solid understanding of core mathematical concepts in at least one area of specialty | X | X | Completion rate of required written comprehensive exam in an additional specialty area |
| 3 | Formulate new research problems | X | X | Completion rate of dissertation proposal |
| 4 | Clearly communicate mathematical research both orally and in writing | X | X | Completion rate of dissertation defense |
PhD Mathematics Education
| # | PhD Mathematics Education PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Demonstrate a solid understanding of core graduate level mathematics |
X | X | Completion rate of required written comprehensive exam in a mathematical topic area |
| 2 | Formulate questions and design studies to address contemporary issues in mathematics education |
X | X | Completion rate of dissertation proposal |
| 3 | Clearly communicate mathematics education research both orally and in writing |
X | X | Completion rate of dissertation defense |
PhD Statistics
| # | PhD Statistics PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Demonstrate a solid understanding of advanced mathematical statistics, Bayesian statistics, and data analysis methods | X | X | Completion rate of required written comprehensive exam components in advanced mathematical statistics. Bayesian statistics, and data analysis methods |
| 2 | Demonstrate a solid understanding of core statistical content in at least one research area of specialty | X | X | Completion rate of required written comprehensive exam component in a research area of specialty |
| 3 | Formulate new research problems | X | X | Completion rate of dissertation proposal |
| 4 | Clearly communicate original statistical research both orally and in writing | X | X | Completion rate of dissertation defense |
MS Mathematics
| # | MS Mathematics PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Demonstrate solid understanding of graduate level real analysis and advanced linear algebra | X | X | Completion rate of required written comprehensive exam |
| 2 | Demonstrate solid understanding of core mathematical concepts in at least one area of specialty | X | X | Completion rate of M511 and M504 |
MS Mathematics Education
| # | MS Mathematics Education PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Demonstrate solid understanding of graduate level mathematics relevant to secondary content in algebra, calculus, geometry and statistics | X | X | Completion rate of 4 required content area courses (M518, M524, M525, M527) |
| 2 | Demonstrate solid understanding of teaching practices that give every student access to rigorous mathematics learning | X | X | Completion rate of 2 required pedagogy courses (M520, M528, M529, M577) |
| 3 | Clearly communicate connections between program coursework and local classroom practice | X | X | Completion of written portfolio and public presentation |
MS Statistics
| # | MS Statistics PLO | 18-19 | 19-20 | Data Source |
|---|---|---|---|---|
| 1 | Possess a solid understanding of core graduate level applied statistics, probability, and mathematical statistics | X | X | Completion of comprehensive exam |
| 2 | Be prepared for career as an applied statistician or a doctoral program in statistics | X | X | Completion of statistical consulting seminar (2 credits of STAT 510) |
| 3 | Clearly communicate results from a statistical data analysis or research problem both orally and in writing | X | X | Completion of writing project or thesis |
3 Threshold Values and Rubrics
The PhD programs in Mathematics and Statistics are assessed by measuring the completion rates of students who advance through the program using a sequence of Milestones. The number of students who complete a given milestone is measured. The time required for each to student to advance between successive milestones is also measured. The threshold values and data sources in the Table 1 below incorporate both of these quantities into the assessment.
The MS programs in Mathematics, Mathematics Education and Statistics are assessed by measuring the completion rates of students who advance through their program by achieving satisfactory performance in coursework, by demonstrating core competencies on a written comprehensive exam and by demonstrating the ability to communicate knowledge relevant to the particular field of study.
The nature of the program determines the structure of the assessment for these various programs, and the threshold values and data sources are described in Table 2 below.
Table 1: Ph.D. Programs
| PLO | Threshold Value | Data Source |
|---|---|---|
|
Demonstrate a solid understanding of PhD core content appropriate to each degree, as listed in program outcomes |
75% fo students who begin the degree will pass the appropriate written comprehensive exam within 2 years 75% of students who begin the degree will pass the written comprehensive exam in an additional specialty area within 2 years |
Milestone 1: Written comprehensive exam in core area Written comprehensive exam in additional specialty area |
| Formulate new research problems | Of those students who have achieved Milestone 1, 75% will pass Milestone 2 within 2 years of the term in which Milestone 1 was achieved | Milestone 2: Oral comprehensive exam |
| Clearly communicate original research appropriate to each PhD degree both orally and in writing | Of those students who have achieved Milestone 2, 75% will pass Milestone 3 within 2 years of the term in which Milestone 2 was achieved | Milestone 3: Defense of dissertation |
Table 2: M.S. Programs
| PLO | Threshold Value | Data Source |
|---|---|---|
|
Demonstrate a solid understanding of MS core content appropriate to the degree (MS Math and MS Stat only) |
75% fo students who begin the degree will pass the appropriate written comprehensive exam within 2 years |
Written comprehensive exam |
| Demonstrate a solid understanding of MS core content appropriate to the degree (MS Math and MSMME only) | 75% of students who begin the MS degree will earn a B or better in appropriate coursework | Coursework |
| Clearly communicate connections between program coursework and local classroom practice (MSMME only) | 75% of students who begin the MSMME will present a portfolio within 3 years | Portfolio and presentation |
| Be prepared for career as an applied statistician or a doctoral program in statistics (MS Stat only) | 75% of students who begin the MS Statistics program will complete 2 credits of STAT 510 with a B or better in 3 years | Coursework |
| Clearly communicate results from statistical data analysis research problem both orally and in writing (MS Stat only) | 75% of students who begin the MS statistics program will complete a writing project or thesis within 3 years | Writing project or thesis and presentation |
Each graduate program is assessed according to the rubrics outlined in the tables below. Table 3 describes the rubric used for all three PhD programs within the department. The MS programs within the department are assessed according to a variety of rubric items, and those vary depending on what is appropriate to the discipline. Table 4 identifies the rubric items that are used to assess each MS program, with the distinction of programs noted in the first column of each row of the table.
Table 3: Ph.D. Assessment Rubric
|
PLO
|
Unacceptable 1
|
Acceptable 2
|
Threshold Values
|
|---|---|---|---|
|
Demonstrate a solid understanding of PhD core content appropriate to each degree, as listed in program outcomes |
Displays limited or mathematical, statistical, or pedagogical content knowledge, demonstrated
by not passing a comprehensive exam within the required time frame.
|
Displays sufficient mathematical, statistical, or pedagogical content knowledge and
passes written comprehensive exams
|
As described in row 1 of Table 1
|
|
Formulate new research problems
|
Does not pass the dissertation proposal/oral comprehensive exam within 2 years of
passing the written comprehensive exam
|
Passes the dissertation proposal/oral comprehensive exam within 2 years of passing
the written comprehensive exam
|
As described in row 2 of Table 1
|
|
Clearly communicate original research appropriate to each PhD degree both orally and
in writing
|
Does not pass the dissertation defense within 2 years of passing the dissertation
proposal
|
Passes the dissertation defense within 2 years of passing the dissertation proposal
|
As described in row 3 of Table 1
|
Table 4: M.S. Assessment Rubric
| PLO | Unacceptable 1 | Acceptable 2 | Threshold Values |
|---|---|---|---|
|
MS Math and MS Stat Demonstrate a solid understanding of MS core content appropriate to the degree |
Displays limited mathematical or statistical, or pedagogical content knowledge, demonstrated by not passing a comprehensive exam within the required time frame. | Displays sufficient mathematical or statistical content knowledge by passing written comprehensive exams | As described in row 1 of Table 2 |
| MS Math and MSMME Demonstrate a solid understanding of MS core content appropriate to the degree | Displays limited mathematical or pedagogical content knowledge and earns lower than a B in the required coursework | Displays sufficient mathematical or pedagogical content knowledge by earning a B or better in required coursework | As described in row 2 of Table 2 |
| MSMME Clearly communicate connections between program coursework and local classroom practice |
Does not present a portfolio within 3 years of beginning the MSMME | Presents a portfolio within 3 years of beginning the MSMME | As described in row 3 of Table 2 |
| MS Stat Be prepared for career as an applied statistician or a doctoral program in statistics |
Does not complete Stat 510 with a B or better within 3 years of beginning MS Stat | Completes Stat 510 with B or better within 3 years of beginning MS Stat | As described in row 4 of Table 2 |
| MS Stat Clearly communicate results from a statistical data analysis or research problem both orally and in writing |
Does not present writing project or thesis within 3 years of beginning MS Stat | Presents writing project or thesis within 3 years of beginning MS Stat | As described in row 5 of Table 2 |
4 What Was Done
Enrollment Summaries By Program
PhD Mathematics
Sixteen students were enrolled in the PhD Mathematics program during the assessment period. One left the program before the two-year comprehensive exam timeframe had expired. Nine students were eligible to take some portion of the written comprehensive exams (Milestone 1) during the assessment period. Eight of those students completed the milestone. Eight of those students are continuing toward the next milestone, and one student chose not to continue and left the program with a MS degree. Five students completed the dissertation proposal/oral comprehensive exam (Milestone 2), and six students completed the dissertation defense (Milestone 3) during the assessment period.
PhD Mathematics – Mathematics Education Option
Seven students were enrolled in the PhD Mathematics – Mathematics Education program during the assessment period. One left the program before the two-year comprehensive exam timeframe had expired, three were not yet eligible to take the comprehensive exam, and one entered the program with a MS in Mathematics and was not required to take the mathematics comprehensive exam. None of the students who were enrolled were eligible to take the oral comprehensive exam during the assessment period.
PhD Statistics
Fifteen students were enrolled in the PhD Statistics program during the assessment period, but only 10 had milestone deadlines occurring during the assessment period. For Milestone 1, 6 of 8 satisfied the core area and 5 of 8 satisfied the research specialty area components within 2 calendar years of entering the PhD program, with one never taking the written comprehensive exam due to transferring from the PhD to the MS program. In an unusual situation, one student completed Milestone 2 before Milestone 1 and within 2 calendar years of entering the PhD program, and then passed Milestone 1 in the following semester. Thus, Milestones 1 and 2 were completed in less than 3 years for this student despite not completing Milestone 1 within 2 calendar years. However, given the circumstances, we consider this student as having successfully completed Milestone 1 in an acceptable timeframe. Thus, overall, 6 of 8 students satisfied Milestone 1. Two of 2 students satisfied Milestone 3 in the assessment period by defending their dissertations within 2 years of passing Milestone 2.
MS Mathematics
A total of thirty students were enrolled in the MS Mathematics program for some portion of the assessment period. Nineteen of those students were scheduled to complete the written comprehensive exam and coursework within the assessment period. All but 3 completed both program requirements (84% completion rate). There were another 8 students that completed both requirements early.
MS Statistics
Of the 19 MS students whose 3-year window fell within the assessment period, 19 passed the required written comprehensive exam, 17 completed Stat 510 with B or better within 3 years of beginning the MS Stat program, and 17 presented their writing project or thesis within 3 years of beginning the MS Stat program.
MSMME
Forty-six students were enrolled in the Master of Science in Mathematics – Mathematics Education Emphasis during the assessment period. Two students left the program and 14 were not yet eligible to present their portfolio.
PhD Assessment Results
Doctoral student progress is assessed using a set of consecutive milestones. The Graduate Program Coordinator maintains a data table which tracks each graduate student enrolled in on of the graduate programs. When a doctoral student begins the program, they are added to the data table, and the date by which they should achieve their first milestone is calculated. As they achieve a milestone, the date of their next milestone is calculated and added to the table. When the date of a milestone is in the 2 year assessment window, the student is counted in the denominator. If they achieved the milestone, they are counted in the numerator. Students who leave the program are counted in the denominator only for the assessment period during which they failed to achieve a milestone.
PhD Mathematics
| PhD Mathematics PLO | Data Source | Results |
|---|---|---|
| Demonstrate a solid understanding of core graduate level real and complex analysis | Completion rate of Milestone 1 – written comprehensive exam in Real and Complex Analysis | 2 – Acceptable Of the 4 students who attempted Milestone 1 during the assessment period, all 4 achieved within 2 years of entering the program. (100%) |
| Demonstrate a solid understanding of core mathematical concepts in at least one area of specialty | Completion rate of Milestone 1 - written comprehensive exam in an additional specialty area |
2 – Acceptable Of the 9 students who attempted Milestone 1 during the assessment period, 8 passed the exam, 7 within the 2 year window. One student failed the exam and left the program. (78%) |
| Formulate new research problems | Completion rate of Milestone 2 | 2 – Acceptable Of the 6 students scheduled to complete Milestone 2, 5 of those students achieved the milestone, all 5 within 2 years of completing the written exam milestone. One student left the program prior to attempting the proposal. (83%) |
| Clearly communicate mathematical research both orally and in writing | Completion rate of Milestone 3 | 2 – Acceptable Of the 6 students who completed Milestone 3 during the review period, 5 achieved the milestone within 2 years of achieving Milestone 2. (83%) |
PhD Mathematics Education
| PhD Mathematics Education PLO | Data Source | Results |
|---|---|---|
| Demonstrate a solid understanding of core graduate level mathematics |
Completion rate of Milestone 1 - written comprehensive exam in a mathematical topic area |
2 – Acceptable. Of the two students who attempted Milestone 1, 2 passed within 2 years. (100%) |
| Formulate questions and design studies to address contemporary issues in mathematics education |
Completion rate of dissertation proposal |
Nothing to report |
| Clearly communicate mathematics education research both orally and in writing |
Completion rate of dissertation defense |
Nothing to report |
PhD Statistics
| PhD Statistics PLO | Data Source | Results |
|---|---|---|
| Demonstrate a solid understanding of advanced mathematical statistics, Bayesian statistics, and data analysis methods | Completion rate of Milestone 1 - required written comprehensive exam components in advanced mathematical statistics. Bayesian statistics, and data analysis methods | 2 – Acceptable (6 of 8 passing within 2.5 years, but with one student completing Milestone 2 before Milestone 1 in 2.5 years.). 75% |
| Demonstrate a solid understanding of core statistical content in at least one research area of specialty | Completion rate of Milestone 1 - written comprehensive exam component in a research area of specialty |
1 – Unacceptable (5 of 8 passing within 2.5 years, but with one student completing Milestone 2 before Milestone 1 in 2.5 years). 62.5% |
| Formulate new research problems | Completion rate of dissertation proposal | Not applicable. There were no PhD students in the assessment period for Milestone 2. |
| Clearly communicate original statistical research both orally and in writing | Completion rate of dissertation defense | 2 – Acceptable (2 of 2 defended their dissertations within 2 years of the dissertation proposal.) 100% |
MS Assessment Results
Students enrolled in one of the MS programs within the department are assessed based on their ability to satisfy the specific program requirements within the expected time window, depending on the specific nature of the program. For the MSMME and Statistics programs this is a 3-year window, and for the MS Mathematics program, the window is two years. Assessment of the MS programs is essentially a reporting of the graduation rate of these programs within the appropriate timeframe.
MS Mathematics
| MS Mathematics PLO | Data Source | Results |
|---|---|---|
| Demonstrate solid understanding of graduate level real analysis and advanced linear algebra | Completion rate of required written comprehensive exam | 2 – Acceptable (16 of 19) (84%) |
| Demonstrate solid understanding of core mathematical concepts in at least one area of specialty | Completion rate of M511 and M504 | 2 – Acceptable (16 of 19) (84%) |
MS Mathematics Education
| MS Mathematics Education PLO | Data Source | Results |
|---|---|---|
| Demonstrate solid understanding of graduate level mathematics relevant to secondary content in algebra, calculus, geometry and statistics | Completion rate of 4 required content area courses (M518, M524, M525, M527) | 2 – Acceptable 23 of 26 (88%) |
| Demonstrate solid understanding of teaching practices that give every student access to rigorous mathematics learning | Completion rate of 2 required pedagogy courses (M520, M528, M529, M577) | 2 – Acceptable 24 of 26 (92%) |
| Clearly communicate connections between program coursework and local classroom practice | Completion of written portfolio and public presentation | 2 – Acceptable 23 of 26 (88%) |
MS Statistics
| MS Statistics PLO | Data Source | Results |
|---|---|---|
| Possess a solid understanding of core graduate level applied statistics, probability, and mathematical statistics | Completion of comprehensive exam | 2 – Acceptable (19 of 19) (100%) |
| Be prepared for career as an applied statistician or a doctoral program in statistics | Completion of statistical consulting seminar (2 credits of STAT 510) | 2 – Acceptable (17 of 19) (89%) |
| Clearly communicate results from a statistical data analysis or research problem both orally and in writing | Completion of writing project or thesis | 2 – Acceptable (17 of 19) (89%) |
5 What Was Learned: Results
The graduate committee analyzed the assessment data and presented these data to the entire faculty in the department. The report was circulated prior to the faculty meeting on October 7 and was discussed during and after that meeting.
Areas of strength
- Success in all three M.S. programs in supporting students to timely degree completion.
- Success in Ph.D. programs in supporting student progress through milestones.
Areas that need improvement
- In the MS Mathematics program, we have no mechanism for assessing student outcomes for those enrolled in the MS thesis option. The committee recommends that MS Mathematics data source for the “solid understanding of core content” should be amended to include written comprehensive exam OR MS thesis defense.
- The MS Data Science is not included in this report. Because the first students enrolled in the program in 2019, there are not enough students to have completed for the assessment to be meaningful. The Data Science Steering Committee should outline an assessment strategy so that the program can be assessed in the 2022 assessment report.
- The Graduate Certificate in Statistics is not included in this report and should be. The Statistics Group should outline an assessment strategy so the program can be assessed in the 2022 report.
- . In future reports, the MSMME assessment could be simplified to list completion rate of the program rather than counting the subcategories, because the subcategories are each required.
- The assessment document does not include any information about recruitment to the programs. For example, the document could include a summary of the number of MS students who move into the PhD programs within the department. The committee should discuss an assessment strategy that measures recruitment so that the programs can be assessed on this item in the 2022 report.
6 How We Responded
Curriculum assessment changes
Based on the faculty responses, there will be no plans for measurable improvements or realignment of learning outcomes.
Assessment of changes
The graduate programs will be next assessed in 2022, with the suggestions 1-5 implemented in the next report.
7 Closing the loop
Demonstration of improvements from changes
In our prior assessments, we identified simplifying the comprehensive exam structure as a goal. These simplifications have continued to take place. The assessment process itself and program learning outcomes of this report were developed following the 2018 assessment report, so the format, outcomes, and measures listed in this document represent a substantial implementation of changes as a result of the 2018 assessment.