​Evaluating the impact of qualification

Evaluating the impact of a qualification based approach to student engagement and success in engineering study

The main project question is focused on whether, and if so how, a whole qualification approach of designing and developing the various components of the recent BEng (Hons) (Q65) and MEng (M04) has had a positive impact on student success. Student engagement and depth of learning are recognised as important criteria for student progression through a qualification. We seek a greater understanding of influencing factors that should help inform teaching quality assessments, such as annual quality reviews for internal and QAA purposes or towards future benchmarks such as TEF criteria.

There will be a narrative section (which will include a literature review) describing the motivation and background of the various factors (internal and external) that influenced decision making around the redesign of the Engineering qualifications, as well as the various innovations implemented under the qualification structure. This has focussed mostly on the Stage 1 curriculum to-date with Stage 2 and Stage 3 qualification renewal in progress. The proposal consists of two parts: Background of factors that informed design of new Engineering curriculum. Impact on student experience and success so far based on: analysis of quantitative and qualitative student experience and retention data. Associate Lecturer (AL) perceptions and observations .

We will partially address the main research question by analysing appropriate quantitative and qualitative data that are readily available in the OU data systems, around student retention, progression and satisfaction. Although raw data is available it requires extensive analysis to determine cause and effect (as opposed to observed correlation alone). Deeper investigation is necessary to isolate causal factors (where possible).

We propose to gather qualitative data on the perceptions of ALs through a survey and focus groups/interviews (to take place possibly at the next Engineering Residential Schools in July 2018). Use of AL feedback is particularly valuable given that a proportion of the engineering ALs have supported students under both the old and new qualification structures. This is in contrast to students who will only have direct study experience of either the old qualification framework or the newly structured and redeveloped qualification. Use of an AL survey will permit deeper investigation of the impact of individual changes made to the curriculum and qualifications. Changes have included. Integration of personal development planning (PDP) and professional skills into the core engineering curriculum for greater relevance and context. This initiative has included the use of a bespoke in-house learning log tool to facilitate pacing of student workload towards assessment points, student self-assessment of learning outcomes achievement, and to develop skills towards employability. A rewrite of core engineering modules at Stage 1 using embedded mathematics teaching and a new presentation pattern to avoid overlapping modules and ensure consistent student workload. Greater use of activities to encourage peer to peer interactions (e.g. Open Engineering Studio and OpenEngineering Laboratory experiments) to combat isolation in a part-time distance learning environment. The overall initiative has been developed to equip new engineering undergraduates with the study skills, knowledge and understanding, and confidence towards progression, while maximising student retention. A qualification based approach to assessment at each Stage of study. This initiative has required innovations to the form and progression of individual student assessments together with implementation of staff Assessment and Tuition Groups. New student assessments have been designed with consideration of progression within each module and from module to module in mind as part of feed forwards support for students. Peer review, staff development, and inherent scholarship all play an important role in the qualification level groups of staff. The AL community bring experience of old and new frameworks, often together with experience across other STEM subjects including mathematics and physics. Feedback from ALs will also provide valuable insight to student behaviour and attainment as some ALs tutor on both the October (J) and April (D) presentations of the first two engineering modules under the new qualification structure.

This has the potential to provide a unique insight to differences in student attainment across the seasonal academic year that can be further explored through the AL experience. It is proposed to conduct the majority of the AL interviews at the next Engineering Residential Schools in July 2018 where a cohort of ALs will be present.