piwik-script

Intern
    Compall

    Evaluation Results

    Evaluation on the impact of the joint module – outcomes of the Winter School

    For assessing the development of the joint module and for measuring the impact of the intensive programme, the Strategic Partnership has implemented a formative evaluation, both qualitative and quantitative that accompanied the whole development process and critically reflected the implementation of all intellectual outputs – within and beyond the learning activities. The evaluation is divided in an external evaluation and an internal evaluation. The external evaluation was conducted by Susanne Lattke of the DIE (German Institute of Adult Education). The evaluation design and interpretation was developed in collaboration with the COMPALL Project Coordinator, Prof. Dr. Regina Egetenmeyer (Julius-Maximilians-University of Würzburg). All citations on this webpage are part of the internal evaluation reports. The evaluation design involved both short and mid-term evaluations of the Winter Schools defining indicators at the participant and the organizational level.

    For the short-term evaluation, participants were requested to fill in a paper questionnaire at the final session of each Winter School. The questionnaires of each year were rather identical with adopted questions on specific components. In addition, a number of interviews were conducted with selected participants during the last days of the Winter School. Information from project documents (e.g. COMPALL application, International Winter School brochure) and information orally provided by the organisers (e.g. regarding total participant numbers) have complemented the data, which was obtained through the surveys and interviews. 

    For the mid-term evaluation, participants of former Winter Schools were requested to fill in an online-survey after more than one year of their participation in the Winter School. Within the COMPALL project two mid-term surveys will be carried out. A first survey was conducted in spring 2017. It addressed all participants from the Winter School 2016 and, in addition, all participants of the previous pilot editions of the years 2014 and 2015, which took place before the COMPALL project. A second midterm survey is taking place in spring 2018 targeting all cohorts of the programme. 

    Both evaluation methods form the comprehensive picture on the impact of the implemented intensive programme focusing on the following targets:

    “The main focus of the short-term evaluations, in a formative perspective, is on participant satisfaction and on the quality of the Winter School and its related offers. Also, the short-term evaluations focus on the academic and personal outcomes and benefits which the participants have gained from the Winter School, mainly in terms of learning outcomes and increased motivation for international activity.” (Midterm-Term Evaluation Report 2017, p.5) 

    “The mid-term surveys, by contrast, focus primarily the development of former participants’ professional careers and/or further studies, and the impact which the Winter School has had on these.” (cf. ibid.)

    The structure of the external evaluation followed a six-step process: 

    ·         Step 1: Ascertainment of the evaluation concept for the three year, development of indicators into questionnaire items

    ·         Step 2: Student questionnaire used for the first cohort after the first intensive phase

    ·         Step 3: Student questionnaire used for the second cohort after the second intensive phase

    ·        Step 4: Mid-term student questionnaire used for asking the first cohort for longer-term effects

    ·         Step 5: Student questionnaire used for the third cohort after the third intensive phase

    ·         Step 6: Mid-term student questionnaire used for asking all cohorts for longer-term effects.

     

    In addition to the external evaluation, the Strategic Partnership conducted an online survey on the developed online tutorials piloted in the intensive programmes 2017 and 2018. A short summary of the results is provided in the corresponding Abstracts.

     

     

     

    Impact Evaluation 2018

    Mid-Term Evaluation 2018

    Extraction of the external evaluation results, 

    which was conducted by German Institute for Adult Education, Susanne Lattke 

    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the Mid-Term Survey

    The midterm survey 2018 was implemented as an online survey. All students who had participated in the Winter Schools in the years 2014-2017 (n=279) received an invitation to fill in the questionnaire which they could easily access through a link which was provided together with the invitation. The years 2014-2015, which do not belong to the ERASMUS+ COMPALL-project where included to ensure a broader database and to raise the value of the data for the analysis of the impact of the COMPALL-project. For data privacy reasons, the invitation was sent by the organisers of the Winter School and not by the evaluator (the author of this report). The questionnaire opened between 24 April and 29 May 2018. Three reminders were sent within this period to reach as many former participants as possible. Ultimately, the questionnaire was fully completed by 68 respondents, which corresponds to a response rate of 24.4%.

    Labour Market

    The midterm survey 2018 shows that a high proportion (three quarters) of respondents currently working are actually facing international aspects in their work, and a similar proportion of those
    currently not working are highly motivated to engage with international issues in their future activity. Respondents not only find an international dimension in their future work highly attractive but also consider this perspective as rather realistic for themselves. Even cross-border mobility for finding employment in another country is considered a likely option by a considerable share of the respondents, in the 2018 survey even much more (mean 4.02) than in 2017 (mean 3.48). Regarding the Winter School’s impact, respondents confirm in particular that the Winter School has strengthened their motivation for international work, but also that it has contributed to developing competencies which are considered as useful in view of labour market and career perspectives. Considerable shares of respondents, both currently working and currently still studying, are also convinced that the Winter School participation has enhanced their competitive advantage on the labour market and, in particular, within the scientific community, in case they aim for a career in that context.

     

    Networking Impact

    About two thirds of the participants have started close collaborations with other participants after the Winter School and almost all respondents have the feeling of being part in an international network which may prove useful to them in case of need. These findings were consistent in both midterm surveys, 2017 and 2018. The finding from the midterm survey 2017, that about 25% of the respondents were still rather active users of the LinkedIn-networks was also repeated in 2018. The number of passive users was with about 60% of the respondents clearly higher. The online linkedIn networks thus seem not to be the most important tool for keeping in touch, but still they offer an additional resource which is used at least passively by a considerable share of the participants.

    Impact on PhD Study

    Another aim of the Winter Schools is to increase the number of highly qualified graduates in the field of adult education in general, and in view of an international study profile in this field in particular. As in 2017 and to an even higher extent, the midterm survey results 2018 show that the Winter School has strengthened in many respondents the motivation to take up a PhD study. Those respondents who have already started their PhD research also believe that the Winter School has contributed to the methodological and analytical quality of that Research.

    Impact on International Dimension of Study

    The majority of respondents had actually included international perspectives in their Master or PhD thesis or were intending to do so. Respondents also stated that the Winter School had had an impact on their thesis. Especially respondents who, at the time of answering the survey, were currently studying for a PhD or who had just finished their Master studies and were planning to take up a study rated this impact as rather high. The same pattern could already be observed in the midterm 2017 survey, with even higher impact rates at that time. It seems likely that these particular groups have participated in the Winter School during a “crucial” phase in their studies; a phase, in which many issues regarding a possible follow up PhD research may not have been firmly fixed but had still to be decided and shaped, so that the Winter School could come in as an influential experience.

    Personal and Professional Outcomes

    Looking back from a distance, most participants had very positive views regarding their individual outcomes from the Winter School. Mmore than three quarters of the respondents agreed that the Winter School had considerably helped them in both their professional and personal development, and respondents were nearly unanimous in saying that the Winter School had been an enjoyable experience.

    Participant Quotes from open Questions:

     

    Winter School Evaluation 2018

    Evaluation of Winter School 2018

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the 2018 Evaluation

    The Winter School 2018 evaluation is based on the paper questionnaire which was distributed by the working group leaders to all participants on the final day of the Winter School. The International Winter School 2018 was attended by a total of 61 participants. The number of filled questionnaires amounted to 59, so that the response rate is 96.7%. About two thirds of the participants (42) were Master students and one third (19) were PhD students. The sample of respondents reflects the same 2:1-relation between MA and PhD students (31 Master vs. 14 PhD). 14 respondents chose not to indicate their study level in the questionnaire. 

    Participants’ Satisfaction

    The Winter School in 2016 had set out already on a very high level, as far as both participant satisfaction and perceived benefits/impact of the Winter School are concerned. The Winter Schools in 2017 and 2018 not only have maintained these level but managed to achieve even further improvements in the participant ratings over the three years. The overall assessment shows that the participants’ general satisfaction with the International Winter School was very high. The mean value was 4.42 (n=48) which means an improvement compared to the already very high values of the two previous years (4.20 and 4.24 respectively).

    Again, participants’ satisfaction turned out to be very high for all aspects with each single mean value being above 4.0. This, too, means a further improvement compared to the previous year(s), although values had been rather high already in 2016 and 2017. In particular, satisfaction with the preparatory phase has increased compared to 2017. This show that the new preparatory offers (video tutorials, online information tools) which have been developed by the organisers since the last year, are well accepted by the participants. Complaints from the first year regarding the academic content of the Winter School or the composition of the participants group (uneven skills levels, e.g. regarding English proficiency, or unbalanced geographical distribution) have greatly diminished over the years. This suggests that the organisers have been successful in establishing an effective recruiting policy which ensures that potential participants are well informed about what they have to expect from the Winter School and what is required from them for a successful participation. At the same time, the geographical distribution of participants’ countries of origin became more balanced over the years: the numerical dominance of a few countries was reduced and it became easier to compose the working groups within the Winter Schools in a way to include a broad range of different country perspectives.

     Participants’ Competence Development

     In the later years 2017 and 2018, the focus on academic and professional content has become clearer, whereas in 2016 participant feedback had still focused very prominently on intercultural and social outcomes, including networking opportunities. These intercultural and social aspects remained important also in 2017 and 2018, however they were no longer the dominating feature in the participant feedback. Rather, the Winter School increasingly appears as a learning offer which combines in an integral way specific subject-related competence development with the development of transversal competences and personal soft skills, besides offering a highly enjoyable experience and an excellent opportunity for building up personal and professional networks.

     

    Impact Career on Career Chances

    The career relevance of the Winter School (outcomes) have been unchangeably rated positively by the participants in all years, which suggests that the Winter School concept reflects well the competence needs in both an academic context and the labour market.

    Summary of external evaluation

    One can conclude that the overall concept of the Winter School seems highly sustainable and does not need to be changed in a substantial way. There will be a need to secure the necessary financial means for the running of the Winter School after the Erasmus project funding has ended. Since the Winter School can rely on a solid consortium of partners and since during the project’s lifetime a substantial basis of materials was built up which can be used also in the future, there are good prospects for the Winter School to become a sustainable offer in the long-term. An appropriate information policy to reach and support (prospective) participants seems essential for the future success of the Winter School. A substantial part of the critical remarks made by participants in all years can be attributed to wrong expectations on the side of the participants or to a (felt) lack of orientation or preparation. The Winter School organisers are aware of this issue and have already developed with some success their information and support offers over the years. Also in the future, this issue of information and preparation of the participants should receive particular attention, as its impact on the Winter School success and the satisfaction on the participants seems considerable. The preparation phase underwent substantial developments over the three years, with many new materials (videos, tutorials, guidelines) being developed to support participants in their own work prior to the intensive phase in Würzburg. These materials are clearly appreciated and found helpful by the participants. They, too, should be kept as a regular feature and be updated as needed. By way of a concluding overall appreciation, it can be stated that the International Winter School obviously provides a very attractive and highly effective opportunity to students from (potentially) all over the world to learn and develop as an individual and as a (future) professional in the field of adult education in an increasingly globalized context. Moreover, the Winter School has shown its ability to develop and improve further over the years so as to satisfy participants’ needs and expectations to an ever higher degree.

     

    Online Tutorial Evaluation

    Evaluation 2017/2018

    How do you asses the online-tutorials all together?

    How helpful do you estimate the online-tutorials for your preparation to the Winter School?

     

    Impact Evaluation 2017

    Mid-Term Evaluation 2017 

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the Mid-Term Survey

    The midterm survey 2017 was implemented as an online survey. All students who had participated in the Winter Schools in the years 2014-2016 (n=189) received an invitation to fill in the questionnaire which they could easily access through a link which was provided together with the invitation. The years 2014-2015, which do not belong to the ERASMUS+ COMPALL-project where included to ensure a broader database and to raise the value of the data for the analysis of the impact of the COMPALL-project. For data privacy reasons the invitation was sent by the organisers of the Winter School and not by the evaluator (the author of this report). The questionnaire was open between 25 April and 21 May. Three reminders were sent within this period to reach as many former participants as possible. Ultimately, the questionnaire was fully completed by 65 respondents which corresponds to a response rate of 34,4%.

    Labour Market

    Regarding work related issues, the Winter School had a particularly high impact on increasing participants’ motivation to work on international issues. This was especially true for respondents who were not (yet) working (mean 4.13 on a five-point-scale). About half of the respondents not (yet) working considered that the Winter School had notably (rating ‘4’ or ‘5’) contributed to their employment perspectives, by giving them a competitive advantage on the labour market and by providing them with information or contacts which would be useful in finding employment (ibid.). The proportion of respondents currently working who were of the same opinion with regard to their current employment was only slightly lower. It ranged between one third and almost one half of the respondent for the different items.

     

    Networking Impact

    Two thirds of the respondents stated that they had started a close exchange or cooperation with some fellow participants, and 75% of the respondents felt they had made some really good friends at the Winter School. The linkedIn networks seem to play only a subsidiary role in that. Only about one quarter of the participants stated that they were very using these networks very actively. About two thirds were at least following the news in the networks more passively. This reduced active use of the network seems however not to compromise the Winter School’s impact as such. As many as 86% of the respondents stated that they were feeling part of an international network of people whom they can contact at any time if need be, no matter if they were permanently in touch with them or not.

    Impact on PhD Study

    About half of the respondents who were currently preparing or studying for a PhD or who had already finished their PhD agreed to a high extent with the statements that the Winter School participation had strengthened their motivation to take up a PhD, that it had contributed to its analytical quality and that it had helped them to better cope with methodological research issues. Furthermore, at least two respondent have obviously found their (co)supervisor for their PhD research through the Winter School. Of those respondents who were not (yet) actively dealing with PhD studies (n=32), 24 stated that it was somehow likely that they would take up a PhD research in the future.

    Impact on International Dimension of Study

    Again, about half of these respondents confirmed that the Winter School had considerably strengthened their motivation for taking up a PhD and that the Winter School had also provided them with ideas for a possible topic. As many as nine respondents agreed that the Winter School had a role in helping them to find a prospective (co)supervisor for their thesis.

    Personal and Professional Outcomes 

    Looking back from a distance, most participants had very positive views regarding their individual outcomes from the Winter School. When asked to rate different outcomes on a five-point scale, about 80% of the respondents agreed by rating “4” or “5” that the Winter School had considerably helped them in both their personal and professional development, and respondents were nearly unanimous in saying that the Winter School had been an enjoyable experience. Regarding improved career chances, about half of the respondents rated the Winter School’s impact “4” or “5”: 45% of the respondents saw a good impact on their labour market chances in general, and almost 60% said the same with respect to a career in the scientific community, in particular.

    Winter School Evaluation 2017

    Evaluation of Winter School 2017

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL- Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the 2017 Evaluation

    The Winter School Evaluation 2017 is based on the paper questionnaire which was distributed by the working group leaders to all participants on the final day of Winter School.
    The number of filled questionnaires also amounted to 84, so that the response rate would be 100%. Master students were with 50 respondents the biggest group compared to 32 PhD students. Two respondents chose not to indicate their study level in the questionnaire. (Master students: 59, 5 %, PhD students: 38, 1, other/no answer: 2,4%).

    Participants‘ Satisfaction

    The overall assessment shows that the participants’ general satisfaction with the International Winter School was very high. The mean value was 4.24 which is almost exactly the same as in 2016 (4.20).

    Again, participants‘ satisfaction turned out to be very high for all aspects with means ranging between 3.8 and 4.15 for almost all aspects. There were only a few comments given in the questionnaire regarding the preparatory phase. Nevertheless, one should note, that especially the preparatory phase is still under development and new features, such as online tutorials, were newly introduced and piloted fort the first time in 2017.

    Participants‘ Competence Developement

    The ratings are very similar to 2016. In a general perspective, participants judged their academic and personal outcomes from the International Winter School very positively. Regarding specific competences, respondents clearly feel that the International Winter School has considerably contributed to their competence development in many different regards, including both “soft” skills” like intercultural competence and “hard” academic skills like methodological research skills.

    Impact Career on Career Chances 

    The career relevance of the Winter School have been rated very high. It shows that the competences which the International Winter School addresses are considered by the participants as highly relevant for their career chances.  

    Summary of external evaluation

    As in 2016, the findings from the Winter School evaluation provide a very positive overall picture. Regarding the perspectives for the future development of the International Winter School, one can conclude that the overall concept of the Winter School seems highly sustainable and should not be changed in a substantial way. By way of a concluding overall appreciation, it can be stated that the International Winter School obviously provides a very attractive and highly effective opportunity to students from (potentially) all over the world to learn and develop as an individual and as a (future) professional in the field of adult education in an increasingly globalized context. Most participants had already dealt with international topics at least to some extent, but for about one half of the participants this was their first international study mobility. Participants’ motivations for attending the International Winter School were multidimensional but the desire to meet people from other countries and to learn about other countries were particularly stressed. Regarding participant satisfaction and impact, the evaluation shows that the International Winter School has succeeded in maintaining the very high level of the previous year in all respects. The Winter School provides a multifaceted and at the same time coherent learning opportunity which makes a valuable contribution to the participants’ both personal and professional development as adult education researchers or experts.          

    Winter School Evaluation 2016

    Evaluation of Winter School 2016

    Extraction of the external results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University of Würzburg, Prof. Dr. Regina Egetenmeyer


    Participation in the 2016 Winter School

    The Winter School 2016 was attended by 91 participants from 19 countries. Of these, 82 participants filled in and returned the questionnaire, corresponding to a response rate of approx. 90%. Master students (51 respondents) outnumbered PhD students (23). The proportion of Non-European participants (at least 25) was considerable, but the focus was clearly on participants from European countries (at least 50).

    Participants’ Satisfaction

    The overall satisfaction with the Winter School was very high. The overall means was 4.20. Participants also appreciated the opportunity to learn about international concepts and about the situation in other countries and to get new ideas and perspectives. Another aspect which was highlighted was the active involvement of students through group discussions, group/team work and self-responsible work of students.

     

    Participants’ Competence Development

    The ratings show that respondents feel the Winter School has considerably contributed to their competence development. The lowest – but still high – mean values were obtained by English language competencies” (mean 3.8) and methodological skills for comparative research (3.9). All other items – analytical competencies, being able to see adult education in one’s own country in a new light, professional networking, interacting with people from other cultural backgrounds, and understanding adult education/lifelong learning in other countries - scored above 4. Overall, participants judged their academic and personal outcomes from the Winter School very positively (means of 3.7 for academic and 4.3 for personal outcomes). The overall picture thus shows that respondents feels they have obtained a broad range of learning outcomes with the foci being very much in line with the intended aims of the Winter School.

     Impact Career on Career Chances

    All competencies were attributed a high importance by the respondents with the mean values ranging from 3.93 to 4.74. If one compares, how respondents rated the career relevance of the different competencies addressed by the Winter School on the one hand, and how they assessed the Winter Schools’ contribution to developing exactly these competencies on the other hand (see section 4.2.2) one can conclude that the Winter School made a significant contribution to enhancing the participants’ future career chances.

    Summary of external Evaluation

    The evaluation showed very high satisfaction levels of the participants concerning the Winter School as a whole as well as its single parts. As regards the Winter Schools’ impact, participants felt that the Winter School had considerably contributed to enhancing their competences in many regards. Most notably, increased competence for intercultural interaction was highlighted as outcome, but also the development of other competences, both content-related and transversal ones, was felt to have been promoted by the Winter School. Participants also considered that the competences which were promoted by the Winter School were highly relevant for their further career chances, thus contributing to their employability. After attending the Winter School, participants also felt more motivated to engage with international topics in their studies or future professional activities as well as to undertake further study mobilities. 
    In conclusion, the evaluation showed that the Winter School reached a high number of its target audience, involving different countries of origin, study levels, motivations and degrees of previous international experience. By all these groups the Winter School was perceived as a high quality event which had much to offer for their own academic/professional as well as personal development. 

    Hinweis zum Datenschutz

    Mit 'OK' verlassen Sie die Seiten der Universität Würzburg und werden zu Facebook weitergeleitet. Informationen zu den dort erfassten Daten und deren Verarbeitung finden Sie in deren Datenschutzerklärung.

    Hinweis zum Datenschutz

    Mit 'OK' verlassen Sie die Seiten der Universität Würzburg und werden zu Twitter weitergeleitet. Informationen zu den dort erfassten Daten und deren Verarbeitung finden Sie in deren Datenschutzerklärung.

    Social Media
    Kontakt

    Professorship for Adult and Continuing Education
    Oswald-Külpe-Weg 82
    97074 Würzburg

    Tel.: +49 931 31-83337
    E-Mail

    Suche Ansprechpartner

    Hubland Nord, Geb. 82
    Hubland Nord, Geb. 82