piwik-script

Intern
    Intall

    Evaluation Results

    INTALL (2018-2021) evaluation

    The results of the evaluation of the ERASMUS+ strategic partnership INTALL (2018-2021) will be published here.

    External Evaluation of INTALL

    The Winter Schools 2019, 2020 and 2021 within the newly launched project INTALL will be accompanied and monitored by an external evaluation conducted by Katrin Riß (Cologne). This evaluation combines assumptive and formative aspects. It will give evidence of the programme’s outcomes and it seeks to support the programme’s effectiveness by providing relevant information on modifications required throughout the three years’ process. In order to ensure a long-term perspective on the Winter Schools in Wuerzburg since their beginning in 2016, the evaluation builds on the COMPALL evaluation conducted by Susanne Lattke of the German Institute for Adult Education in Bonn. Again, it involves a short-term perspective and a mid-term perspective on the programme covering the following objectives.
    The short-term evaluation: This part of the evaluation mainly focusses on the students’ assessment of the programme: First, it will cast a light on the participants’ motivation for attending the Winter School, ask about the support provided by the students’ home universities to enable and back their participation and give some information about their content-related knowledge and competencies before the Winter School. Second, it will outline how participants assess the quality of the different programme modules as well as how relevant they consider the different components for their own development. And third, the evaluation addresses the effects of the Winter School. This concerns effects on the students’ competence development and their interest in adult education and lifelong learning (especially regarding the international focus), study prospects, professional career prospects, and study mobility. 
    The mid-term evaluation: This part of the evaluation lays emphasis mainly on the effects of the Winter School programme. It asks how the international focus in the participants’ studies and professional practice develops over the course of the programme. Also, it will address the question to what extent the participation has influenced the students’ further study aspirations (taking up a PhD or post-doc research) as well as the motivation to undertake further study mobilities. And finally, the mid-term evaluation deals with the competences gained or enhanced by attending the Winter School.
    In order to produce relevant information to these questions, six surveys will be conducted addressing the participants or former participants of the Winter Schools. At the end of each Winter School (February 2019, 2020, and 2021) three mainly quantitative surveys will compile information regarding the participants’ immediate assessment of the joint module (short-term). Three surveys about 15 months after each Winter School will keep track later on (mid-term).


    Evaluation of the Winter School 2019

    Summary of the external evaluation results
    Author: Katrin Riß
    INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Approach

    The short-term evaluation in 2019 took a mainly quantitative approach. The questionnaire used was based on the instrument applied in the years before and adjusted to the current Winter School by the organisers. The findings result from a total of 67 questionnaires (response rate 100%), filled in online at the end of the Winter School. 

    Sample/group characteristics 

    The group of participants was formed by students from 20 different countries, about two third of them being master students (43 persons, 64%), 1 out of 4 being a doctoral student (15 persons, 22%), and about 1 out of 7 working as practitioners in the field of adult education and lifelong learning (9 persons, 14%).

    Participants’ Satisfaction

    The evaluation clearly shows that, all things considered, the participants were highly satisfied with the Winter School. On a 5-point scale (1 = poor, 5 = excellent) their overall assessment reached a mean value of 4,39. These very high ratings confirm the results of the COMPALL evaluations in the years 2016 to 2018 assessed with similar mean values (2016: 4,20, 2017:4,24). Taking a closer look, we also see that no outliers at all can be found in that question and the high ratings apply to all student groups (master students, doctoral students, practitioners) with the ratings of the practitioners being the lowest.

    With respect to the different programme components, especially Week 1 and the Comparative Groups during Week 2 achieved very high ratings in the categories organisational, academic, didactical quality (all means values ranging between 4,13 and 4,40 on a 5-point scale with 1 = not satisfied, 5 = very satisfied). Ratings a little lower were given to the Preparatory Phase (mean values for the three categories 4,04 – 3,93 – 3,99) varying from country to country. The lowest ratings finally were registered with regard to the Employability Day at the beginning of Week 2 (mean values 3,72 – 3,67 – 3,79).

    According to this data but also to the participants’ qualitative feedback the Winter School appears to be a learning setting well accepted and appreciated by the students. Various comments highlighted the didactical quality of the sessions, a learner centric and participative approach including a broad variety of (innovative) teaching methods, as well as respectful, unformal and supportive atmosphere at the Winter School.

    Effects of the Winter School

    The International Winter School mainly intends to introduce its participants to international issues and comparative aspects within adult education and lifelong learning and to provide them with the respective competencies and skills required in international or internationally influenced working contexts. Based on the participants’ answers it can be concluded that the Winter School widely accomplishes these goals. The majority of the students strongly agrees that their interest in trans-/international and comparative aspects of adult education and lifelong learning has increased and that they are motivated to further focus on these aspects in their studies and even in their future professional activities (see table). Concerning the question if their motivation to undertake further study mobility increased, more than 90% of the master and doctoral students (N=58) scored a 4 or 5 on the 5-point scale. Also, the motivation to take up further studies was increased: 56% of the master students and practitioners rated a 4 or 5 on the scale expressing their motivation to take up a PhD. 62% of the doctoral students (9 persons) stated with a score of 4 or 5 that their interest in taking up postdoc research after their graduation increased due to the participation in the Winter School.

    With regard to the effects of the Winter School, the questionnaire also asked about the development of competencies induced by participating in the programme. As the chart below illustrates all ratings are considerably high, especially concerning aspects that refer to “international” competencies and knowledge. For example, almost all participants (93%) claimed that their understanding of adult education and lifelong learning in other countries has increased very much or much. 78% of the group have ticked a 4 or 5 when being asked if their intercultural competencies have improved. The lowest values – even if still high – on the other hand we find for the question concerning the increase of analytical competencies. At this point, it has to be kept in mind that 2/3 of the participants had stated that these competencies had been high already before the Winter School.


    When taking a closer look at the outcomes for the different groups of participants, we see that master and doctoral student obviously profited the most from participating in the Winter School. This regards the academic outcomes (mean value 4,38 on the 5-point scale) as well as the personal ones (mean value 4,43). For the group of practitioners, who were newly addressed by the INTALL Winter School the outcome, however, resulted a little lower. This group rated their academic outcome with a mean value of 3,56 (on the same 5-point scale), the personal outcome with 4,0, and finally the outcome for the employment with a mean value of 3,88.

    Conclusions

    The evaluation’s findings confirm and continue the consistent positive or more precisely rising results of the COMPALL evaluation and attest the Winter School to be a well-designed and effective learning opportunity for its participants. The results prove a respectful learning environment that obviously encourages its participants to develop confident research activities and improves employability skills. Apart from social and especially intercultural outcomes the evaluations points to considerable academic learning effects for the participants. This regards knowledge about international contexts of adult education and lifelong learning as well as the improvement of academic skills – most striking of course comparative skills.

    Despite this already high standard, the evaluation’s results point to some modifications on the concept and programme. This concerns the following aspects: (1) Some findings suggest that the concept regarding the new target group of practitioners can be further adapted in order to increase the effects and to fully exploit the chances arising from bringing students and practitioners together. Obviously, the role of the practitioners was not very clear to the group at every point of the Winter School and chances to contribute their specific perspective on the subjects discussed should be inhanced. Also, it seems that the preparation of the practitioners should receive more attention. (2) Other findings indicate that the concept of the Employability Day needs to be revised. Obviously, this module did not fully match the participants’ needs and some of them were hoping for more profound information. At this point, some more thought might be given to the variety of participants and strategies should be found to take the differing prior knowledge more into account (splitting the group, using methods that allow more options and flexibility, using the expertise of the practitioners etc.). (3) This piece of work obviously plays an important part during the preparatory phase of the Winter School and contributes remarkably to the learning effects of this programme. In this process it requires considerable effort and time to prepare and thus seems to be highly important to the participants. At some points the findings of the evaluation suggest that during the intensive phase some more time and focus should be spent on these essays in order to provide more content-related feedback as well as appreciation for the effort taken. Considering the little data base this fact might only regard single comparative groups. In any case, we recommend ensuring the same intensity in dealing with the transnational essays in all comparative groups. (4) As in the years of the COMPALL Winter School, the programme schedule remains a quite striking aspects in the evaluation. Various comments suggest that especially during Week 1 efforts should be taken to keep participants attentiveness high. This might include changes in the schedule like shifting intense lectures to an earlier day time, but also using (even) more interactive teaching methods and providing more opportunities for interaction among the group.

     

     

     

    Mid-term Evaluation 2019 -Summary

    Author: Katrin Riß
    INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Approach

    The mid-term survey 2019 was conducted as an online survey. The questionnaire (based on the instrument implemented during the mid-term survey of the predecessor programme COMPALL) was sent to the former participants of the Winter Schools 2014, 2015, 2016, 2017 and 2018 by the organisers themselves and thus was addressed to a total of 331 persons. 71 respondents fully completed the questionnaire and the survey achieved a response rate of 21,5%.

    Sample/group characteristics 

    The Winter School addresses students of the INTALL partner universities and organisations as well as students from cooperating partners and thus addressed a variety of nationalities and countries. These include a number of non-European countries where some of the students of the partner universities originate from. In the mid-term survey 2019, the number of countries where the respondents have or will earn their degrees (master, PhD or both) amount to a total of 21. Regarding the master’s degrees a little less than half of them were achieved in European countries (47,9%). Among the PhD degrees (already earned or still to be earned) 57,6% were located at European universities.

    Question: In which country did (or will) you earn your degree(s)?

     

    master’s degree

    PhD degree

     

    Quantity

    Percentage

    Quantity

    Percentage

    Austria

    1

    1,4%

     

     

    Brazil

    1

    1,4%

     

     

    Canada

    1

    1,4%

    2

    2,8%

    Denmark

    1

    1,4%

     

     

    Egypt

    1

    1,4%

    1

    1,4%

    Germany

    7

    9,9%

    6

    8,5%

    Greece

    2

    2,8%

     

     

    Hungary

    3

    4,2%

    1

    1,4%

    India

    12

    16,9%

    5

    7%

    Italy

    5

    7%

    5

    7%

    Macedonia

    1

    1,4%

     

     

    Malaysia

    1

    1,4%

     

     

    Nigeria

    5

    7%

    3

    4,2%

    Portugal

    7

    9,9%

    3

    4,2%

    Romania

    1

    1,4%

     

     

    Serbia

    1

    1,4%

    1

    1,4%

    Slovenia

    2

    2,8%

    1

    1,4%

    Turkey

    1

    1,4%

    1

    1,4%

    UK

    2

    2,8%

    1

    1,4%

    USA

    2

    2,8%

    1

    1,4%

    France

     

     

    1

    1,4%

    international

    (binational doctoral programme)

     

     

    1

    1,4%

    Total

    57

    80,3%

    33

    46,5%

    No mention

    14

    19,7%

    38

    53,5%

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    Table: Countries where Respondents did (or will) earn their Degree(s)

    With regard to the respondents’ study status at the time of the survey approximately three out of four respondents were still studying or intended to continue their master studies with PhD studies (53 respondents, 74,6%, green background in the pie chart). 15 respondents (21,2%) on the other hand had finished their studies already (blue background in the pie chart). 14 respondents were already employed at the time of the survey.

    Figure: Study Status

     

    Results

    Like in the years before during the COMPALL evaluation[1] the mid-term survey shows that – apart from being a very memorable and valued event – there are verifiable effects the Winter School has on its participants’ further study and work careers.

    Firstly, regarding their further studies the great majority of the respondents felt quite encouraged to take up next steps of studies (PhD). Also, the Winter School obviously succeeded in its intention to motivate and encourage students to deal with international or comparative perspectives in their studies respectively their theses.

    Figure: Effects on Thesis

     

    Secondly, the respondents confirm considerable effects on their qualification profile. All competencies the questionnaire referred to – including intercultural competencies, network competencies, methodological competencies for comparative research, language competencies, analytical competencies and knowledge on adult education and lifelong learning in other countries – achieved very high ratings.

    Figure: Competencies - Effects of the Winter School

     

    Thirdly, the survey shows that the Winter School indeed works as a platform that establishes lasting professional and personal contacts and networks among its participants. Even if the respective ratings turn out to be a little lower compared to other findings, the very big majority felt to be part of an international network they can use according to their needs.

    And fourthly, according to the large majority the Winter School has a noticeable influence on the respondents’ interest and motivation to work in international contexts or on international issues. At the time of the survey about two third of the respondents being employed dealt with international aspects in their work (considering that the total number of respondents at the time was rather low). Nevertheless, the fact that the very big majority of the respondents not yet employed also hoped to deal with international aspects in their work one day supports the finding that the international dimension continues to be a relevant factor to the students. Further, a considerable number of the respondents felt that the Winter School was able to enhance their (future) chances on the labour market. Almost two third felt it could have enhanced their chances for an academic career.

    Figure: Effects regarding Work Issues (Respondents currently not working)

     

    Finally, the open-ended answers also support the impression that the Winter School seems to be able to offer an effective mix of knowledge transfer, training opportunities for several research, professional and personal competencies and also chances of networking. Summarizing all these facts, one can state that – according to the participants – the Winter School provides an effective learning setting which offers significant inspirations regarding the respondents’ vision of their own development and career and also regarding their competence profile.

    An aspect interesting to follow in the further evaluations might be the question whether there are any differences between the students coming from different countries of study. However, the limited number of cases does not allow such analysis at the moment. Also, a closer look on the group of practitioners, who were included in 2019 for the first time, might be of interest in the mid-term surveys scheduled for 2020 and 2021. This will depend on the number participants and thus potential respondents in the Winter Schools to come.  

     


    [1] As mentioned before the Winter School of the predecessor programme COMPALL also was monitored by an extern evaluation. In the years 2017 and 2018 a mid-term survey kept track of the effects of the programme.

     

    COMPALL (2015-2018) evaluation

    You here can find the results of the evaluation of the ERASMUS+ strategic partnership COMPALL (2015-2018).

    Evaluation on the impact of the joint module – outcomes of the Winter School

    For assessing the development of the joint module and for measuring the impact of the intensive programme, the Strategic Partnership has implemented a formative evaluation, both qualitative and quantitative that accompanied the whole development process and critically reflected the implementation of all intellectual outputs – within and beyond the learning activities. The evaluation is divided in an external evaluation and an internal evaluation. The external evaluation was conducted by Susanne Lattke of the DIE (German Institute of Adult Education). The evaluation design and interpretation was developed in collaboration with the COMPALL Project Coordinator, Prof. Dr. Regina Egetenmeyer (Julius-Maximilians-University of Würzburg). All citations on this webpage are part of the internal evaluation reports. The evaluation design involved both short and mid-term evaluations of the Winter Schools defining indicators at the participant and the organizational level.

    For the short-term evaluation, participants were requested to fill in a paper questionnaire at the final session of each Winter School. The questionnaires of each year were rather identical with adopted questions on specific components. In addition, a number of interviews were conducted with selected participants during the last days of the Winter School. Information from project documents (e.g. COMPALL application, International Winter School brochure) and information orally provided by the organisers (e.g. regarding total participant numbers) have complemented the data, which was obtained through the surveys and interviews. 

    For the mid-term evaluation, participants of former Winter Schools were requested to fill in an online-survey after more than one year of their participation in the Winter School. Within the COMPALL project two mid-term surveys will be carried out. A first survey was conducted in spring 2017. It addressed all participants from the Winter School 2016 and, in addition, all participants of the previous pilot editions of the years 2014 and 2015, which took place before the COMPALL project. A second midterm survey is taking place in spring 2018 targeting all cohorts of the programme. 

    Both evaluation methods form the comprehensive picture on the impact of the implemented intensive programme focusing on the following targets:

    “The main focus of the short-term evaluations, in a formative perspective, is on participant satisfaction and on the quality of the Winter School and its related offers. Also, the short-term evaluations focus on the academic and personal outcomes and benefits which the participants have gained from the Winter School, mainly in terms of learning outcomes and increased motivation for international activity.” (Midterm-Term Evaluation Report 2017, p.5) 

    “The mid-term surveys, by contrast, focus primarily the development of former participants’ professional careers and/or further studies, and the impact which the Winter School has had on these.” (cf. ibid.)

    The structure of the external evaluation followed a six-step process: 

    ·         Step 1: Ascertainment of the evaluation concept for the three year, development of indicators into questionnaire items

    ·         Step 2: Student questionnaire used for the first cohort after the first intensive phase

    ·         Step 3: Student questionnaire used for the second cohort after the second intensive phase

    ·        Step 4: Mid-term student questionnaire used for asking the first cohort for longer-term effects

    ·         Step 5: Student questionnaire used for the third cohort after the third intensive phase

    ·         Step 6: Mid-term student questionnaire used for asking all cohorts for longer-term effects.

     

    In addition to the external evaluation, the Strategic Partnership conducted an online survey on the developed online tutorials piloted in the intensive programmes 2017 and 2018. A short summary of the results is provided in the corresponding Abstracts.

     

     

     

    Mid-Term Evaluation 2018

    Extraction of the external evaluation results, 

    which was conducted by German Institute for Adult Education, Susanne Lattke 

    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the Mid-Term Survey

    The midterm survey 2018 was implemented as an online survey. All students who had participated in the Winter Schools in the years 2014-2017 (n=279) received an invitation to fill in the questionnaire which they could easily access through a link which was provided together with the invitation. The years 2014-2015, which do not belong to the ERASMUS+ COMPALL-project where included to ensure a broader database and to raise the value of the data for the analysis of the impact of the COMPALL-project. For data privacy reasons, the invitation was sent by the organisers of the Winter School and not by the evaluator (the author of this report). The questionnaire opened between 24 April and 29 May 2018. Three reminders were sent within this period to reach as many former participants as possible. Ultimately, the questionnaire was fully completed by 68 respondents, which corresponds to a response rate of 24.4%.

    Labour Market

    The midterm survey 2018 shows that a high proportion (three quarters) of respondents currently working are actually facing international aspects in their work, and a similar proportion of those
    currently not working are highly motivated to engage with international issues in their future activity. Respondents not only find an international dimension in their future work highly attractive but also consider this perspective as rather realistic for themselves. Even cross-border mobility for finding employment in another country is considered a likely option by a considerable share of the respondents, in the 2018 survey even much more (mean 4.02) than in 2017 (mean 3.48). Regarding the Winter School’s impact, respondents confirm in particular that the Winter School has strengthened their motivation for international work, but also that it has contributed to developing competencies which are considered as useful in view of labour market and career perspectives. Considerable shares of respondents, both currently working and currently still studying, are also convinced that the Winter School participation has enhanced their competitive advantage on the labour market and, in particular, within the scientific community, in case they aim for a career in that context.

     

    Networking Impact

    About two thirds of the participants have started close collaborations with other participants after the Winter School and almost all respondents have the feeling of being part in an international network which may prove useful to them in case of need. These findings were consistent in both midterm surveys, 2017 and 2018. The finding from the midterm survey 2017, that about 25% of the respondents were still rather active users of the LinkedIn-networks was also repeated in 2018. The number of passive users was with about 60% of the respondents clearly higher. The online linkedIn networks thus seem not to be the most important tool for keeping in touch, but still they offer an additional resource which is used at least passively by a considerable share of the participants.

    Impact on PhD Study

    Another aim of the Winter Schools is to increase the number of highly qualified graduates in the field of adult education in general, and in view of an international study profile in this field in particular. As in 2017 and to an even higher extent, the midterm survey results 2018 show that the Winter School has strengthened in many respondents the motivation to take up a PhD study. Those respondents who have already started their PhD research also believe that the Winter School has contributed to the methodological and analytical quality of that Research.

    Impact on International Dimension of Study

    The majority of respondents had actually included international perspectives in their Master or PhD thesis or were intending to do so. Respondents also stated that the Winter School had had an impact on their thesis. Especially respondents who, at the time of answering the survey, were currently studying for a PhD or who had just finished their Master studies and were planning to take up a study rated this impact as rather high. The same pattern could already be observed in the midterm 2017 survey, with even higher impact rates at that time. It seems likely that these particular groups have participated in the Winter School during a “crucial” phase in their studies; a phase, in which many issues regarding a possible follow up PhD research may not have been firmly fixed but had still to be decided and shaped, so that the Winter School could come in as an influential experience.

    Personal and Professional Outcomes

    Looking back from a distance, most participants had very positive views regarding their individual outcomes from the Winter School. Mmore than three quarters of the respondents agreed that the Winter School had considerably helped them in both their professional and personal development, and respondents were nearly unanimous in saying that the Winter School had been an enjoyable experience.

    Participant Quotes from open Questions:

     

    Evaluation of Winter School 2018

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the 2018 Evaluation

    The Winter School 2018 evaluation is based on the paper questionnaire which was distributed by the working group leaders to all participants on the final day of the Winter School. The International Winter School 2018 was attended by a total of 61 participants. The number of filled questionnaires amounted to 59, so that the response rate is 96.7%. About two thirds of the participants (42) were Master students and one third (19) were PhD students. The sample of respondents reflects the same 2:1-relation between MA and PhD students (31 Master vs. 14 PhD). 14 respondents chose not to indicate their study level in the questionnaire. 

    Participants’ Satisfaction

    The Winter School in 2016 had set out already on a very high level, as far as both participant satisfaction and perceived benefits/impact of the Winter School are concerned. The Winter Schools in 2017 and 2018 not only have maintained these level but managed to achieve even further improvements in the participant ratings over the three years. The overall assessment shows that the participants’ general satisfaction with the International Winter School was very high. The mean value was 4.42 (n=48) which means an improvement compared to the already very high values of the two previous years (4.20 and 4.24 respectively).

    Again, participants’ satisfaction turned out to be very high for all aspects with each single mean value being above 4.0. This, too, means a further improvement compared to the previous year(s), although values had been rather high already in 2016 and 2017. In particular, satisfaction with the preparatory phase has increased compared to 2017. This show that the new preparatory offers (video tutorials, online information tools) which have been developed by the organisers since the last year, are well accepted by the participants. Complaints from the first year regarding the academic content of the Winter School or the composition of the participants group (uneven skills levels, e.g. regarding English proficiency, or unbalanced geographical distribution) have greatly diminished over the years. This suggests that the organisers have been successful in establishing an effective recruiting policy which ensures that potential participants are well informed about what they have to expect from the Winter School and what is required from them for a successful participation. At the same time, the geographical distribution of participants’ countries of origin became more balanced over the years: the numerical dominance of a few countries was reduced and it became easier to compose the working groups within the Winter Schools in a way to include a broad range of different country perspectives.

     Participants’ Competence Development

     In the later years 2017 and 2018, the focus on academic and professional content has become clearer, whereas in 2016 participant feedback had still focused very prominently on intercultural and social outcomes, including networking opportunities. These intercultural and social aspects remained important also in 2017 and 2018, however they were no longer the dominating feature in the participant feedback. Rather, the Winter School increasingly appears as a learning offer which combines in an integral way specific subject-related competence development with the development of transversal competences and personal soft skills, besides offering a highly enjoyable experience and an excellent opportunity for building up personal and professional networks.

     

    Impact Career on Career Chances

    The career relevance of the Winter School (outcomes) have been unchangeably rated positively by the participants in all years, which suggests that the Winter School concept reflects well the competence needs in both an academic context and the labour market.

    Summary of external evaluation

    One can conclude that the overall concept of the Winter School seems highly sustainable and does not need to be changed in a substantial way. There will be a need to secure the necessary financial means for the running of the Winter School after the Erasmus project funding has ended. Since the Winter School can rely on a solid consortium of partners and since during the project’s lifetime a substantial basis of materials was built up which can be used also in the future, there are good prospects for the Winter School to become a sustainable offer in the long-term. An appropriate information policy to reach and support (prospective) participants seems essential for the future success of the Winter School. A substantial part of the critical remarks made by participants in all years can be attributed to wrong expectations on the side of the participants or to a (felt) lack of orientation or preparation. The Winter School organisers are aware of this issue and have already developed with some success their information and support offers over the years. Also in the future, this issue of information and preparation of the participants should receive particular attention, as its impact on the Winter School success and the satisfaction on the participants seems considerable. The preparation phase underwent substantial developments over the three years, with many new materials (videos, tutorials, guidelines) being developed to support participants in their own work prior to the intensive phase in Würzburg. These materials are clearly appreciated and found helpful by the participants. They, too, should be kept as a regular feature and be updated as needed. By way of a concluding overall appreciation, it can be stated that the International Winter School obviously provides a very attractive and highly effective opportunity to students from (potentially) all over the world to learn and develop as an individual and as a (future) professional in the field of adult education in an increasingly globalized context. Moreover, the Winter School has shown its ability to develop and improve further over the years so as to satisfy participants’ needs and expectations to an ever higher degree.

     

     

    Evaluation 2017/2018

    How do you asses the online-tutorials all together?

    How helpful do you estimate the online-tutorials for your preparation to the Winter School?

     

    Mid-Term Evaluation 2017 

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the Mid-Term Survey

    The midterm survey 2017 was implemented as an online survey. All students who had participated in the Winter Schools in the years 2014-2016 (n=189) received an invitation to fill in the questionnaire which they could easily access through a link which was provided together with the invitation. The years 2014-2015, which do not belong to the ERASMUS+ COMPALL-project where included to ensure a broader database and to raise the value of the data for the analysis of the impact of the COMPALL-project. For data privacy reasons the invitation was sent by the organisers of the Winter School and not by the evaluator (the author of this report). The questionnaire was open between 25 April and 21 May. Three reminders were sent within this period to reach as many former participants as possible. Ultimately, the questionnaire was fully completed by 65 respondents which corresponds to a response rate of 34,4%.

    Labour Market

    Regarding work related issues, the Winter School had a particularly high impact on increasing participants’ motivation to work on international issues. This was especially true for respondents who were not (yet) working (mean 4.13 on a five-point-scale). About half of the respondents not (yet) working considered that the Winter School had notably (rating ‘4’ or ‘5’) contributed to their employment perspectives, by giving them a competitive advantage on the labour market and by providing them with information or contacts which would be useful in finding employment (ibid.). The proportion of respondents currently working who were of the same opinion with regard to their current employment was only slightly lower. It ranged between one third and almost one half of the respondent for the different items.

     

    Networking Impact

    Two thirds of the respondents stated that they had started a close exchange or cooperation with some fellow participants, and 75% of the respondents felt they had made some really good friends at the Winter School. The linkedIn networks seem to play only a subsidiary role in that. Only about one quarter of the participants stated that they were very using these networks very actively. About two thirds were at least following the news in the networks more passively. This reduced active use of the network seems however not to compromise the Winter School’s impact as such. As many as 86% of the respondents stated that they were feeling part of an international network of people whom they can contact at any time if need be, no matter if they were permanently in touch with them or not.

    Impact on PhD Study

    About half of the respondents who were currently preparing or studying for a PhD or who had already finished their PhD agreed to a high extent with the statements that the Winter School participation had strengthened their motivation to take up a PhD, that it had contributed to its analytical quality and that it had helped them to better cope with methodological research issues. Furthermore, at least two respondent have obviously found their (co)supervisor for their PhD research through the Winter School. Of those respondents who were not (yet) actively dealing with PhD studies (n=32), 24 stated that it was somehow likely that they would take up a PhD research in the future.

    Impact on International Dimension of Study

    Again, about half of these respondents confirmed that the Winter School had considerably strengthened their motivation for taking up a PhD and that the Winter School had also provided them with ideas for a possible topic. As many as nine respondents agreed that the Winter School had a role in helping them to find a prospective (co)supervisor for their thesis.

    Personal and Professional Outcomes 

    Looking back from a distance, most participants had very positive views regarding their individual outcomes from the Winter School. When asked to rate different outcomes on a five-point scale, about 80% of the respondents agreed by rating “4” or “5” that the Winter School had considerably helped them in both their personal and professional development, and respondents were nearly unanimous in saying that the Winter School had been an enjoyable experience. Regarding improved career chances, about half of the respondents rated the Winter School’s impact “4” or “5”: 45% of the respondents saw a good impact on their labour market chances in general, and almost 60% said the same with respect to a career in the scientific community, in particular.

    Evaluation of Winter School 2017

    Extraction of the external evaluation results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL- Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

    Participation in the 2017 Evaluation

    The Winter School Evaluation 2017 is based on the paper questionnaire which was distributed by the working group leaders to all participants on the final day of Winter School.
    The number of filled questionnaires also amounted to 84, so that the response rate would be 100%. Master students were with 50 respondents the biggest group compared to 32 PhD students. Two respondents chose not to indicate their study level in the questionnaire. (Master students: 59, 5 %, PhD students: 38, 1, other/no answer: 2,4%).

    Participants‘ Satisfaction

    The overall assessment shows that the participants’ general satisfaction with the International Winter School was very high. The mean value was 4.24 which is almost exactly the same as in 2016 (4.20).

    Again, participants‘ satisfaction turned out to be very high for all aspects with means ranging between 3.8 and 4.15 for almost all aspects. There were only a few comments given in the questionnaire regarding the preparatory phase. Nevertheless, one should note, that especially the preparatory phase is still under development and new features, such as online tutorials, were newly introduced and piloted fort the first time in 2017.

    Participants‘ Competence Developement

    The ratings are very similar to 2016. In a general perspective, participants judged their academic and personal outcomes from the International Winter School very positively. Regarding specific competences, respondents clearly feel that the International Winter School has considerably contributed to their competence development in many different regards, including both “soft” skills” like intercultural competence and “hard” academic skills like methodological research skills.

    Impact Career on Career Chances 

    The career relevance of the Winter School have been rated very high. It shows that the competences which the International Winter School addresses are considered by the participants as highly relevant for their career chances.  

    Summary of external evaluation

    As in 2016, the findings from the Winter School evaluation provide a very positive overall picture. Regarding the perspectives for the future development of the International Winter School, one can conclude that the overall concept of the Winter School seems highly sustainable and should not be changed in a substantial way. By way of a concluding overall appreciation, it can be stated that the International Winter School obviously provides a very attractive and highly effective opportunity to students from (potentially) all over the world to learn and develop as an individual and as a (future) professional in the field of adult education in an increasingly globalized context. Most participants had already dealt with international topics at least to some extent, but for about one half of the participants this was their first international study mobility. Participants’ motivations for attending the International Winter School were multidimensional but the desire to meet people from other countries and to learn about other countries were particularly stressed. Regarding participant satisfaction and impact, the evaluation shows that the International Winter School has succeeded in maintaining the very high level of the previous year in all respects. The Winter School provides a multifaceted and at the same time coherent learning opportunity which makes a valuable contribution to the participants’ both personal and professional development as adult education researchers or experts.          

    Evaluation of Winter School 2016

    Extraction of the external results,
    which was conducted by German Institute for Adult Education, Susanne Lattke
    COMPALL-Project Coordinator: Julius-Maximilian University of Würzburg, Prof. Dr. Regina Egetenmeyer


    Participation in the 2016 Winter School

    The Winter School 2016 was attended by 91 participants from 19 countries. Of these, 82 participants filled in and returned the questionnaire, corresponding to a response rate of approx. 90%. Master students (51 respondents) outnumbered PhD students (23). The proportion of Non-European participants (at least 25) was considerable, but the focus was clearly on participants from European countries (at least 50).

    Participants’ Satisfaction

    The overall satisfaction with the Winter School was very high. The overall means was 4.20. Participants also appreciated the opportunity to learn about international concepts and about the situation in other countries and to get new ideas and perspectives. Another aspect which was highlighted was the active involvement of students through group discussions, group/team work and self-responsible work of students.

     

    Participants’ Competence Development

    The ratings show that respondents feel the Winter School has considerably contributed to their competence development. The lowest – but still high – mean values were obtained by English language competencies” (mean 3.8) and methodological skills for comparative research (3.9). All other items – analytical competencies, being able to see adult education in one’s own country in a new light, professional networking, interacting with people from other cultural backgrounds, and understanding adult education/lifelong learning in other countries - scored above 4. Overall, participants judged their academic and personal outcomes from the Winter School very positively (means of 3.7 for academic and 4.3 for personal outcomes). The overall picture thus shows that respondents feels they have obtained a broad range of learning outcomes with the foci being very much in line with the intended aims of the Winter School.

     Impact Career on Career Chances

    All competencies were attributed a high importance by the respondents with the mean values ranging from 3.93 to 4.74. If one compares, how respondents rated the career relevance of the different competencies addressed by the Winter School on the one hand, and how they assessed the Winter Schools’ contribution to developing exactly these competencies on the other hand (see section 4.2.2) one can conclude that the Winter School made a significant contribution to enhancing the participants’ future career chances.

    Summary of external Evaluation

    The evaluation showed very high satisfaction levels of the participants concerning the Winter School as a whole as well as its single parts. As regards the Winter Schools’ impact, participants felt that the Winter School had considerably contributed to enhancing their competences in many regards. Most notably, increased competence for intercultural interaction was highlighted as outcome, but also the development of other competences, both content-related and transversal ones, was felt to have been promoted by the Winter School. Participants also considered that the competences which were promoted by the Winter School were highly relevant for their further career chances, thus contributing to their employability. After attending the Winter School, participants also felt more motivated to engage with international topics in their studies or future professional activities as well as to undertake further study mobilities. 
    In conclusion, the evaluation showed that the Winter School reached a high number of its target audience, involving different countries of origin, study levels, motivations and degrees of previous international experience. By all these groups the Winter School was perceived as a high quality event which had much to offer for their own academic/professional as well as personal development.