Unofficial translation from Lithuanian

RULES OF QUALITY ASSESSMENT FOR INSTITUTIONS
OF RESEARCH AND HIGHER EDUCATION

Approved by Order No. 1055
of the Ministry of Education and Science
of the Republic of Lithuania
on June 28, 2001

       I General Part

        1. Rules of quality asessment for institutions of research and higher education (hereinafter – the Rules) are intended for regulating the quality assessment of the activities of state and non-state institutions of research and higher education as well as study programmes quality, research and development quality and general qualitative assessment of institutions of research and higher education of the Republic of Lithuania. The quality assessment is organized and performed by the Centre for Quality Assessment in Higher Education (hereinafter – the Centre).

2. The Government of the Republic of Lithuania or the Ministry of Science and Education (hereinafter – the Ministry) may initiate the quality assessment of the activity of research and higher education institutions, and to commit to execute it to special expert groups or the Centre indicating concrete aims and order of assessment.

3. The major purpose of the quality assessment is to stimulate institutions of higher education and research, their subdivisions and all the scientists to seek a clear perception of the needs of Lithuania, the mission, objectives and tasks of the institution, to facilitate for the institution to reveal of its weaknesses and strengths, and to provide for raising its work efficiency.

The quality assessment is intended to determine whether the institution of research and higher education meets the qualitative requirements formulated in the legal acts approved by the Government of the Republic of Lithuania and the Ministry.

The quality assessment has to provide information about the work efficiency and level of activities (in comparison with the international level) of institutions of research and higher education, about the recommended state measures for improving the system of research and higher education, and about the quality of study programmes offered by the institutions of higher education.

4. The quality assessment consists of two parts - an internal quality assessment (a self-assessment) of an institution of research and higher education, and its external quality assessment (peer review) performed by expert groups formed in accordance with these Rules.

5. Self-assessment is the basic part of any assessment. An institution evaluating its own activities formulates its objectives and tasks , analyses its activities applying different approaches, reveals its activity problems and foresees the ways of resolving them.

A self-assessment is regulated by the senate (council or academic council) of an institution of research and higher education. A head (or senate) of an institution appoints officials that are constantly responsible for the institution's activity quality and its evaluation. An institution of research and higher education performs its self-assessment at regular intervals of time established by the senate (council or academic council). A self-assessment is to be carried out before each external quality assessment.

6. External quality assessment is organised by the Centre according long term (a few-year) and annual external assessment plans. Long-term plans establish which subject areas of research and higher education are to be evaluated in respective years, while annual plans are intended to schedule the evaluation of study programmes, institutions and their subdivisions (indicating the year quarter). The Centre in co-ordination with the Rectors’ Conference of Lithuanian Universities, Directors’ Conference of Lithuanian College’s, the State Research Institutes Directors’ Conference and the Science Council of Lithuania prepares the draft plans. Long–term and annual external assessment plans are to be approved by the Minister of Education and Science (hereinafter – the Minister).

The time of any scheduled external assessment (the term to bring a self-analysis to the Centre) is to be known to an institution of research and higher education to be assessed not later than three months before the external assessment.

7. The Ministry forms an Expert Council for activity assessment of research and higher education institutions (hereinafter – the Expert Council), to coordinate the quality assessment process of research activity and study programmes in research and higher education institutions, to summarize conclusions of the assessment. The Expert Council is acting according to regulations approved by the Department of Science and Studies under the Ministry of Science and Education (hereinafter – the Department).

8. External quality assessment is planed on the basis of three streams:

a); Evaluation of study programmes (covering all study subject areas over 8 years);

b) Research and development evaluation (covering all research areas over 8 years);

c) Qualitative assessment of research and higher education institutions (covering all the higher education institutions over 8 year).

9. If quality of any evaluated unit (a study programme, a research and development or an institution of research and higher education) is dissatisfied and competent State institutions or founder does not take a decision to stop the activity of the unit, the Centre has to organize a repeated assessment of the unit after one or two years.

10. The procedures of external quality assessment are the following:

a) Experts’ analysis of the self-assessment of an institution of research and higher education,

b) Experts’ site visit for obtaining information about the unit being assessed,

c) Formulation of the experts' report and discussing it with the representatives of the institution of research and higher education,

d) Delivery of the external assessment report to the assessed institution of research and higher education and the Centre,

e) Assessment report (summarized by the Council of Experts if needed) delivery to the Department,

f) Publication of the summarized final assessment report in an informative issue of the Centre.

11. After receiving the external assessment report, an institution of research and higher education examines it according to its internal regulations, and not later than in three months works out its action programme based on the experts' recommendations. This programme is presented to the Centre and the Department.

12. Every year in February the Centre renders an account to the Department on external quality assessment of research and higher education institutions during the past calendar year.

II Formation of Expert Groups

13. Experts are proposed to the Centre by:

a) the institutions of research and higher education, the Science Council of Lithuania, the Lithuanian Academy of Sciences;

b) boards or councils of professional societies (e.g. doctors' unions, engineers' unions, scientists' unions, students’ unions of research and higher education institutions), creative organisations;

c) ministries or other state institutions concerned with higher education or research;

d) scientists and other experts with experience of evaluation.

14. The experts proposed to the Centre are to satisfy the following requirements:

a) research and pedagogical activity of proposed experts has to be active and efficient during last five years, i. e. they have to be published sufficient number (minimal qualitative requirements for lecturers and scientists) of scientific articles in research publications that are on the list of Institute of Scientific Information, and a list, approved by the Department;

b) other proposed experts are to be familiar with the problems of higher education and research of Lithuania and with the foreign experience in respective subject areas.

The recommendation of experts is based not only on their professional competence, the expert work experience, but also on such personal qualities as adherence to principle, goodwill, fairness and ability to make impartial evaluations, readiness to realise progressive ideas.

15. The Centre organises workshops during which the experts and the officials of institutions, concerned with the internal evaluation of institutions of research and higher education, get acquainted with the aims, tasks and procedures of the assessment as well as with the practice of other countries.

16. For any external assessment the Centre forms an expert group or several groups taking experts proposed specially for that assessment.

The Centre can invite foreign experts who work together with the Lithuanian experts or forms separate groups.

Scientists from other subject areas than an evaluating subject area (field) can be involved in expert groups.

17. An expert group for whole research and higher education area assessment consists of not less than 3 individuals, and for general qualitative assessment of institutions of research and higher education not less than 5 individuals are required.

18. One member of the Expert Group is appointed as a leader of the group by the Centre. The leader distributes the work functions among the experts, organizes the assessment process, determines the scale of work to be performed by the experts, and is responsible for a proper presentation of the assessment report and recommendations in time.

19. Expert Groups are formed by the Centre on the basis of the following principles:

19.1. All experts are to possess an adequate competence for evaluating the activities of institutions of research and higher education according to a planned task. 

19.2. An expert Group contains individuals from different institutions, having experience in research, teaching and expert work.

19.3. More than a half of the experts in a Group are to be employees from institutions of research and higher education.

19.4. The relationship between the experts interests and the unit to be assessed is determined as definitely as possible, seeking elimination or reduction of a probable conflict between the interests of the experts and those of the expertise. The leader of the Expert Group must be impartial.

20. Institutions of research and higher education to be assessed can propose to the Centre their candidates to an expert Group beforehand, arguing their suitability. The Centre informs all the institutions to be assessed with the planned composition of an expert Group. Institutions have the right to pay the Centre's attention to the fact that for some grounded reasons some planned experts are unacceptable. Institutions can propose to eliminate some experts from Expert Group arguing his/her unacceptability.

21. Experts’ work is paid by the Centre according to the concluded agreements from certain state budgetary allocations assigned for this purpose, and from other resources.

III Assessment of Study Programmes

22. These Rules regulate the assessment of all sequential (university and non-university) study programmes. Study programme to be assessed means not only a programme description, but it also includes a programme realisation or ability to realize it (in a case of new programmes) in an institution of higher education.

23. External (expert) assessment of a study programme may be exhaustive or partial. Annual quality assessment plans contain indications, which of the two assessment types is applicable. A planned exhaustive external assessment is to be preceded by an exhaustive internal assessment.

24. The task of a partial assessment of a study programme is to find out whether it satisfies the requirements of sequential study programmes regulations approved by the Minister. A checklist of subjects (topics) to be analysed during such an assessment is presented in Appendix 1 of these Rules. For a partial external assessment, experts only consider the submitted internal evaluation material and present their critical remarks and conclusions on it.

25. In case the experts indicate, during a partial assessment, that a study programme does not satisfy all the requirements specified, the Centre has to organise an exhaustive assessment of whole study programme after one year. 

26. An additional task of an exhaustive assessment of a study programme is to reveal its strengths and weaknesses, the shortcomings and their causes in realising the programme as well as to publish the quality assessment of the programme and to recommend ways of its improvement. A checklist of assessment subjects (topics) to be discussed during such an assessment is given in Appendix 2 of these Rules. For an exhaustive assessment, an experts' visit to a higher education institution as well as a discussion of the assessment report with the study programme leaders are required.

27. During an exhaustive assessment research activities of the academic staff realising the study programme are also evaluated (see Paragraph 35 of these Rules). Professional experience of pedagogical staff teaching professional subjects as well as of teachers of artistic subjects is also assessed.

28. When assessing a fresh study programme proposed, it is to be determined whether a proposed new programme is needed and whether an institution proposing the programme is capable of realising it successfully. Proposing a new study programme an institution of research and higher education has to present all study programmes of that subject area, that are active in the institution. A checklist of additional subjects (topics) to be analysed is presented in Appendix 3 of these Rules. The Department takes the decision proposing the Ministry to register a new study programme, which was evaluated positive.

29. When assessing study programmes for providing professional qualifications it is established whether a programme satisfies the state requirements for that particular profession. If an assessed unit is non-university study programme, it is established, whether it is up to standards of such professional training.

IV Assessment of Research and Development

30. These Rules are intended for evaluating the quality of research and development in a scientific subject area (sometimes in a subjects field) at an institution of research and higher education.

31. Quality assessment of research and development is concerned with scientific efficiency, significance and viability of research activities over a specified period of time.

32. An assessment of the efficiency of research is based on the indicators given in Appendix 4 of these Rules, and development efficiency – in Appendix 5. Also it is taken into consideration supplementary indications given in Appendix 6.

Efficiency of research activity may be assessed considering not only quantity indications, but also quality parameters of research efficiency.

For assessing the research quality of an institution of research and higher education (or of its subdivision) with a non-uniform scientific efficiency, it must be determined average indicator of efficiency as well as the research groups both of the highest and the lowest efficiency.

33. When assessing the scientific significance of research and development, two approaches are applied:

a) significance for the subject area;

b) significance for the science, culture and economy of Lithuania.

During an assessment the most important research results are determined (the chief publications, the most useful applications, and the like), their significance being argued.

34. In assessing the perspectives of research and development three approaches are considered:

a) an extent up-to date research (concerning the theories, methods and equipment applied in research);

b) the amount of sufficiently experienced scientists in an institution or its subdivision who are capable of performing all types of research work (both theoretical and experimental), the ways of developing scientific qualification and regularity of training a new generation of scientists;

c) the degree of sufficiency of demand for the research and development performed (in Lithuania and in the world), its probable research market competitiveness, its inclusion into international projects, number of commissions with names of customers.

The experts are to give recommendations for making corrections in the topics of the research in order to raise its prospects. During both internal and external assessment it should be revealed what other important problems can be successfully solved by these scientists if they receive support from the state or from some other sources.

35. When assessing the research activities of the faculty, department or of any other subdivision of an institution, the links of the research with the study programmes offered are considered:

a) the degree of contribution of the research to maintaining a required level of teaching the respective subjects,

b) the degree of the connection between the research and doctoral studies,

c) the number of students involved into the research and the extent of developing their skills of doing research.

36. When assessing the activities of a research institute or laboratory, the scientists' participation in the educational process is indicated.

37. After every assessment the Centre publishes the research and development efficiency indicators of institutions of research and higher education taken from their internal assessment (self-study) reports and supplemented with corrections according to the results of external assessment (if such an assessment was done that year).

V General Qualitative Assessment of Institutions of Research and Higher Education

38. Qualitative assessment of the whole institution of research and higher education, performed according to these Rules, embraces all its academic subdivisions as well as the whole combination of all its academic (education and research) activity spheres. Both an internal and external assessment of the whole institution of research and higher education is based on the previous assessment reports concerning the educational programmes as well as those of research in different subject areas, the reports being summarised and strategic matters considered. Only in the case of a small institution, the activity fields of that are not numerous, a general qualitative assessment of research and higher education can be coincide with an assessment of its research or study programmes.

39. For a general qualitative assessment of an institution of research and higher education, it is to be found out:

a) how an institution of research and higher education satisfies the qualitative requirements, to what extent its tasks and basic activities (research work, development, educational programmes, including artistic activities for institution of higher art education) correspond to the activity areas established by its foundation documents, statute (regulations) and other standard documents and to what degree an institution satisfies the state and public needs;

b) how an institution of research and higher education fulfils its tasks, what are the reasons of its failures, what are the valuable elements of its activity with which other institutions of research and higher education could be acquainted;

c) how an institution of research and higher education takes care of the quality of its activities, what is the effectiveness of its internal systems of quality maintenance, what was its response to its preceding external assessment reports and recommendations;

d) what is the concern of an institution of research and higher education for the perspectives of its activities (the qualifications of its scientists and teaching staff, their individual assessment and systems, its material resources, international co-operation and activity competitiveness);

40. Laws and other legal acts, regulating activity of institutions of research and higher education can be changed or corrected on the basis of a general qualitative assessment report.<

VI Quality assessment report

41. On completing an external quality evaluation, an expert group formulates its final assessment report.

42. Basic parts of an expert assessment report are the following:

42.1. Short description of the assessment process.

42.2. Assessment of the aims and objectives of a unit had been assessed (of a study programme, research and development or of the whole institution of research and higher education).

42.3. Basic quantitative indicators of a unit had been assessed (with short comments).

42.4. Quality assessment of a unit.

42.5. Assessment of current efforts for unit quality improvement.

42.6. Recommendations to an institution of research and higher education.

42.7. Recommendations to the state power institutions.

42.8. Summarized assessment about assessed unit quality.

43. An expert assessment report is written in accordance with the procedure laid down in the assignment of the Centre. All conclusions are to be clearly argued, grounded by the facts, and quantitative and qualitative indicators, analysis of indicators alternation during the assessment.

When assessing the activity of an institution of research and higher education, the existing situation is compared with the changes over the last several years (on the basis of systematic internal assessment reports presented by the institution), with the reports and recommendations of previous assessments (if any), and with the action programmes worked out (after assessments) by the institution itself.

Expert recommendations to an institution of research and higher education are associated with the previously formulated conclusions, with the aim of providing assistance in improving the quality of assessed unit. If it is recommended to restrict, limit or cease some type of activity, suggestions are to be made for utilising the material and intellectual resources available. One of the aims of the recommendations is to stimulate the actions of an institution, especially those of its leaders, to reveal the reasons of the shortcomings and to eliminate them.

The expert recommendations to the state power institutions are primarily oriented towards concrete means and actions, which could promote quality improvement in the activities of an institution. The attention of the state power institutions is attracted to the drawbacks in regulations for the activities of institutions of research and higher education and those in the law.

44. Summarised quality assessment of an object is marked by rating on a four-point scale. The ratings are defined as follows:

4 - an assessment only for perfect objects of an international quality level;

3 - an assessment for objects of good quality when institution orients its activity towards international level;

2 - an assessment for objects satisfying all or almost all qualitative requirements and the efforts of an institution towards quality improvement are significant.

1 - an assessment for objects of an unsatisfactory quality when they fail to meet many qualitative requirements and the efforts of an institution towards quality improvement are insignificant.

When assessing the quality of research and development, with summarized mark of quality assessment, separate marks of assessments have to be given for the scientific efficiency, scientific relevance and viability of the research (together with research and all scientific activity). The Centre preparing a task to experts can ask to give marks and to other aspects of assessment. 

The experts are to ground the determined quality grade (given mark) of an assessed unit on concrete facts.

45. The experts seek an agreement when formulating their conclusions. Otherwise formulations have to be accepted by a majority votes and the opposite opinions (authorised and argued) being also included.

46. The experts present the draft report of an external assessment (without mark of assessment) to an institutions of research and higher education to be assessed. The institution within 10 days' period has to indicate in written form the experts' mistakes and to make argued proposals for some modifications in certain conclusions and recommendations. On receiving these remarks the experts prepare the final external quality assessment report and submit it to an institution concerned and to the Centre.

47. The final conclusions of external quality assessment, which can stipulate important changes of research and higher education system or separate institutions’ of research and higher education activities, has to be delivered to the Expert Council by the Centre for evaluation and summarization.

48. The Centre informs other concerned institutions about the results of external quality assessment by endorsement of the Department. Summarized final report is to be published in informative issue of the Centre. Heads of research and higher education institutions have to be familiarized with the publishing final report of an assessment in advance. If they give some remarks on summary it has to be published together with the summary.

Appendix 1

Topics (issues) to be analysed during partial study programme assessment

P1.1. Study programme title and type (non-university, university, undergraduate, special professional studies, master, residence, one-stage studies).

P1.2. Obtained education (qualification).

P1.3. Duration of the study - the total number of credit points (or hours), the number of contact hours (the number of contact hours for the study by correspondence or in the evening departments, if any).

P1.4. List of study subjects classified to blocks according to Regulations of Sequential study programmes. The study load of each subject and block is indicated in credit points (or hours) and contact hours. Groups of alternative subjects and optional subjects are listed. Practical and other compulsory assignments are indicated (term paper and so on).

P1.5. Qualifications of the academic staff realising the programmes are described. The number of full-time staff members in higher education institution concerned is given.

P1.6. Description of teaching facilities provided (textbooks, computer facilities, etc.).

P1.7. Description of material programme provision (laboratory equipment, a base for practices, etc.).

P1.8. Variations in the number of students taking courses according to a programme concerned (over the last several years, according to the study years).

P1.9. If a study programme concerned provides for obtaining a professional qualification, links with institutions, determining qualitative requirement for the professional qualification, are indicated.

Appendix 2

Topics (issues) to be analysed during exhaustive study programme assessment

P2.1. Topics analysed during partial study programme assessment. (Appendix 1, P1.1-P1.9).

P2.2. The programmes (modules) of the courses offered, their validity (necessity) and quality, description of their logical interrelationship and sequence.

P2.3. Description of final (graduation) thesis and assignments (the experts select them with different marks for analysis).

P2.4. Programme demand characteristics (competition level for admission, information about the prospects of employment for the graduates, etc.).

P2.5. Student evaluations of the study. Opinion of employers and the profession representatives about the graduates.

P2.6. Information about study programme outside contacts (with prospective employers of the graduates, with other institutions of higher education realising similar study programmes, international exchange of the academic staff members and students.

P2.7. Information about the actions aimed at programme quality improvement in an institution of higher education.

Appendix 3

Topics (issues) to be analysed at fresh study programme submission

P3.1. Some topics analysed during partial study programme assessment (Appendix 1, P1.1-P1.4).

P3.2. Argumentation of the necessity of a study programme (its aims, a planned number of students, the advantages of the programme in comparison with similar programmes already existing in Lithuania, etc.). The programme is compared with programmes of the same study areas functioning in an institution of research and higher education, which renders a new study programme. If a new study programme not much differs (more than a half of a new programme extent coincides with existing programme) from an existing study programmes in an institution, it is not recommended to register the programme. Also experts checks if other existing programmes of a subject area are not specializations of one of the programmes. It is recommended to join such specializations into one programme. <

P3.3. Characteristics of the intellectual and material resources available to be utilised in realising a study programme concerned (prospective analysis of topics P1.5-P1.7 from Appendix 1).

P3.4. If a study programme concerned is intended for obtaining a professional qualification, an opinion of official representatives of the profession (societies, departments) about the necessity of the programme and its content is required.

Appendix 4

Research productivity indicators

P4.1. Number of issued scientific articles (including articles in electronic magazines as well as articles of which future publishing there is an official announcement of editorial office):

1.1. Issues that are on the list of the Institute of Scientific Information;

1.2. Other reviewed international and foreign research magazines;

1.3. Lithuanian scientific publications that are on the list approved by the Department; 

1.4. Other scientific publications.

P4.2. Number and scope of published scientific monographs:

2.1. Published abroad,

2.2. Published in Lithuania in foreign languages,

2.3. Published in Lithuania in Lithuanian language.

P4.3. Number of scientific reports presented at conferences (excluding ordered reports and reports at plenary meetings of large conferences):

3.1. Organized by international scientific organisations,

3.2. Organized by others abroad,

3.3. Organized by others in Lithuania.

P4.4. Number and extent of grants for research received (in Litas):

4.1. from international funds and programs,

4.2. from Lithuanian funds and programs.

P4.5. Number of Doctoral students:

P4.6. Average number of research employees during the assessment):

6.1. Doctors Habil,

6.2. Doctors,

6.3. Other research workers (without scientific degree, with university education),

6.4. Taking chief research employees position,

6.5. Taking senior research employees position,

6.6. Taking research employees position,

6.7. Taking junior research employees position.

P4.7. Number of research attendants:

7.1. Assistants (technicians, laboratory assistants, etc.) involved in research work,

7.2. Economy and administrative employees.

NOTE.

Assessing efficiency experts (and authors of self-analysis):

1. Analyse not only absolute quantitative indicators, but also comparative – how much of production fall on each scientist (6.1+6.2) or each research employee (6.1+6.2+6.3).

2. Includes only part of production, which falls on employees (eliminating part of production falling on co-authors from other institutions or its subdivisions) of institution (or its subdivision) being assessed.

Appendix 5

Indicators of development efficiency

P5.1. Scope of commissions of research and development (in Litas):

1.1. received from ministries, departments or other state institutions of Lithuania (except municipalities);

1.2. received from municipalities of Lithuania;

1.3. received from non-state Lithuanian subjects;

1.4. received from foreign subjects and international organizations.

P5.2. Number (except technologies used in other institutions) of formulated or improved and functioning new technologies (products, kinds of plants and animals, units of computer software, etc.).

P5.3. Number of sold licences.

P5.4. Number of received invention patents:

4.1. foreign patents

4.2. Lithuanian patents

P5.5. Number and scope (printer’s sheets) of published books:

5.1. manuals;

5.2. dictionaries;

5.3. textbooks and other training issues;

5.4. other books.

P5.6.  Number of formulated standards, regulations and other drafts of legal acts.

P5.7. Number of accomplished research expertises:

7.1. for foreign and international organizations;

7.2. for Lithuanian organizations.

NOTE

Assessing efficiency, experts (and authors of self-analysis) follows note of Appendix 4.

Appendix 6

Additional indicators concerned with research and qualification

P6.1. Indicators for recognition of scientist’s competence:

1.1. number of members of Lithuanian and foreign Science academies

1.2. number of members of international and foreign scientific journal’s editorial staff;

1.3. number of members of scientific journal’s editorial staff recognized in Lithuania;

1.4. number of members of international research organizations office;

1.5. number of members of Lithuanian research organizations office;

1.6. number of members of Councils coordinating Lithuanian science and studies;

1.7. number of scientists receiving significant awards:

1.7.1. awarded by international and foreign institutions (indicating names of awards),

1.7.2. Lithuanian scientific awards,

1.7.3. national awards of Lithuanian culture and art,

1.7.4. other significant awards (i.e. state J. Basanavičius

1.8. total duration (in months) of Lithuanian scientists’ work and internship abroad;

1.9. total duration (in months) of foreign scientists’ work and internship in Lithuania;

1.10. number of scientists participated in passed (or approved) bills, programs, standards of the Republic of Lithuania;

1.11. number of international scientific conferences and other events of international level;

1.12. number of all Lithuanian level conferences or other events of the same level

P6.2. Number of published research popularization and research publicistics articles.

P6.3. Scientists’ educational indicators, expressed by number of scientists that during the assessed period:

3.1. acquired Doctor’s Habil degree,

3.2. acquired Doctor’s degree

3.2.1 those who published at least one article in an issue from the list of Scientific Information Institute;

3.3. were members of Habilitation committees;

3.4. were supervisors of doctoral students, who maintained doctoral dissertations during assessed period;

3.5. were opponents of doctoral dissertations.