The HCERES evaluation of the establishment’s departments: what lessons?

In order to be better prepared for their integration within Université Gustave Eiffel, IFSTTAR's departments and the establishment underwent so-called Wave E evaluations by HCERES. These were therefore conducted simultaneously with those of UPEM's laboratories. The evaluations of the departments were completed in 2019 with the release of the evaluation reports written by the inspection committees.

The evaluation of the establishment took place in autumn 2019 and the report was delivered at the end of the same year. All the institutions of the future University (EIVP, ESIEE Paris, ENSG, ENSAVT, UPEM, and IFSTTAR) were evaluated at the same time, which made the process highly complex.

What general lessons can be drawn from these evaluations from Université Gustave Eiffel's perspective? Clearly, both time and an intense effort are required in order to prepare for and receive the evaluation committees. The value of the process lies as much - and perhaps even more - in the capacity of the scientific teams themselves to take stock of their achievements and the future prospects for their work as in the evaluations themselves, in which the scientific aspect is sometimes pared down to the bare essentials. Nevertheless, the evaluation reports, which are available on line on the HCERES and the institution's website, give a very full picture of scientific activities in the departments and the institution and outline possibilities for the years to come.

 

 

HCERES Evaluation Report [.pdf]

An overview of the evaluation of the departments

The HCERES committee that inspected the AME department acknowledged the value of the department as "the only major multidisciplinary research structure in France likely to play a major role in decision-making", and noted that it runs smoothly and has very good internal unity, fostered by unifying scientific activities. The committee also drew attention to the very high standard of publications (for which efforts to obtain citations from the international community will have to be stepped up), very good relations with the socio-economic world and excellent facilities.

The department's scientific agenda, based on three themes (Mobility and systems of actors; Mobility and technical systems; Mobility and ecosystems) has been validated. For this agenda, the committee pointed out the potential benefits of interdisciplinarity, which can enhance both the questions that are posed and the approaches used to tackle them, without sacrificing a high level of technical expertise in each of the fields the department covers.

The inspection committee made a number of recommendations. The following are particularly relevant in the context of the creation of Université Gustave Eiffel: the need to continue to apply and to intensify an interdisciplinary approach, which is all the more necessary as subjects become more complex; to engage in forward thinking to identify and meet the challenges posed by today's mobility and transport issues; and to become more involved in doctoral training, which will become even more important following the creation of Université Gustave Eiffel.

Finally, one of the inspection committee's recommendations concerned the need to maintain the stability of the AME Department in a context of institutional and organisational change. This will be the main focus in the coming months. At the end of 2019 a process of reflection had already been initiated within the department and its laboratories on the best possible way(s) of implementing the department's scientific agenda and the research activities of its laboratories.

The inspection committee highlighted many very good aspects of the department’s policies during the last few years in terms of its strategy and approaches, ambitious agenda, strong relations with socio-economic players and international action. The report paints a glowing and inspiring picture of the science and innovation carried out within the department and stresses its influence and utility. It is unconditionally positive about the quality of the department's achievements and its links with the socio-economic environment. This evaluation answers most of the Department's doubts about its quality and direction, conferring legitimacy. 

The two-day visit was organised like a congress and provided an opportunity to present a coherent and comprehensive set of results and outstanding achievements. The garnered information has been used on several occasions since. The department has improved the management of its scientific activities in the themes it covers, in particular by creating specific workshops for each theme, mobilising all of the staff and systematically including colleagues from Université Gustave Eiffel, as was done during the department's seminar or the setting up of FUTURE projects.

Furthermore, in its governance, COSYS has also implemented task- and challenge-based management of its activities in the same spirit as Horizon Europe, in order to focus resources on a very limited number of critically important subjects so as, ultimately, to help the public authorities decide on the social choices they will have to make. The sharing of intelligence between the infrastructure and vehicles, the validation of artificial intelligence (AI) for automated mobility, the roads of the Anthropocene, or monitoring throughout the different scales of the territorial metabolism are examples of topics of this type that address two major challenges: making new types of mobility instruments of carbon neutrality and ensuring that we have breathable air.

The HCERES evaluation was on the whole very positive. In particular, the committee noted the effort to restructure and refocus the department's research activities. This has made it possible, in the context of a significant reduction in staff numbers and basic support, to increase and diversify budgetary resources and to relaunch a policy of investing in and replacing major facilities. This innovative, cutting-edge scientific equipment is operated by committed and competent technical staff.

The scientific output of GERS has increased during the period under evaluation (452 papers in international journals by the 48 researchers who published during this period), many of them in leading journals. The teams participated in a considerable number of competitive projects (ANR, FUI, European...). Previous efforts to join and lead European networks are beginning to bear fruit. The department's research work is clearly focused on themes that can have an impact on the economy and society. Interactions with the socio-economic world are varied and fruitful. Finally, the Department's staff play an active role in expert appraisals and standardisation bodies and, more generally, the inspection committee highlighted the efforts made to transmit research results to professionals, through learned societies and the general public. This part of the department's activities is deemed important and should be kept in the future.

The department's agenda, whose roots lie in the creation of Université Gustave Eiffel, was deemed to be clear and realistic. In particular, the committee recommends continuing the effort to increase international publications, paying particular attention to the publication of work by doctoral students. The department should also take advantage of the resources available at the FUTURE I-Site to reinforce its influence and international partnerships, by actively welcoming foreign researchers and encouraging the mobility of its staff. A more proactive policy should also be implemented to use patents to better exploit its scientific advances.

To conclude, within its remit, GERS appears to be a coherent and valuable research structure. The way the department is organised provides a high-quality working environment and facilitates interaction between the teams which are located at several sites. However, some leadership actions will be required to strengthen the staff's sense of belonging to the department and to position its activities within the Université Gustave Eiffel project, while taking care to maintain and develop the very fruitful local and regional partnerships.

The HCERES evaluation of the MAST department has strengthened the focus of its scientific agenda around four themes: the durability of construction materials, the control of ageing and risks in structures and infrastructure, the circular construction economy and innovations in infrastructure and construction. The inspection committee underlined the potential for synergy between topics linked to urban construction and the infrastructure for urban transport and networks. It encouraged the department's commitment to maintaining its skills and research base at the current level in order to monitor the sustainability and ageing of infrastructures, by further improving its effectiveness and efficiency in the service of public contracting authorities. The committee also recommended increasing the academic and international recognition of the theme of the circular economy.

The department reasserted its strategy with regard to striking a balance between activities and was encouraged to maintain an innovative approach, at the boundary between the academic and industrial spheres, through work based on its major experimental research facilities and through significant involvement in expert appraisal, certification and standardisation, which account for approximately 10% of its time. As part of Université Gustave Eiffel, MAST will therefore stress how important it is for these activities to receive academic recognition. The ways in which beneficiaries are listened to and the nature of partnerships will be revised in this new context in order to effectively manage and finance applied research and expert appraisals.

Finally, the department took careful note of the recommendation to make the creation of Université Gustave Eiffel into an opportunity rather than a threat. In this context, it intends to modify the existing organisational equilibrium between hierarchical structuring and leadership based on scientific projects in order to encourage synergies, while remaining watchful with regard to operational efficiency in an environment where the way resources will be organised and deployed is still undecided. MAST was encouraged to maintain its strengths in terms of major facilities and the expertise of its research support staff. In this connection, it took note of the encouragement to aim to manage its major experimental research facilities more efficiently. Finally, the Department noted the positive reaction to its internal seminars, for which a process of continuous innovation must be maintained so that they support scientific synergy between different areas of the department's work and the links between the different sites.

 

The HCERES inspection committee acknowledged the cross-cutting and motivating role of the TS2 department, which has "a collective vision and a clear scientific identity", permits "cross-fertilisation between interdisciplinary projects in its laboratories" and increases "their visibility at international and national levels in terms of research, expert appraisal and decision-making support". Concerning the department's agenda, the inspection committee agreed that the envisaged scientific avenues based on the 5 themes (automated driving, new road safety and acceptability issues; new road safety issues; mobility and health for all; modelling the human being; the design, development and operation of digital databases) correspond to major societal issues and that they are coherent, relevant and achievable. This should enable TS2 to consolidate its international standing. The committee also stressed the need, which is all the stronger in an interdisciplinary entity, for researchers and research groups to nurture their links with academia.

In order to consolidate the themes of transport, health and safety within Université Gustave Eiffel, the inspection committee's recommendations included several noteworthy points. The department's policy of strengthening its links with partner universities should be pursued and strengthened, in particular through the creation of a Joint Research Unit (UMR). Efforts made in connection with CIFRE-funded theses should be stepped up, both with industry and local authorities. Finally, more attention should be drawn to our outstanding facilities (testing facilities and databases), particularly on the websites.

The inspection committee took the view that a structure such as TS2 should be maintained within Université Gustave Eiffel, at least in spirit, in order to enable researchers to continue to nurture an effective synergy and structure. This is the key challenge of the Institute’s "Transport, Health, Safety and Risks" project which is currently being developed within the department. This aims both to ensure continuity and to extend its scope to include other laboratories and teams at Université Gustave Eiffel.

Seminar organised by the COSYS department during the visit of the HCERES inspection committee
From Challenges to Missions image: RTD © Mazzucato
Perméafor tests developed by the University of New Hampshire (UNH) with the future Université Gustave Eiffel for the New Hampshire Department of Transport (NHDOT) © Carlos Minatchy - Ifsttar
The HCERES committee’s visit to MAST

Evaluation of the establishment: consolidating academic ties

As part of the evaluation of the establishment, IFSTTAR performed the challenging task of presenting what it sees as its strengths but also ways of improving its organisation, without forgetting to map out initial avenues for development within the new university. On this basis, the report submitted by the evaluators makes several significant points. It supports a number of the strategic choices, in particular the importance of consolidating academic ties, for example in the form of UMRs, IFSTTAR's duty to contribute to the organisation of scientific life in France on themes related to the city and mobility, and preserving the Institute's international reputation.

The evaluators identified a number of points requiring attention and made several recommendations that will be valuable for the future Université Gustave Eiffel. We should ensure that we support the staff in the creation of this new university, bolster our international strategy, build a comprehensive policy concerning scientific facilities and maintain the “very good ratio between the number of administrative staff, engineers and technicians and the number of researchers”, which will be an asset for the future university. Some secondary proposals and remarks nevertheless allow us to state principles that seem essential to us. We should not modify our economic model by recruiting permanent staff on contracts. We should guarantee the quality of the opportunities for PhD graduates and the quality of the research methods to be adopted in order to meet our partners' needs. Lastly, we should continue to see public policy support activities as being compatible with high-quality research, helping to guide them in the right direction, exploiting them and maintaining the links with their beneficiaries.