Fotis Mystakopoulos

This paper refers to an evaluation project conducted at the Alexander Technological Educational Institute of Thessaloniki during 2014. The project aimed to evaluate the information literacy programme Orion. Orion is a useful tool that constitutes a most vital part of the information literacy skills evolution. The project aimed to evaluate the usability of Orion, specifying whether or not the programme improves information literacy skills of students providing content and functionality excellence. Additionally, it is very important to measure the usability of the structure that the programme follows. The method applied was measuring effectiveness, efficiency, learnability, and satisfaction. Most users appeared to be positively inclined towards the service, but during task performance and as they became more acquainted with the system, they became more critical as they confronted difficulties; such as the terminology used in it.

Panel titleSSH services beyond SSH: how LUMEN’s outcomes can improve cross-disciplinary and cross-domain data findability and (re)use Panel’s description: Social Sciences and Humanities (SSH) produce diverse research outputs, yet these often remain fragmented across local, language-specific, and discipline-bound repositories. While the GoTriple platform is an European hub for SSH multilingual scientific outputs, moving beyond domain boundaries remains a challenge ensuring SSH data is truly FAIR for the global scientific community. The EU-funded project LUMEN (https://lumenproject.eu/) aims to advance cross-domain collaboration and discovery processes, enhancing the Findability and Interoperability of SSH data. The goal is to connect diverse scientific fields (SSH, Mathematics, Earth System Sciences, and Molecular Dynamics) through a Federation of platforms (that includes GoTriple) and enable researchers to collaborate and access data from multiple disciplines. LUMEN adopts a Data Mesh approach in which each community retains ownership of its data and publishes governed Data Products under explicit Data Contracts, ensuring transparent governance and interoperability across domains. Key challenges reside in governance across autonomous communities, semantic alignment between vocabularies that were never designed to interoperate, and the structural asymmetries in how different domains document and share their research outputs. To address them, LUMEN develops a set of core components. The Meta-Search provides cross-domain discovery across all federated platforms. LUMIS offers an integrated environment for building, testing and reusing semantic artefacts that enable alignment across domains. GoTriple's unique value lies not in SSH content alone but in its multilingual processing pipeline, semantic enrichment capabilities, and an architecture that already supports domain-agnostic deployment. These are the assets that a White Label solution, also produced in LUMEN, will make available to other domains. It will allow any scientific community to adapt GoTriple’s core technologies, fostering an interdisciplinary ecosystem where SSH insights are seamlessly accessible to other domains. Advanced tools such as AIDA, an AI-Driven Discovery Assistant, will transform research discovery from simple keyword searches into natural language conversations. The panel will dive into questions such as: How can these outcomes improve SSH data findability? Why is SSH data valuable for other domains such as Earth System Sciences? How an SSH-born service, GoTriple, can be the basis of platforms for other domains? Panel participants: The Technical perspective: Julien Homo (Foxcub) will present the LUMEN’s federated discovery architecture, including its core building blocks, interoperability approach, and mechanisms for surfacing results from heterogeneous platforms. He will also introduce LUMIS as the environment supporting the lifecycle of semantic artefacts, and how this contributes to cross-domain enrichment and alignment of metadata. Luca de Santis (Net7) will explain the White Label solution’s development and how reusable components allow SSH-born services to adapt across domains while preserving specific governance and semantics. He will mention the integration of advanced AI tools within the White Label framework. The SSH Perspective: Fotis Mystakopoulos (OPERAS) will bring the SSH domain perspective and reflect on how SSH both shapes and is shaped by LUMEN's broader technical developments. The Cross-Domain Perspective Christelle Pierkot (CNRS/DataTerra) will highlight how LUMEN services will enable the ESS community to use resources from the SSH community to improve the outcomes of their research. Moderation: Carol Delmazo You can also watch the discussion as it happened via the Youtube Livestream (recording)

Abstract This document presents guidance and recommendations for responsible research assessment in the Social Sciences and Humanities (SSH), drawing on lessons learned from the SSH pilot conducted within the GraspOS Horizon Europe project (2023-2025). Led by OPERAS Research Infrastructure, the pilot engaged diverse stakeholders through a Community of Practice and validation workshops to develop Open Science-aware assessment approaches responsive to SSH particularities. The document is structured in two parts. Part 1 offers four key guidelines for shaping effective assessment in SSH: involving the community in assessment design; considering values and context before selecting metrics; valuing Open Science practices and diverse outputs; and prioritising qualitative assessment, particularly peer review and narrative CVs. Part 2 provides specific recommendations aligned with the SCOPE Framework for research evaluation, tailoring each step, Start with values, Context considerations, Options for evaluating, Probe deeply, and Evaluate your evaluation, to the SSH domain. The guidance addresses the distinct challenges SSH faces in research assessment, including methodological diversity, the prevalence of books and multilingual publications, and the limitations of traditional bibliometric indicators. It emphasises value-led rather than data-driven assessment, the recognition of diverse scholarly outputs and activities, and the importance of embedding Open Science principles into evaluation systems. The document also identifies key risks such as gaming behaviours and structural biases, proposing mitigation strategies to ensure fair and inclusive assessment practices.