2024, Articolo in rivista, ENG
Potortì F.; Crivello A.; et al.
Indoor positioning is a thriving research area which is slowly gaining market momentum. Its applications are mostly customised, ad hoc installations; ubiquitous applications analogous to GNSS for outdoors are not available because of the lack of generic platforms, widely accepted standards and interoperability protocols. In this context, the Indoor Positioning and Indoor Navigation (IPIN) competition is the only long-term, technically sound initiative to monitor the state of the art of real systems by measuring their performance in a realistic environment. Most competing systems are pedestrian-oriented and based on the use of smartphones, but several competing Tracks were set up, enabling comparison of an array of technologies. The two IPIN competitions described here include only off-site Tracks. In contrast with on-site Tracks where competitors bring their systems on site -- which were impossible to organise during 2021 and 2022 -- in off-site Tracks competitors download pre-recorded data from multiple sensors and process them using the EvaalAPI, a real-time, web-based emulation interface. As usual with IPIN competitions, Tracks were compliant with the EvAAL framework, ensuring consistency of the measurement procedure and reliability of results. The main contribution of this work is to show a compilation of possible indoor positioning scenarios and different indoor positioning solutions to the same problem.
2023, Rapporto di progetto (Project report), ENG
Accorinti M., Cerbara L., Ciancimino G., Crescimbene C., Sperandio L.
In accordance with the European Commission Communication No. 173 of April 4th, 2011, the National Strategy for the Inclusion of Roma, Sinti and Travellers (RST) 2012-2020 was developed, which is referred to in this report. The objective of the Strategy was to guide concrete inclusion activities for Roma, Sinti and Travellers, overcoming the emergency phase that had characterized government action in the previous years, especially in large metropolitan areas of the country. The Minister for International Cooperation and Integration was entrusted with the responsibility of constructing, in collaboration with the Ministers of Labor and Social Policies, Interior, Health, Education, University and Research and Justice, a Steering Committee. This committee involved representatives from regional and local entities, including mayors of major urban areas, as well as representatives of the Roma, Sinti and Traveller communities present in Italy. The Steering Committee's efforts were supported by the UNAR - National Office Against Racial Discrimination, serving as the national focal point. UNAR, activated in 2003 (by Legislative Decree No. 215/03) in line with EU Directive No. 2000/43/EC aimed at countering all forms of discrimination, is thus a reference point for the Strategy itself and for initiatives supporting RST communities. It ensures effective coordination among diverse Stakeholders in terms of roles, functions and competences and serves as a privileged observatory for both discriminatory phenomena and positive integration practices. From a methodological standpoint, the Strategy identified four intervention axes that led to the establishment of four working groups and respective project areas: housing, health, education and employment. Alongside these axes, UNAR carried out additional cross-cutting actions over time to counter discrimination and encourage RST community participation in the Strategy's implementation. This in-depth analysis of the project known as "Evalu-Action - Evaluation Activities of the National Strategy for RST 2012-2020" represents a synthesis of what has been implemented following the territorial/national intervention promoted by the 2012-2020 national strategy. Therefore, this report presents the overview of the presence of Roma, Sinti and Traveller communities (hereinafter referred to as "RST communities") based on various sources of information. The report is structured around available information sources and potential insights related to the axes of the 2012-2020 Strategy. It also includes some comprehensive analyses. Starting from these sources, the evolution of the presence of RST communities in Italy (an element that, as will be repeatedly mentioned, is of extremely challenging analysis) and their living conditions are described. Defining the presence of RST communities and the characteristics of the individuals comprising them is indeed part of a research effort aimed at gaining an in-depth understanding of the living situations of Roma and Sinti groups across the national territory. By understanding housing conditions, territorial contexts, work and education profiles, relationships with local institutions and origins, this research provides a framework to identify both critical issues and priorities for resource allocation and intervention. The research, however, becomes more rigorous when it encompasses a variety of sources, including direct engagement with RST communities, observation of living environments, analysis of local policies, involvement of thirdsector entities and more.
2022, Contributo in atti di convegno, ENG
Roberto Zamparelli, Shammur A Chowdhury, Dominique Brunato, Cristiano Chesi, Felice Dell'Orletta, Arid Hasan, Giulia Venturi
We report the results of the SemEval 2022 Task 3, PreTENS, on evaluation the acceptability of simple sentences containing constructions whose two arguments are presupposed to be or not to be in an ordered taxonomic relation. The task featured two sub-tasks articulated as: (i) binary prediction task and (ii) regression task, predicting the acceptability in a continuous scale. The sentences were artificially generated in three languages (English, Italian and French). 21 systems, with 8 system papers were submitted for the task, all based on various types of fine-tuned transformer systems, often with ensemble methods and various data augmentation techniques. The best systems reached an F1-macro score of 94.49 (sub-task1) and a Spearman correlation coefficient of 0.80 (sub-task2), with interesting variations in specific constructions and/or languages.
2022, Contributo in atti di convegno, ENG
Faggioli G.; Ferrante M.; Ferro N.; Perego R.; Tonellotto N.
The rapid growth in the number and complexity of conversational agents has highlighted the need for suitable evaluation tools to describe their performance. The main evaluation paradigms move from analyzing conversations where the user explores information needs following a scripted dialogue with the agent. We argue that this is not a realistic setting: different users ask different questions (and in a diverse order), obtaining distinct answers and changing the conversation path. We analyze what happens to conversational systems performance when we change the order of the utterances in a scripted conversation while respecting temporal dependencies between them. Our results highlight that the performance of the system widely varies. Our experiments show that diverse orders of utterances determine completely different rankings of systems by performance. The current way of evaluating conversational systems is thus biased. Motivated by these observations, we propose a new evaluation approach based on dependency-aware utterance permutations to increase the power of our evaluation tools.
2021, Contributo in atti di convegno, ENG
De Mattei L.; Lai H.; Dell'Orletta F.; Nissim M.
We take a collection of short texts, some of which are human-written, while others are automatically generated, and ask subjects, who are unaware of the texts' source, whether they perceive them as human-produced. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that the production of this fine-tuned model is indeed perceived as more human-like than that of the original model. Contextually, we show that our automatic evaluation strategy correlates well with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language.
2021, Contributo in atti di convegno, ENG
Torres-Sospedra J.; Silva I.; Klus L.; Quezada-Gaibor D.; Crivello A.; Barsocchi P.; Pendao C.; Lohan E.S.; Nurmi J.; Moreira A.
The evaluation of Indoor Positioning Systems (IPSs) mostly relies on local deployments in the researchers' or partners' facilities. The complexity of preparing comprehensive experiments, collecting data, and considering multiple scenarios usually limits the evaluation area and, therefore, the assessment of the proposed systems. The requirements and features of controlled experiments cannot be generalized since the use of the same sensors or anchors density cannot be guaranteed. The dawn of datasets is pushing IPS evaluation to a similar level as machine-learning models, where new proposals are evaluated over many heterogeneous datasets. This paper proposes a way to evaluate IPSs in multiple scenarios, that is validated with three use cases. The results prove that the proposed aggregation of the evaluation metric values is a useful tool for high-level comparison of IPSs.
2021, Articolo in rivista, ENG
Brambilla, Cristina; Pirovano, Ileana; Mira, Robert Mihai; Rizzo, Giovanna; Scano, Alessandro; Mastropietro, Alfonso
Electroencephalography (EEG) and electromyography (EMG) are widespread and well-known quantitative techniques used for gathering biological signals at cortical and muscular levels, respectively. Indeed, they provide relevant insights for increasing knowledge in different domains, such as physical and cognitive, and research fields, including neuromotor rehabilitation. So far, EEG and EMG techniques have been independently exploited to guide or assess the outcome of the rehabilitation, preferring one technique over the other according to the aim of the investigation. More recently, the combination of EEG and EMG started to be considered as a potential breakthrough approach to improve rehabilitation effectiveness. However, since it is a relatively recent research field, we observed that no comprehensive reviews available nor standard procedures and setups for simultaneous acquisitions and processing have been identified. Consequently, this paper presents a systematic review of EEG and EMG applications specifically aimed at evaluating and assessing neuromotor performance, focusing on cortico-muscular interactions in the rehabilitation field. A total of 213 articles were identified from scientific databases, and, following rigorous scrutiny, 55 were analyzed in detail in this review. Most of the applications are focused on the study of stroke patients, and the rehabilitation target is usually on the upper or lower limbs. Regarding the methodological approaches used to acquire and process data, our results show that a simultaneous EEG and EMG acquisition is quite common in the field, but it is mostly performed with EMG as a support technique for more specific EEG approaches. Non-specific processing methods such as EEG-EMG coherence are used to provide combined EEG/EMG signal analysis, but rarely both signals are analyzed using state-of-the-art techniques that are gold-standard in each of the two domains. Future directions may be oriented toward multi-domain approaches able to exploit the full potential of combined EEG and EMG, for example targeting a wider range of pathologies and implementing more structured clinical trials to confirm the results of the current pilot studies.
2021, Contributo in volume, ENG
Pocobello, Raffaella
The chapter introduces a research project that assessed the transferability of the Open Dialogue approach in the context of Italian mental health departments
2021, Rapporto di progetto (Project report), ENG
Dodis M.; Ledwon Z.; Tarkowski P.; Loven T.; Carlini E.; Zadtootaghaj S.; Dazzi P.
This deliverable provides a first release of the report on the plans to implement and evaluate the scenarios addressed for each project Use Case. The deliverable comprises and reports on tree main subtasks. The first subtask relates to a detailed description of the Use Cases and the scenarios under which evaluation which revolve. The second subtask relates to the description of the pilot prototypes, the evaluation methodology, the experimentation requirements, the modules/functionalities that will be assessed and the metrics for assessing the value of ACCORDION in terms of technology and subjective Quality of Experience. The third subtask specifies the integration plan for the components and technologies of ACCORDION, the design of the infrastructure along with the testbed combinations for pilot execution and evaluation and the execution methodology. Regarding ethical and privacy issues, all necessary measures have been considered as part of conducting the subjective Quality of experience evaluation and are reported.
2021, Articolo in rivista, ENG
Potortì F.; Torres-Sospedra J.; Quezada-Gaibor D.; Jiménez A.R.; Seco F.; Pérez-Navarro A.; Ortiz M.; Zhu N.; Renaudin V.; Ichikari R.; Shimomura R.; Ohta N.; Nagae S.; Kurata T.; Wei D.; Ji X.; Zhang W.; Kram S.; Stahlke M.; Mutschler C.; Crivello A.; Barsocchi P.; Girolami M.; Palumbo F.; Chen R.; Wu Y.; Li W.; Yu Y.; Xu S.; Huang L.; Liu T.; Kuang J.; Niu X.; Yoshida T.; Nagata Y.; Fukushima Y.; Fukatani N.; Hayashida N.; Asai Y.; Urano K.; Ge W.; Lee N.T.; Fang S.H.; Jie Y.C.; Young S.R.; Chien Y.R.; Yua C.C.; Ma C.; Wub B.; Zhangc W.; Wang Y.; Fan Y.; Poslad S.; Selviah D.R.; Wangd W.; Yuan H.; Yonamoto Y.; Yamaguchi M.; Kaichi T.; Zhou B.; Liue X.; Gu Z.; Yang C.; Wu Z.; Xie D.; Huang C.; Zheng L.; Peng A.; Jin G.; Wangh Q.; Luo H.; Xiong H.; Bao L.; Zhangi P.; Zhao F.; Yuj C.A.; Hung C.H.; Antsfeld L.; Chidlovskii B.; Jiang H.; Xia M.; Yan D.; Li Y.; Dong Y.; Silva I.; Pendão C.; Meneses F.; Nicolau M.J.; Costa A.; Moreira A.; De Cock C.; Plets D.; Opiela M.; Dzama J.; Zhang L.; Li H.; Chen B.; Liu Y.; Yean S.; Lim B.Z.; Teo W.J.; Leep B.S.; Oh H.L.
Every year, for ten years now, the IPIN competition has aimed at evaluating real-world indoor localisation systems by testing them in a realistic environment, with realistic movement, using the EvAAL framework. The competition provided a unique overview of the state-of-the-art of systems, technologies, and methods for indoor positioning and navigation purposes. Through fair comparison of the performance achieved by each system, the competition was able to identify the most promising approaches and to pinpoint the most critical working conditions. In 2020, the competition included 5 diverse off-site off-site Tracks, each resembling real use cases and challenges for indoor positioning. The results in terms of participation and accuracy of the proposed systems have been encouraging. The best performing competitors obtained a third quartile of error of 1m for the Smartphone Track and 0.5m for the Foot-mounted IMU Track. While not running on physical systems, but only as algorithms, these results represent impressive achievements.
2021, Articolo in rivista, ENG
Coccia Mario
This paper analyzes first and second wave of COVID-19 pandemic in one of largest European countries, Italy, to show how the first wave of COVID-19 pandemic had a high negative effect on public health that reduced intensity with the summer season and with containment policies; second wave of the COVID-19 pandemic, from August 2020 onwards, showed increasing confirmed cases but general impact in society seems to be of a lower intensity in society. This study can support best practice of crisis management to cope with future recurring waves of COVID-19 pandemic and similar epidemics.
2020, Contributo in atti di convegno, ENG
De Mattei L., Cafagna M., Dell'Orletta F., Nissim M.
We automatically generate headlines that are expected to comply with the specific styles of two different Italian newspapers. Through a data alignment strategy and different training/testing settings, we aim at decoupling content from style and preserve the latter in generation. In order to evaluate the generated headlines' quality in terms of their specific newspaper-compliance, we devise a fine-grained evaluation strategy based on automatic classification. We observe that our models do indeed learn newspaper-specific style. Importantly, we also observe that humans aren't reliable judges for this task, since although familiar with the newspapers, they are notable to discern their specific styles even in the original human-written headlines. The utility of automatic evaluation goes therefore beyond saving the costs and hurdles of manual annotation, and deserves particular care in its design.
2020, Rapporto di ricerca (Research report), ENG
Sorin Hermon, Laura Benassi, Athanasios Koutoupas, Elisabetta Andreassi
The deliverable "Report on the implementation and evaluation of communication" analyses communication and dissemination activities carried out in the E-RIHS PP project and underlines criticalities and possible future implementation. The analytics tools for the website and social media and the feedback reports for events provide, as a result, an overview of the impact of the communication and dissemination during the E-RIHS PP project.
2020, Articolo in rivista, ENG
Muntean C.I.; Nardini F.M.; Perego R.; Tonellotto N.; Frieder O.
We observe that in curated documents the distribution of the occurrences of salient terms, e.g., terms with a high Inverse Document Frequency, is not uniform, and such terms are primarily concentrated towards the beginning and the end of the document. Exploiting this observation, we propose a novel version of the classical BM25 weighting model, called BM25 Passage (BM25P), which scores query results by computing a linear combination of term statistics in the different portions of the document. We study a multiplicity of partitioning schemes of document content into passages and compute the collection-dependent weights associated with them on the basis of the distribution of occurrences of salient terms in documents. Moreover, we tune BM25P hyperparameters and investigate their impact on ad hoc document retrieval through fully reproducible experiments conducted using four publicly available datasets. Our findings demonstrate that our BM25P weighting model markedly and consistently outperforms BM25 in terms of effectiveness by up to 17.44% in NDCG@5 and 85% in NDCG@1, and up to 21% in MRR.
2020, Articolo in rivista, ENG
Lucchese C.; Muntean C.I.; Nardini F. M.; Perego R.; Trani S.
RankEval is a Python open-source tool for the analysis and evaluation of ranking models based on ensembles of decision trees. Learning-to-Rank (LtR) approaches that generate tree-ensembles are considered the most effective solution for difficult ranking tasks and several impactful LtR libraries have been developed aimed at improving ranking quality and training efficiency. However, these libraries are not very helpful in terms of hyper-parameters tuning and in-depth analysis of the learned models, and even the implementation of most popular Information Retrieval (IR) metrics differ among them, thus making difficult to compare different models. RankEval overcomes these limitations by providing a unified environment where to perform an easy, comprehensive inspection and assessment of ranking models trained using different machine learning libraries. The tool focuses on ensuring efficiency, flexibility and extensibility and is fully interoperable with most popular LtR libraries.
2020, Articolo in rivista, ENG
Agostinetti P.; Franke T.; Fantz U.; Hopf C.; Mantel N.; Tran M.Q.
DEMO is a first-of-a-kind DEMOnstration fusion power plant [1,2] and is intended to follow the ITER experimental reactor. The main goal of DEMO will be to demonstrate the possibility to produce electric energy for the grid from the fusion reaction early in the second half of the century. The injection of high energy neutral (1 MeV) particle beams is one of the main tools to heat the plasma up to fusion conditions, control the plasma burn phase and ramp the plasma down. Within the EUROfusion Framework a conceptual design of the Neutral Beam Injector (NBI) for the DEMO fusion reactor is currently being developed. Thereby, Reliability, Availability, Maintainability and Inspectability (RAMI) have to be taken into consideration for the conceptual design of the DEMO NBI, together with the exploitation of the currently available return of experience from the ITER NBIs. Comparing the failure risk of two different source concepts due to the considered failure modes has allowed for further design developments aiming at exploiting the advantages of the modular approach while minimizing its drawbacks.
2020, Altro prodotto, ENM
Vito Di Maio
Evaluation as referee of a research projected submitted by the Soutern California State University to the ARO (Army Research Office) of the United States
2019, Articolo in rivista, ENG
Stefanini Alberto Eugenio, Nicolosi Anika , Monachini Monica
Ancient Greek poetry is an essential part of the western cultural heritage; thus, it is important that people have access to its texts and whatever relates to their understanding in a reliable and easy way. Whenever user evaluation is concerned, mock-ups are used by designers to acquire feedback from users. A mock-up is defined as a model of the final product, and may be used for demonstration, evaluation and other purposes. The authors prototyped a mock-up for focusing on the requirements of a scholarly digital edition of Archilochus. This was put under evaluation to assess its usability: it was submitted to extensive use and testing by a sample of prospective users, to better focus on the requirements from a product's perspective. Experimentation involved a group of university students, attending a Greek Philology course at Parma University. More than half of the respondents considered the mock-up a useful study support. The evaluation also pointed out that the mock-up had to be revised, so as to guarantee better cognitive simplicity of the user interface.
2019, Articolo in rivista, ENG
Emanuela Reale, Antonio Zinilli
The paper investigates the characteristics of researchers with interdisciplinary research ori-entation with respect to those more disciplinary oriented. The research question is: how aca-demic researchers dealing with interdisciplinary research perceive the quality of the projects developed, the stability of the collaborations of the research teams, and the importance of in-terdisciplinarity in the assessment of the projects submitted for funding, comparing with those dealing with disciplinary research? Because motivations and reputation of academics are not based on interdisciplinarity, we expect that researchers do not perceive any specific ad-vantage/disadvantage of interdisciplinarity.
2019, Contributo in atti di convegno, ENG
I. Gagliardi and M.T. Artese
Massive volumes of images of museums or art collections, or made available by artists and photographers, more and more often, are available on the web, along with some metadata, essential for their characterization and retrieval. A set of (scored) keywords/keyphrases that characterize the semantic content of the documents should be, automatically or manually, extracted and/or associated. We present here a work-in-progress to evaluate different methods for the unsupervised keyword extraction to Italian and English datasets. In the paper datasets, algorithms and approaches are presented and discussed together with some preliminary results referred to relatedness of terms.