HINTS: Human-Centered Intelligent Realities

—During the last decade, we have witnessed a rapid development of extended reality (XR) technologies such as augmented reality (AR) and virtual reality (VR). Further, there have been tremendous advancements in artificial intelligence (AI) and machine learning (ML). These two trends will have a significant impact on future digital societies. The vision of an immersive, ubiquitous, and intelligent virtual space opens up new opportunities for creating an enhanced digital world in which the users are at the center of the development process, so-called intelligent realities (IRs). The “Human-Centered Intelligent Realities” (HINTS) profile project will develop concepts, principles, methods, algorithms, and tools for human-centered IRs, thus leading the way for future immersive, user-aware, and intelligent interactive digital environments. The HINTS project is centered around an ecosystem combining XR and communication paradigms to form novel intelligent digital systems. HINTS will provide users with new ways to understand, collaborate with, and control digital systems. These novel ways will be based on visual and data-driven platforms which enable tangible, immersive cognitive interactions within real and virtual realities. Thus, exploiting digital systems in a more efficient, effective, engaging, and resource-aware condition. Moreover, the systems will be equipped with cognitive features based on AI and ML, which allow users to engage with digital realities and data in novel forms. This paper describes the HINTS profile project and its initial results.


I. INTRODUCTION
Nowadays, digitalization is an inseparable part of everyone's lives, e.g., how we work, interact, and spend our free time. During the last decade, we have seen that technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), all encompassed in extended reality (XR), have emerged and become available to a broader audience. There have also been numerous discussions on future immersive realities. This development has and will also be propelled by the increase in home work due to the COVID-19 pandemic and sustainability challenges. Consequently, in 2020 the XR market was valued at USD 25.84 billion and is expected to reach USD 397.81 billion by 2026 [1]. Further, there has been tremendous development in artificial intelligence (AI) and machine learning (ML), fueled by vast amounts of data, powerful hardware, and algorithm advances. Both these trends will have a significant impact on future digital societies.
Since 2021, there has been increased focus and discussions on the metaverse, the vision of an immersive and ubiquitous virtual space. It has been proposed to open new opportunities for creating an enhanced human-centered digital world. In such environments, users could experience and interact in a persistent alternative reality that can be modified and customized according to their preferences.
In the interplay of human-computer interaction (HCI), using novel immersive interaction and visualization technologies, a current addition is the dimension of AI. Based on recent developments, it is expected that the future of immersive environments cannot be envisioned without AI. In the last couple of years, new conferences have appeared, such as the IEEE International Conference on Intelligent Reality (ICIR) [2], first organized in 2021, and the Visualization Meets AI [3] workshop in 2020.
AI can enrich the user experience (UX) by making smarter and personalized human-centered choices in future intelligent realities (IRs). However, at the core is still the choice-making human. Therefore, novel interactions and visual analytics techniques in such settings have the potential to influence and aid the entire decision-making process hugely. Furthermore, this process is largely becoming bidirectional, wherein the system learns and predicts based on the user's interests and where the user seeks to guide and play a more active role in personalizing novel immersive environments for interaction.
Digitalization and demands for a more sustainable society will change how we live, develop new products and services, and do business. Enablers have been, e.g., software-intensive systems, mobile communication, and powerful computers. New drivers and enablers have emerged when taking the steps into the next generations of digital societies. Some current trends that will shape future digital societies are: • users will be able to switch between IRs, expecting seamless and high-quality experiences, • users expect to interact in visually intuitive ways using XR techniques and new realities, • data are produced at an increasing pace by vast numbers of heterogeneous sensors and devices, • there is a tremendous development in AI and ML, and • information processing is virtualized and seamlessly transmitted across cloud, fog, and edge services. The remainder of the paper is organized as follows. In Section II, we introduce the background of the HINTS project in light of recent similar projects in Sweden and internationally. Next, Section III discusses our project's objectives and research focus and the novelty it brings to the state-of-the-art. The theme structure of HINTS is presented in Section IV followed by the initial results obtained in Section V. Finally, we conclude this paper with a summarised discussion (Section VI) and conclusions (Section VII).

II. BACKGROUND AND RELATED WORK
Computer graphics (CG), visualization, and interactive media have been used for decades in, e.g., computer games and the visualization of complex data. However, over the last decade, we have seen tremendous development in VR, AR, and MR interaction techniques within the umbrella term XR. These technologies initially became available to a broad public audience through commercial alternatives such as Microsoft Hololens and Oculus Rift. However, there are now more recent headsets available that, for example, include technologies such as eye tracker foveated rendering (ETFR), such as Meta Quest Pro.
Further, gestures, gaze, and eye tracking provide additional interaction capabilities. Another significant development in the last decade is the advancement and use of AI and ML, which have impacted and revolutionized many areas, including recommender systems, language and text analysis, translation, computer vision, and autonomous systems. Large amounts of data, powerful hardware, and algorithm development enable this progress. With the explosion of visual content and distributed visual sensor networks, developing novel systems for interaction, visual analysis, scalable communication and computation, and secure and integritypreserving data handling is required more than ever.
The advancement of CG techniques has substantially improved XR instruments. Those can further be revolutionized by embedding AI to enable XR to communicate and interact effectively with users [4]. This will lead to the development of novel IR systems that can understand and adapt to human behavior and effectively navigate complex user-system interactions. Including intelligence in XR can be helpful for different applications, such as cancer detection [5], gaming [6], advanced visualization [7], driver training [8], and medical training [9]. In addition, such IRs will, in return, support building reliable and transparent AI systems by serving as an extended learning environment for AI ensuring data diversity and representativeness. Training AI using real-world data can be difficult since complicated aspects of human behavior, physical phenomena, and robot dynamics can be challenging to precisely capture in the real world and often require considerable infrastructure costs and manpower [10], [11], [4]. Hence, IR systems can be applied to simulate various cost-intensive, challenging, and potentially dangerous scenarios to enrich real-world data and avoid and mitigate biases.
The Human-Centered Intelligent Realities (HINTS) profile project, presented in this paper, is in line with the recent trends in European research advances integrating XR and AI to design intelligent solutions that benefit humans. For example, in the framework of the recently finished Iv4XR H2020 project, a novel verification and validation technology for XR systems based on techniques from AI is developed to provide learning and reasoning over a virtual world. Another H2020 project, EXPERIENCE, has combined VR and AI to explore novel diagnoses and treatment of affective disorders commonly associated with altered multi-sensory perception like depression, anxiety, and eating disorders. The EO4EU is also a European Commission-funded innovation project that aims to make Earth observation data accessible to users through next-generation tools. The Horizon Europe TRANSMIXR project relies on the maturity of XR and AI with the goal of creating a range of human-centric tools for remote content production and consumption via social VR. The SUN is also a social and human-centered project funded by the Horizon Europe programme that aims at investigating and developing XR solutions that integrate the physical and the virtual world in a convincing way from a human and social perspective. The XR2Learn is a Horizon Europe project aiming to combine human-machine interactions with real, mixed, augmented, and virtual environments for the creation of human-centric XR applications in education.
On a Swedish national level, the University of Skövde has a research profile named VF-KDO (Virtual Factory with Knowledge-Driven Optimization) focusing on building and optimizing digital models for the future's production facilities. VF-DKO also uses advanced data mining and interactive visual analytics to extract decision-support knowledge. The Wallenberg AI, Autonomous Systems and Software Program (WASP) is also the largest research initiative in Sweden with a focus relevant to the HINTS profile project.

III. THE HUMAN-CENTERED INTELLIGENT REALITIES PROFILE PROJECT
The HINTS profile builds upon existing experiences and competencies in the areas above and combines these in a synergetic manner to develop novel human-centered intelligent realities. The research profile, HINTS, is hosted by the Department of Computer Science (DIDA) at Blekinge Institute of Technology (BTH). HINTS is a six-year project, partly funded by the Knowledge Foundation, which started on the first of September 2022. The profile is led by an established team of researchers at the department with interdisciplinary expertise representing the human-centered IRs perspective. The work is carried out in collaboration with members from initially six external industrial partners: Blackdrop Interactive, Ericsson, IKEA Marketing & Communication, NODA Intelligent Systems, Spotify, and Virotea. The HINTS profile contributes to the strategic direction of BTH and the recruitment of key new personnel in the department.
The project seeks scientific breakthroughs in five interrelated strategic research areas of human-centered IRs: novel experience assessment methodologies, novel environments and interaction techniques, visual analytics, adaptive and distributed AI, and networking. The needs of our industrial partners are grouped into seven industrial challenges. Based on these challenges, five research themes, see Figure 1, are defined with their core research questions, and together, they will address the overall aim of HINTS, i.e., developing human-centered IRs.

A. Objective of Research Profile
The unique potential of the HINTS research profile is the combined and necessary competence in experience assessment, visual and interactive computing, visualization and visual analytics, ML and data analytics, and cloud-to-edge computing. Furthermore, being the core part of the strategy at BTH, i.e. digitalization and sustainability, the vision and commitment of HINTS are to establish an internationally recognized competence center focusing on human-centered IRs. A step towards that vision is to establish a competence center at the national level.
The research profile builds upon previous successful projects at BTH. The ViaTecH Synergy project, funded by the Knowledge Foundation (2017-2022), is the main stepping stone for the profile application, where we continue the developments from the last four years to solve challenges for future digital societies.
The proposed profile will also build on the competence developed in the research profile BigData@BTH, funded by the Knowledge Foundation (2014-2020), from where we will utilize the competence in data analytics and ML. Finally, we will build upon the competence developed within the EU projects Bonseyes and BonsApps, which both aim at the more rapid engineering and deeper integration of AI into edge or user equipment, as also foreseen by the HINTS project. Examples of expected results and outcomes include: • Development of research area excellence, industrial impact, and business value. • Establishment of a nationally leading and internationally recognized research environment. • Competence development at BTH within the profile area, i.e., human-centered IRs. • Development of new and current educational program improvement that reflects the knowledge and competence gained from the profile project. • Establishment of a stronger and more intimate coproduction with existing and new industry partners to ensure successful conditions for future cooperation and development. • Development of methods and design principles for industrial applications and use cases, validated through demonstrators with industry partners, e.g., tools, support systems, prototypes, proof-of-concept applications, and other artifacts.

B. Overall Objective and Research Question
As outlined in the Background and Related Work in Section II, there are several challenges to address to design and develop novel human-centered IRs for future digital societies. Hence, the overall objective of this research profile is summarized as follows: • In co-production with industrial partners and society, develop concepts, principles, methods, algorithms, and tools for human-centered IRs to lead the way for future immersive, user-aware, and smart interactive digital environments.
In order to pursue the proposed objective, HINTS will investigate the following main research question: • How shall we design effective, efficient, and distributed analytical and computational methods for humancentered IRs? The main research question is broken down into subquestions for each research theme. It is important to note that collaboration and interaction between the themes are necessary to address the research questions.

C. Profile Connection and Benefit to Education
The higher education at BTH will benefit strategically from the HINTS profile in several ways. Recently, BTH decided to centralize game-related education at Campus Karlskrona while opening up a new educational media technology track on digital and immersive experiences hosted by the Department of Technology and Aesthetics (DITE) at Campus Karlshamn. Furthermore, HINTS will be a crucial enabler when reshaping the Computer Science-hosted education programs in game technology. Hence, BTH has strategically decided to update and reshape three new game technology programs at BTH, where the theoretical and technical foundations of graphics programming will be a fundamental part.
The computer game industry has long been one of Sweden's fastest-growing industries. However, studies have shown that the supply of skills in the computer games industry needs to be improved and that skills are recruited abroad. It is vital that we can provide education to satisfy the needs of the industry and that both societal and technical research on computer games and interactive techniques should be strengthened [12]. The primary market for students who have completed the program is professional roles in the gaming industry, focusing on game technical implementation and/or design.
However, the increasing interests in visual and data-driven computing and human-centered IRs in several other industries also emphasize the need for broader knowledge that can be applied to digital activities outside the gaming industry. Examples are companies in media production, health, advertising, visualization, architecture, film, training & simulation, and digital vehicles.
As education at DITE focuses on sound, visual, and immersive experiences in both digital and physical environments, the immersive experiences designed by media technology students will provide a promising basis for two-way cross-campus cooperation with HINTS researchers, namely evaluating innovative digital and immersive experiences, and test innovations made within HINTS. Further, HINTS will support the development of advanced-level courses in our newly started (2019) 5-year engineering program "AI and machine learning".
The team connected to HINTS is also responsible for supervising many MSc thesis projects in computer science and most of the BTH 5-year engineering program degree projects, "Game and software engineering". Thus, HINTS will contribute significantly to the strategic development of education at BTH and strengthen the connection between education and research and between departments.

IV. RESEARCH THEMES
In this profile, we will primarily focus on the following aspects organized into five research themes:

A. Theme A: Novel Experience Assessment Methodologies for Intelligent Realities
Novel IR approaches capturing the entire virtual-reality continuum of networked virtual worlds are at the center of Theme A. As the end-user is the final judge of the quality of such applications, it is of utmost importance to understand the behavior and intent of humans in novel IRs as a solid foundation to successfully develop and implement IRs.
Theme A focuses on measuring, assessing, and modeling UX in IRs. These components will lead to an ecosystem of novel experience assessment methodologies ranging from controlled subjective tests to user studies in real environments. This ecosystem goes well beyond the traditional quality of experience (QoE) [13] research of the telecommunications community but also embraces human-centered methods of the UX [14] research from the human-computer interaction community to support the field of quality of user experience (QUX).
QUX, covering QoE and UX, resorts to the concepts of eudaimonia [15]. Eudaimonia aims to understand one's wellbeing not as an "outcome or end state" but as a "process of fulfilling one's virtuous potentials and living as one was inherently intended to live" [15]. It goes beyond the hedonic and pragmatic usage aspects of interactive applications as it targets the long-term perceived value and usefulness of applications beyond the short-term experience when using an application. Eudaimonia has also been discussed within the context of UX, e.g., [16]. However, to date, only the recent work [17] proposed a multidimensional construct to measure eudaimonia, complementing both QoE and UX. In Theme A, to facilitate the measuring, assessing, and modeling of QUX and its personalization in novel IRs as a formal framework to reveal humans' behavior and intent, we aim at solving the essential challenges associated with immersive computing taking a human-centric approach.
The work toward QUX will reveal an understanding of the user's intent and maneuvering in IRs, which is essential in developing novel human-centered systems that offer the user more interactive and personalized experiences. The development of technical solutions will be grounded on the humancentric approach of the QUX method to include mechanisms of the human visual system, locomotion and navigation, selfmotion perception, multimodal and cross-modal interaction and perception, quality perception, cybersickness, presence, task performance, and other human factors. The results from the QUX assessment of human-centric solutions will be used as input to machine learning models to investigate which aspects can be predictors of QUX. Such results will help understand human behavior when using immersive solutions and to identify objective perceptual quality assessment aspects that are relevant within the context of IRs.

B. Theme B: Novel Environments and Interaction Techniques for Intelligent Realities
Theme B aims to investigate the next-generation, novel interaction techniques in XR-based human-centered intelligent 3D virtual spaces or IRs [18]. In these new smart spaces, each user can create personalized virtual environments and interfaces and choose from various modes to interact with them. These spaces are continually enriched by advanced interaction technologies such as haptics, eyetracking, etc., [19] also allowing multiple users to interact with each other, thereby giving new opportunities for collaboration within the same virtual space (at similar or different locations). On the one hand, Theme B directly interacts with the visualization of these user spaces. On the other, it is itself under evaluation for improvement on the novelty of experiences. With the added dimension of AI, the system can acquire the capability to predict and personalize the visual content and interaction techniques, taking both the explicit and the tacit hints on user requirements and preferences. Theme B aims to construct the future's intelligent, responsive AI-guided 3D environments and develop the next-generation interaction techniques for single-/multi-users in these virtual environments. A key novelty in these futuristic virtual spaces would be able to adapt to the user's intentions on both these fronts while maintaining efficient performance. This humanmachine interaction would thus thrive on the bidirectional feedback between both components.

C. Theme C: Visual Analytics for Intelligent Realities
In the era of the Internet of Things (IoT), large-scale, dynamic, and heterogeneous datasets and the everlasting thrust for exploiting the wealth of information coming out of that, a high-level information abstraction from complex data has become a challenge to existing systems. Therefore, it is paramount to augment human capital using machine intelligence to assist decision-makers in helping them make sense of large quantities of data and use their time effectively. Visual analytics combines our visual intelligence and analysis techniques with visual technology to get relevant information from data [20]. It aims at managing a large amount of data from various sources. It requires a blend of computational analyses and visualizations that can facilitate the understanding and monitoring of complex processes (e.g., machine learning-based 3D immersive and interactive models, unpopulated aerial vehicle vision, and self-driving cars). Current visual analytics systems face multifaceted challenges in adapting to the advancements made by digital realities. In these immersive visual analytics, human-machine (computing intelligence) partnerships will continue to evolve, purposely helping augment cognition in users to answer complex questions. In a nutshell, Theme C revolves around devising efficient (e.g., user non-distractive and comfortable experience) and immersive mechanisms to present and communicate data of complex systems (e.g., analytics of largescale and multi-source visual data). Theme C's central focus is deep visual analytics which could be transferable to digital realities.

D. Theme D: Adaptive and Distributed AI for Intelligent Realities
Theme D focuses on studying novel resource-efficient adaptive and distributed AI/ML approaches to equip immersive systems with distributed perception and intelligence. Many applications of such immersive systems conferred with intelligence can be identified today, e.g., autonomous cars, sports activities/exercise, gaming applications, advanced visualization methods, smart homes, and many others [4]. These applications require new robust and adaptive AI models that can be run on smart interactive devices with limited power and storage [21], [22]. The models must employ efficient learning and evaluation algorithms capable of dealing with multi-source information varying over time and additionally understanding and predicting user intention and being able to adapt to it. Theme D's main interests are intelligent data-centered solutions focusing on interaction, explainability, and adaptivity, leveraging a wide range of techniques from distributed ML, frequent pattern mining, edge-based AI, continuous learning and domain adaptation, data integration, and analysis. For instance, multi-view data mining algorithms [23] can be used to analyze the heterogeneous data generated by immersive systems and detect user behavioral patterns. These patterns can facilitate the understanding and interpretation of user performance and can be further used to adapt and improve the user behavioral models and experience assessment metrics. Another ambition is to develop novel hybrid immersive analytical techniques that can support reasoning and decision-making in complex data exploration scenarios [24]. AI-assisted visual and advanced data analytics can be combined and used to provide multilayered analysis of potential problems and direct users to specific points that need attention.

E. Theme E: Networking for Intelligent Realities
IRs will be typically distributed systems. The storage of data and computation will be done at multiple locations to take advantage of remote accelerations hardware, GPUs, or protected data only available at places. IRs will exchange significant amounts of data, often with complex structures requiring highly efficient and resource-aware hardware. Hence, the data transport for IRs is expected to demand, in general, a) ultra-low delays (because of the immersiveness), and even for extensive data, b) no or extremely small delay variation, e.g., aiming at the predictability for starting computation, and c) a network with rapid adaptation towards the efficiency needs for IRs such as enabling the very high-bandwidth data flows to increase the efficiency of the computing resources even on very short notice. The latter task is challenged by the inevitable and potentially long end-to-end delay between two control and processing elements, e.g., between a client on a device and a server or acceleration hardware in the cloud. The delay might block network-application interactions that are across multiple layers. Fast network adaption or data flow management can be executed efficiently within the network layer by "in-network" mechanisms. SDN technologies, such as OpenFlow, P4, or eBPF, have recently demonstrated such capabilities and are now components of modern communication networks 5G and future 6G systems [25].
Each of the three general network performance objectives ((a) -(c)) for IRs is difficult to tackle. Hence, any combination of them is even more demanding. However, networking mechanisms that address them jointly will have the necessary capabilities for IRs. The pragmatic formulation of the objective is to provide the "right data at the right time and at the right place".
The pragmatic notion can be translated into three subparts of a networking system model in HINTS, as shown in Figure 2. The model is initial and continuously refined. The model's first subpart (Part 1) is the detailed specification of QUX and network performance requirements. This part aims at a calculation model to relate both performance views, i.e., the one from the user experience and the one from the network performance. A significant challenge is the use of new assessment concepts (cf. Theme A) and the relations of perceived immersiveness and QUX with network performance. The second subpart of the model (Part 2) is the minimization of the latency in accessing the network for 5G and 6G by applying sophisticated radio resource management. The third subpart of the model (Part 3) is the use of SDN technologies for adaptive, high-bandwidth, and ultra-low-latency data flows management and interconnection services in cloud and edge computing environments for IRs. For example, Part 3 on SDN-enabled management may use the relationship model from Part 1 to control its in-network mechanisms, i.e., application AI, to manage the network.

Networking for IRs
In the future, the HINTS networking model needs to address more objectives for enabling sustained efficiency for IRs. These objectives comprise factors such as the reduction of energy consumption in data transmission as well as in computation, e.g., by enabling data forwarding to more energy-efficient acceleration hardware, which is eventually located in the Cloud-Edge computing continuum. Another future objective is to increase the security of IRs by the expanding use of distributed algorithms, such as Blockchains for secure data exchange in federated learning (FL) [26].

V. INITIAL RESULTS
Apart from the initial works being broken down in more detail, the major developments so far are highlighted below. A new laboratory has been set up to promote collaboration in a complete environment merging strong education programs, research excellence, and industrial and societal collaboration. The laboratory has already started to be used in courses relevant to the area at BTH, which are further exemplified below. A HINTS web page has also been established as part of the project startup phase.

A. Human-Centered Intelligent Realities Laboratory
The Human-Centered Intelligent Realities Laboratory (HCIRL) is a centrally funded strategic infrastructure initiative at BTH aimed at creating a strong research and education environment. This is part of a more considerable BTH strategic investment in infrastructures, where university-wide laboratory facilities are essential. The goal of the laboratory is to have a shared, flexible, creative space for research, education, and activities with industry and society. Expected synergies via the laboratory are foreseen with new project developments. Example activities planned in the laboratory are as follows: • joint educational and research environment, • internal and external collaborations, • basic and advanced level undergraduate and postgraduate courses, • joint training workshops with industrial partners, • regional test-bed initiative, • targeted course moments in education, • demo room for targeting public and external visits, • dedicated staff for long-term operation, and • strategic space for new joint initiatives. In the laboratory under development, we have the following research equipment and software: i) MR smartglasses (two HoloLens 2 units), ii) VR headsets (one Meta Quest 2 unit), iii) AR tablets (one Samsung Galaxy Tab S7 series), iv) Eye trackers (three Tobii Eye Tracker 5 units), and v) cleaning devices for AR/VR HMDs (two CleanBox CX1 (Cone A/Cone B) units). The lab also has the iMotions biometric research software platform with the incorporated VR Eye Tracking Module. A screen has also been mounted outside the laboratory to disseminate results, invitations, and contact information for research studies and ongoing activities. The HINTS profile 1 brings 12 servers for AI computing and storage. More specifically, HINTS supplies 1) two GPU compute-oriented servers, each with an Nvidia Tesla-T4 GPU, a pair of Intel Xeon 6326 CPUs with 32 cores, 512 GB of RAM, and 12 TB of storage, 2) four general compute servers, each with a pair of Intel Xenon 4314 CPUs with 32 cores, 512 GB of RAM, and 12 TB of storage, 3) six general-purpose nodes, each with 32 TB of storage. In addition to these, HCIRL has a setup of four highend desktop PCs with various spreads in the combination of CPUs and GPUs. Two systems are based on Intel Core i9-12900K CPU, and the others have AMD Ryzen 9 5950X CPU. The GPUs are Nvidia RTX-3080Ti and AMD Radeon 6900XT. Each desktop is connected to two high-end screens with 1440p@170Hz or 4K@144Hz.

B. Research-Augmented Education
There have been several courses at BTH using the equipment so far, in addition to the research activities. A researcher from the HINTS profile is involved in the PA2570: Behavioural Software Engineering course, which explores human factors in software engineering. As part of the course involvement, the eye tracking technology in the environment is used together with analysis software during a lab assignment. Eye tracking has previously been used in education at BTH, for example, in a Visualization course curriculum [27]. The HoloLens 2 headset is also used by a student in the TE2502: Degree Project in the Master of Science in Engineering course for the final thesis project to evaluate gesture-based interaction in XR.
Networking research topics for IRs (cf. Section IV-E) have also been already included in BTH's education. For example, a degree project in the Master of Science in Telecommunication Systems (course ET2606) addresses the applicability of eBPF for in-network management for IRs. This educational work was reinforced by publishing a book on the programmability and virtualization of 5G and Beyond 5G networks [25]. Researchers from the HINTS profile have also separately been responsible for developing a new course, DV2583: Digital Ethics, as part of the 5-year engineering program "AI and machine learning". The course is crucial, covering issues related to AI and human behavior in the digital society.

C. Initial HINTS Publications
The Theme D researchers involved in HINTS are active in two main areas of the project: data mining and analysis and distributed and adaptive ML. In the first area, the researchers have recently published a comprehensive survey of stateof-the-art intelligent fault detection and diagnosis (FDD) in district heating (DH) systems [28]. The survey analyzes the developments in intelligent FDD for the DH domain, identifies current research gaps and techniques limitations, and supplies recommendations for future studies. These all will put the baseline and boost the planned collaboration research with some industrial partners involved in HINTS, e.g., NODA Intelligent Systems. In the second area, one of the pursued research directions involved studying approaches that bring efficiency and robustness to FL settings, as discussed in [29]. In a paper [30], researchers from Theme D have proposed a novel FL model that tries coping with statistically heterogeneous environments by introducing a group-personalized FL method. Such solutions are in the research focus of some of our industrial partners, e.g., Ericsson. Furthermore, the study published in [26] investigates Blockchains for the secure data exchange for distributed and federated AI/DL learning. Theme B has ongoing conference/journal submissions related to VR, CG, and AI. The research in these fields forms a crucial basis for multiple themes and subprojects in HINTS and reflects on this project's intertwined multidisciplinary aspect.

VI. DISCUSSION
The recent digitization of industrial and social processes generated not only a tsunami of data and information but also made users thinking how they can prevail in their interaction with digital systems and processes. In addition, the current COVID-19 pandemic forced people to find new ways of working, living, and interacting. As a result, they needed to use digital tools quickly, as recently researched by McKinsey [31]. However, the pandemic has also demonstrated how indispensable digital technologies are, as outlined on the international governmental level [32]. As a result, it became evident that current means to digitize processes and handle digital information are insufficient. Hence, new ways are required to understand, collaborate with, and control future generations of digital systems. HINTS will address such issues using a novel approach founded in visual and datadriven platforms which enable tangible, immersive cognitive interactions within real and virtual realities.
HINTS will address the identified challenges in the related literature as follows: i) concerning immersive computing, an iterative approach will be employed, where three areas will co-create results by exchanging each other's methods and results. These areas are to respectively measure, assess and model the QUX in novel IRs; ii) concerning novel environments and interaction techniques for IRs, HINTS will contribute via three pillars to various aspects of novel IRs, i.e., smart immersive virtual 3D spaces, smart interaction mechanisms, and performance efficiency; iii) about enhancing visual analytics for IRs, it will compare and assess visual analytics solutions, leverage advanced visualizations, and put forward and validate immersive visual analytics optimization; iv) within the context of adaptive and distributed AI for IRs, it will implement three types of solutions: distributed and efficient AI, adaptive AI, and advanced hybrid data analytics; and v) regarding networking for IRs, it aims at tackling the relationship of QUX and network performance, defining a relationship model that can be inverted, i.e., one that use network performance as input and activates network actions in the network if QUX objectives are not met, and investigating SDN-and in-network based flow management mechanisms for high-bandwidth and ultra-low-latency IR services. In addition, novel engineering concepts of 5G and 6G radio resource management for IRs will be researched.

VII. CONCLUSIONS
This paper has introduced the profile project HINTS -"Human-Centered Intelligent Realities". HINTS is centered around an ecosystem combining XR and communication paradigms to form novel intelligent digital systems and builds upon four previously funded projects, two financed by the Knowledge Foundation (the ViaTecH Synergy project and the research profile BigData@BTH) and two EU projects -Bonseyes and BonsApps. HINTS addresses the needs of six industrial partners: Blackdrop Interactive, Ericsson, IKEA Marketing & Communication, NODA Intelligent Systems, Spotify, and Virotea. Such needs were arranged into five interrelated strategic research themes of human-centered IRs, and are as follows: 1) novel experience assessment methodologies, 2) novel environments and interaction techniques, 3) visual analytics, 4) adaptive and distributed AI, and 5) networking.
HINTS aims to be the central Swedish node with a high international impact in human-centered IRs for next-generation digital societies. Finally, the HINTS profile contributes to the strategic direction of BTH (both in research and education) and the recruitment of competent new personnel at the Computer Science department. Being at the center of the BTH strategy towards digitalization, the HINTS focus and its complete environments are built upon strong academic programs, research excellence, and co-production with external partners.