Learning assessments in search-as-learning: A survey of prior work and opportunities for future research

People often search for information in order to learn something new. In recent years, the “search-as-learning” movement has argued that search systems should be better designed to support learning. Current search systems (especially Web search engines) are largely designed and optimized to fulfill simple look-up tasks (e.g., navigational or fact-finding search tasks). However, they provide less support for searchers working on complex tasks that involve learning. Search-as-learning studies have investigated a wide range of research questions. For example, studies have aimed to better understand how characteristics of the individual searcher, the type of search task, and interactive features provided by the system can influence learning outcomes. Learning assessment is a key component in search-as-learning studies. Assessment materials are used to both gauge prior knowledge and measure learning during or after one or more search sessions. In this paper, we provide a systematic review of different types of assessments used in search-as-learning studies to date. The paper makes the following three contributions. First, we review different types of assessments used and discuss their potential benefits and drawbacks. Second, we review assessments used outside of search-as-learning, which may provide insights and opportunities for future research. Third, we provide recommendations for future research. Importantly, we argue that future studies should clearly define learning objectives and develop assessment materials that reliably capture the intended type of learning. For example, assessment materials should test a participant’s ability to engage with specific cognitive processes, which may range from simple (e.g., memorization) to more complex (e.g., critical and creative thinking). Additionally, we argue that future studies should consider two dimensions that are understudied in search-as-learning: long-term retention (i.e., being able to use what was learned in the long term) and transfer of learning (i.e., being able to use what was learned in a novel context).     

Competition favors the prepared firm: Firms’ R&D responses to competitive market pressure

This paper aims to contribute to the literature on the long-debated relationship between market competition and firm research and development (R&D) by investigating the effect of competitive market pressure on firms’ incentives to invest in R&D. The paper shows that a firm's R&D response to competitive market pressure depends primarily on its level of technological competence or R&D productivity: firms with high levels of technological competence tend to respond aggressively (i.e., exhibit a higher level of R&D efforts) to intensifying competitive market pressure, while firms with low levels of technological competence tend to respond submissively (i.e., exhibit a lower level of R&D efforts). The differential effect of competitive market pressure on firm R&D, conditioned primarily by the level of firms’ technological competence, is empirically supported by unique firm-level data from the World Bank. Furthermore, the role of firm-specific technological competence in conditioning the R&D-competition relationship is more evident and statistically more significant for firms facing consumers whose utility is relatively more elastic to product quality than to price.     

More is not always better: Effects of collaboration breadth and depth on radical and incremental innovation performance at the project level

This study investigates the effect of project-level collaboration breadth (i.e., the number of collaboration partner types) and collaboration depth (i.e., the intensity of the interactions with these partners) on the incremental and radical innovation performance of innovation projects. The econometric analyses, based on a Community Innovation Study sample of 218 innovation projects conducted in German manufacturing companies, reveal inverted U-shaped relationships between collaboration breadth and radical innovation performance and between collaboration depth and incremental innovation performance. These curvilinear effects speak to the high sensitivity of innovation projects to the extent of collaborative activity, which implies that practitioners should exert caution in managing collaborative innovation projects. This study contributes to the understanding of project-level open innovation and to the overall understanding of the performance effects of innovation collaboration, which, following recent assertions in the literature, could be flawed by analyses conducted at the organizational level.     

To have or to be? The interplay between knowledge structure and market identity in knowledge-based alliance formation

We explore how a firm’s knowledge structure characteristics and market identity—as reflected in its position in the industry value chain and market status—influence knowledge-based alliance (KBA) formation. We propose that a firm’s propensity to form KBAs is affected by the congruence (or misalignment) between its knowledge structure characteristics (i.e., knowledge depth and scope) and the role it plays in the industry, as demanded by its position in the value chain. We further argue that while a firm’s market status can amplify the positive effect of the congruence, thereby enhancing a firm’s proclivity to form KBAs, being prominent can also hurt a firm when it faces a misalignment between its knowledge structure characteristics and externally induced expectations related to its value chain position. We validate our arguments using panel data on 1051 firm-year observations for 197 firms patenting in the global fuel cell industry for the period 1999–2009. By developing a richer understanding of the relationship between a firm’s knowledge-based resources and KBA formation, we answer the call for a better understanding of how and when firms can leverage their internal knowledge configurations to improve their proclivity to form alliances.     

中国水鳖科植物种皮微形态特征及其系统学意义

通过扫描电镜对国产水鳖科植物(包括6属13种)的种皮微形态特征进行观察，并作了系统描述。根据种皮细胞形态、外种皮表面纹饰和内种皮内层小瘤状突起的特点将水鳖科植物的种皮微形态特征划分为3种类型，即海菜花型(海菜花属)、水鳖型(水鳖属)和苦草型(苦草属、水筛属、虾子草属和黑藻属)，并作出了分属检索表。本文结果表明，种皮微形态特征可作为该科族、属以及属内种级水平分类的依据，对探讨属间关系和该科的系统发育关系亦具有重要的价值。种皮微形态特征支持Hutchinson(1959)和Eckhardt(1964)将海菜花属和水鳖属分别作为一个独立的族处理的观点。苦草属、水筛属和虾子草属种皮微形态特征的高度相似性表明它们间有密切的联系，不支持将它们置于不同亚科和族的分类处理。黑藻属虽与上述3属近缘，但其外种皮特征则较为独特，因此与水筛属放在不同族中更为合理。本文种皮微形态特征的研究结果支持iki1937)和Shaffer-Fehre(1991b)等关于水鳖科与茨藻科近缘的观点。     

On the role of human and machine metadata in relevance judgment tasks

In order to evaluate the effectiveness of Information Retrieval (IR) systems it is key to collect relevance judgments from human assessors. Crowdsourcing has successfully been used as a method to scale-up the collection of manual relevance judgments, and previous research has investigated the impact of different judgment task design elements (e.g., highlighting query keywords in the document) on judgment quality and efficiency. In this work we investigate the positive and negative impacts of presenting crowd human assessors with more than just the topic and the document to be judged. We deploy different variants of crowdsourced relevance judgment tasks following a between-subjects design in which we present different types of metadata to the human assessor. Specifically, we investigate the effect of human metadata (e.g., what other human assessors think of the current document, as in which relevance level has already been selected by the majority crowd workers), machine metadata (e.g., how IR systems scored this document such as its average position in ranked lists, statistics about the document such as term frequencies). We look at the impact of metadata on judgment quality (i.e., the level of agreement with trained assessors) and cost (i.e., the time it takes for workers to complete the judgments) as well as at how metadata quality positively or negatively impact the collected judgments.     

Societal trust and open innovation

While open innovation provides a new paradigm to sustain a firm’s competitive advantage, opening up to external knowledge also entails substantial risks of appropriation and opportunism. Building on this “open paradox” framework, this study investigates whether societal trust—a key aspect of informal cultural norms—serves as an effective mechanism in improving relational governance among partners, thereby leading to better collaborative outcomes. Using a novel panel data on co-owned patents across 29 countries, we show that firms in high trust countries are able to produce a higher level of joint output (i.e., co-owned patents). This effect is more pronounced when perceived opportunism is higher (i.e., firms in high-tech industries, or in countries with less disclosure transparency), and when formal contracts are less enforceable (i.e., in countries with relatively weak legal systems). We further show that open innovation is the channel through which societal trust promotes innovative efficiency. Overall, our study establishes societal trust as a key factor in influencing the efficiency of open innovation.     

Journalism and Democracy: An Evaluation of the Political Public Sphere,by Brian McNair

In this article, the author discusses how computer virtualization technologies (e.g., VMWare) provoke metaphysical and epistemological problems (e.g., a version of the experimenter's regress) similar to but not identical to familiar philosophical concerns. A sketch at resolution of one is proposed; the others are suggested as interesting open research problems in the area of potential limitations of computing.     

Systemic technology innovation management and analysis of other forms of IP protection

Industry 4.0 and the pandemic have caused recent changes in innovative development, economic situations, and restrictions in various companies and regions of the world, with new stricter legislation related to working conditions, hygiene standards, and health requirements. New technologies are a challenge to the transformation of problem companies and regions, as they often significantly impact economic development. Technological emissions, growing information on dementia among young people, cyber risks, and the risks of so-called advanced technologies (e.g., bio- and nanotechnologies, Industry 4.0 tech) may raise societal concerns about technological progress. This study is part of our long-term research into technological innovation in different companies and regions. Its primary goal is to methodically emphasize the importance and role of the systemic management of technological innovation in companies and regions and analytically compare the innovation success of differently developed regions from a global perspective¹. This research deals with an important aspect of technology management, namely, the processes of technological identification and the protection of related intellectual property (IP). The primary research question is: How has technological development been distributed across different regions over the last two decades? We hypothesize that Asian countries are increasingly moving away from Europe and North America in terms of technological innovation, other forms of technological IP protection, and overall access to innovation, while the focus is on the development of three non-patent forms of technological innovation protection: industrial designs, trademarks, and utility models. The results of our study reveal a significant difference in the approach to technological innovation management and the form of IP protection between so-called advanced and problem companies and regions. This study was conducted between 2018 and 2022.     

社交媒体倦怠:含义、前因及后果

[目的/意义]把握国内外关于社交媒体倦怠的研究现状和发现可能的研究机会。[方法/过程]从社交媒体倦怠的含义、前因和后果三方面对相关文献予以系统分析。[结果/结论]社交媒体倦怠主要从情感体验、行为表现和两者的综合3个视角来定义;社交媒体倦怠的前因主要包括社交媒体相关因素（系统特征因素、信息特征因素）和用户相关因素（心理性因素、行为性因素和社会性因素）;社交媒体倦怠的后果主要表现为影响用户的心理（不满意等）和行为（社交媒体不持续使用/转移行为、消极使用行为等）。     

Developing a platform-specific framework for web credibility assessment: A case of social Q&A sites

Social question-and-answer (Q&A) sites have the potential to serve as a useful source of online information based on their content-focused and collaborative nature. Although previous research has examined various attributes of high-quality information on social Q&A sites (e.g., best answers), relatively less attention has been paid to what affects users’ credibility assessments of information in the social Q&A context. The present study developed a social Q&A platform-specific framework for web credibility assessment, including 21 criteria under six types of web credibility, based on a literature analysis and case study of two online Q&A communities, Stack Exchange and Wikipedia Reference Desk. Using the selected sites’ policies and guidelines (n = 46) as the source of evidence, the case study revealed that content-related attributes (e.g., evidence-based, pertinence) were most frequently identified (12 of 21 criteria) as potential cues and heuristics for web credibility assessments of social Q&A sites, followed by author-related (five of 21; e.g., reputation) and design-related (four of 21; e.g., engaging design) factors. Design-related criteria were rarely included in previous models of web credibility on social Q&A or similar peer-knowledge production platforms. However, our findings showing that both Stack Exchange and Wikipedia Reference Desk have policies regarding all four design-related criteria in our framework—engaging design, moderation, design appropriateness, and ease of use—indicate the potential influences of design features on users’ web credibility assessment on social Q&A sites. Some differences emerged between the two cases, such as policies regarding the answerer's credentials or semantic accuracy that are present on Wikipedia Reference Desk but absent on Stack Exchange. Such differences in the sites’ policies reflect how they position themselves as social Q&A communities—Wikipedia, of which Wikipedia Reference Desk is a part, as an encyclopedia, and Stack Exchange as a community-based platform for learning, sharing knowledge, and building careers of users.     

Strong similarity measures for ordered sets of documents in information retrieval

A general method is presented to construct ordered similarity measures (OS-measures), i.e., similarity measures for ordered sets of documents (as, e.g., being the result of an IR-process), based on classical, well-known similarity measures for ordinary sets (measures such as Jaccard, Dice, Cosine or overlap measures). To this extent, we first present a review of these measures and their relationships.The method given here to construct OS-measures extends the one given by Michel in a previous paper so that it becomes applicable on any pair of ordered sets. Concrete expressions of this method, applied to the classical similarity measures, are given.Some of these measures are then tested in the IR-system Profil-Doc. The engine SPIRIT^© extracts ranked document sets in three different contexts, each for 550 requests. The practical usability of the OS-measures is then discussed based on these experiments.     

The reflection of offline activities on users’ online social behavior: An observational study

The ever increasing presence of online social networks in users’ daily lives has led to the interplay between users’ online and offline activities. There have already been several works that have studied the impact of users’ online activities on their offline behavior, e.g., the impact of interaction with friends on an exercise social network on the number of daily steps. In this paper, we consider the inverse to what has already been studied and report on our extensive study that explores the potential causal effects of users’ offline activities on their online social behavior. The objective of our work is to understand whether the activities that users are involved with in their real daily life, which place them within or away from social situations, have any direct causal impact on their behavior in online social networks. Our work is motivated by the theory of normative social influence, which argues that individuals may show behaviors or express opinions that conform to those of the community for the sake of being accepted or from fear of rejection or isolation. We have collected data from two online social networks, namely Twitter and Foursquare, and systematically aligned user content on both social networks. On this basis, we have performed a natural experiment that took the form of an interrupted time series with a comparison group design to study whether users’ socially situated offline activities exhibited through their Foursquare check-ins impact their online behavior captured through the content they share on Twitter. Our main findings can be summarised as follows (1) a change in users’ offline behavior that affects the level of users’ exposure to social situations, e.g., starting to go to the gym or discontinuing frequenting bars, can have a causal impact on users’ online topical interests and sentiment; and (2) the causal relations between users’ socially situated offline activities and their online social behavior can be used to build effective predictive models of users’ online topical interests and sentiments.     

On the class separability of contextual embeddings representations – or “The classifier does not matter when the (text) representation is so good!”

The literature has not fully and adequately explained why contextual (e.g., BERT-based) representations are so successful to improve the effectiveness of some Natural Language Processing tasks, especially Automatic Text Classifications (ATC). In this article, we evince that such representations, when properly tuned to a target domain, produce an extremely separable space that makes the classification task very effective, independently of the classifier employed for solving the ATC task. To demonstrate our hypothesis, we perform a thorough class separability analysis in order to visualize and measure how well BERT-based embeddings separate documents of different classes in comparison with other widely used representation approaches, e.g., TFIDF BoW, static embeddings (e.g., fastText) and zero-shot (non-tuned) contextual embeddings. We also analyze separability in the context of transfer learning and compare BERT-based representations with those obtained from other transformers (e.g., RoBERTa, XLNET). Our experiments covering sixteen datasets in topic and sentiment classification, eight classification methods and three class separability metrics show that the fine-tuned BERT embeddings are highly separable in the corresponding space (e.g., they are 67% more separable than the static embeddings). As a consequence, they allow the simplest classifiers to achieve similar effectiveness as the most complex methods. We also find moderate to high correlations between separability and effectiveness in all experimented scenarios. Overall, our main finding is that more discriminative (i.e., separable) textual representations constitute a critical part of the ATC solutions that, given the current state-of-the-art in classification algorithms, are more prominent than the algorithmic (classifier) method for solving the task.     

Review Article��Dielectrophoresis: Status of the theory, technology, and applications

A review is presented of the present status of the theory, the developed technology and the current applications of dielectrophoresis (DEP). Over the past 10 years around 2000 publications have addressed these three aspects, and current trends suggest that the theory and technology have matured sufficiently for most effort to now be directed towards applying DEP to unmet needs in such areas as biosensors, cell therapeutics, drug discovery, medical diagnostics, microfluidics, nanoassembly, and particle filtration. The dipole approximation to describe the DEP force acting on a particle subjected to a nonuniform electric field has evolved to include multipole contributions, the perturbing effects arising from interactions with other cells and boundary surfaces, and the influence of electrical double-layer polarizations that must be considered for nanoparticles. Theoretical modelling of the electric field gradients generated by different electrode designs has also reached an advanced state. Advances in the technology include the development of sophisticated electrode designs, along with the introduction of new materials (e.g., silicone polymers, dry film resist) and methods for fabricating the electrodes and microfluidics of DEP devices (photo and electron beam lithography, laser ablation, thin film techniques, CMOS technology). Around three-quarters of the 300 or so scientific publications now being published each year on DEP are directed towards practical applications, and this is matched with an increasing number of patent applications. A summary of the US patents granted since January 2005 is given, along with an outline of the small number of perceived industrial applications (e.g., mineral separation, micropolishing, manipulation and dispensing of fluid droplets, manipulation and assembly of micro components). The technology has also advanced sufficiently for DEP to be used as a tool to manipulate nanoparticles (e.g., carbon nanotubes, nano wires, gold and metal oxide nanoparticles) for the fabrication of devices and sensors. Most efforts are now being directed towards biomedical applications, such as the spatial manipulation and selective separation∕enrichment of target cells or bacteria, high-throughput molecular screening, biosensors, immunoassays, and the artificial engineering of three-dimensional cell constructs. DEP is able to manipulate and sort cells without the need for biochemical labels or other bioengineered tags, and without contact to any surfaces. This opens up potentially important applications of DEP as a tool to address an unmet need in stem cell research and therapy.     

Unravelling knowledge creation and management in educational organisations: barriers and enablers

Knowledge Creation and Management (KCM) is one of the main strategies for fostering improvement in any kind of organisation, including educational organisations, on which this paper is focused. While KCM is being widely studied, little attention has been given to KCM in organisations supplying education, training and/or assessment services to learners (i.e., primary and secondary schools or higher education institutions, among others). From an educational management approach, the aim of this paper is to describe and analyse the factors that foster or inhibit KCM processes conducted within educational organisations. Professionals (n=84), moderators and managers or institutional leaders of four selected cases were surveyed and interviewed. Results show the importance of the main organisational and procedural factors (i.e., organisational values and goals, leadership, structure, culture, dynamics, Information and Communications Technologies, KCM processes and people) to improve KCM results in educational organisations.     

Work-related factors influencing doctors search behaviors and trust toward medical information resources

Regulation of clinical practice is a characteristic aspect of the medical profession. Regardless of whether this regulation derives from government-sourced guidelines or materials from government-sponsored institutions, it results in a high production of information resources (institutional information resources), which are disseminated to the clinical stuff in order to ensure compliance. In that case, the issue of credibility of these information resources might arise, since medical practice is characterized by a high frequency of change. The latter involves a continuous effort on the part of the clinical staff, which is motivated by work-related factors (e.g., need for compliance) or personal motivation (e.g., need for self-improvement). In this study we consider a simple trust model, according to which we assume that perceived trust is a direct antecedent of perceived credibility. We evaluate whether work-related or personal motivating factors influence the relation between perceived credibility and trust toward institutional information sources and how the effect of each factor affects this relation. Findings suggest that work-related factors have a higher impact on the relation between credibility and trust than personal motivation factors, while they are stressing the important role of hospital libraries as a dissemination point for government-sponsored information resources.     

人际不信任、消极情感与知识隐藏行为研究

人际不信任作为判断人际关系质量的重要标准会导致知识隐藏,但至今鲜有研究探讨二者之间的内在机制,削弱了实践指导性。依据情感事件理论和心理所有权理论,重点关注消极情感在人际不信任影响员工知识隐藏行为过程中的中介作用,以及员工知识心理所有权在上述关系中的调节作用。通过对324个样本的层级回归分析发现：（1）消极情感部分中介人际不信任对员工知识隐藏行为的影响;（2）知识心理所有权不仅对消极情感与知识隐藏行为之间的关系具有显著的调节作用,而且显著调节人际不信任通过消极情感影响员工知识隐藏行为的中介作用,表现为员工的知识心理所有权水平越高,消极情感在人际不信任和员工知识隐藏行为关系间的中介作用越弱。