Regularizing query-based retrieval scores

We adapt the cluster hypothesis for score-based information retrieval by claiming that closely related documents should have similar scores. Given a retrieval from an arbitrary system, we describe an algorithm which directly optimizes this objective by adjusting retrieval scores so that topically related documents receive similar scores. We refer to this process as score regularization. Because score regularization operates on retrieval scores, regardless of their origin, we can apply the technique to arbitrary initial retrieval rankings. Document rankings derived from regularized scores, when compared to rankings derived from un-regularized scores, consistently and significantly result in improved performance given a variety of baseline retrieval algorithms. We also present several proofs demonstrating that regularization generalizes methods such as pseudo-relevance feedback, document expansion, and cluster-based retrieval. Because of these strong empirical and theoretical results, we argue for the adoption of score regularization as general design principle or post-processing step for information retrieval systems.

Semiconductor Research Corporation: A Case Study in Cooperative Innovation Partnerships

In the study of innovation institutions, it is important to consider how different institutional models can affect a research organization in conducting or funding successful work. As an industry collaborative, Semiconductor Research Corporation (SRC) provides an example of a privately funded institution that leverages the inputs of several member companies, along with federal funding, to accomplish innovation in its mission area. SRC has several component programs, all attempting to find innovative solutions to semiconductor problems, but on different time scales, and in different technology areas. But how does SRC use its resources to ensure these goals? Through data gathered from semi-structured qualitative interviews and SRC documentation, this paper addresses that question. SRC has found a way to leverage industry money to motivate and develop a robust field of university research for over 30 years. SRC uses several mechanisms for maintaining an application focused, member-centered decision process, institutional flexibility, and strong ties between industry contributors and university researchers. SRC has continued to keep its members satisfied by training thousands of graduate students for employment in their member companies, by focusing on precompetitive research that addresses industry requirements, and doing so in a manner that operates leanly, with low overhead to its funders. Given these successes, we identify aspects of SRC operations, such as a focus on its member company needs, frequent interactions between funders and researchers, flexible funding mechanisms, and focus on workforce development, that may be diffusible to innovation institutions, including federal research efforts.