Drive More Effective Data-Based Innovations: Enhancing the Utility of Secure Databases

Yi Qian, Hui Xie

NBER Working Paper No. 19586
Issued in October 2013
NBER Program(s):Productivity, Innovation, and Entrepreneurship

Databases play a central role in evidence-based innovations in business, economics, social, and health sciences. In modern business and society, there are rapidly growing demands for constructing analytically valid databases that also are secure and protect sensitive information in order to meet customer and public expectations, to minimize financial losses, and to comply with privacy regulations and laws. We propose new data perturbation and shuffling (DPS) procedures, named MORE, for this purpose. As compared with existing DPS methods, MORE can substantially increase the utility of secure databases without increasing disclosure risk. MORE is capable of preserving important nonmonotonic relationships among attributes, such as the inverted-U relationship between competition and innovation. Maintaining such relationships is often the key to determining optimal levels of policy and managerial interventions. MORE does not require data to be of particular types or have particular distributional shapes. Instead, it provides unified, flexible, and robust algorithms to mask general types of confidential variables with arbitrary distributions, thereby making it suitable for general-purpose data masking. Since MORE nests the commonly used generalized linear models as special cases, a much wider range of statistical analyses can be conducted using the secure databases with results similar to those using the original databases. Unlike existing DPS approaches which typically require a joint model for all variables, MORE requires no modeling of nonconfidential variables, and thus further increases the robustness of secure databases. Evaluation of MORE through Monte Carlo simulation studies and empirical applications demonstrates that it performs better than existing data masking methods.

download in pdf format
   (407 K)

email paper

Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w19586

Published: Yi Qian & Hui Xie, 2015. "Drive More Effective Data-Based Innovations: Enhancing the Utility of Secure Databases," Management Science, vol 61(3), pages 520-541.

Users who downloaded this paper also downloaded* these:
Luo, Lovely, and Popp w19518 Intellectual Returnees as Drivers of Indigenous Innovation: Evidence from the Chinese Photovoltaic Industry
Nanda and Rhodes-Kropf w19379 Innovation and the Financial Guillotine
Qian w17849 Brand Management and Strategies Against Counterfeits
Qian w16785 Counterfeiters: Foes or Friends? How Do Counterfeits Affect Different Product Quality Tiers?
Qian, Gong, and Chen w18784 Untangling Searchable and Experiential Quality Responses to Counterfeits
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us