5월 2주차

Online reviews can shape where people stay, eat, and shop, but businesses often struggle to keep up with the flood of customer feedback. Although generative artificial intelligence (AI) offers a promising solution, general-purpose models are not designed for the specific judgment, tone, and accuracy required in customer review responses. This study introduces a new fine-tuning method that helps large language models generate review replies that better match human preferences in real business settings. The paper makes several technical advances. It identifies why review-response systems hallucinate and introduces a context-augmentation strategy to reduce factual errors. It also develops a theory-driven way to automatically construct preference data from existing review-response records, overcoming a major barrier in preference fine-tuning. In addition, the study proposes a curriculum learning design and a new support-constraint method that reduces the overconservatism of existing offline optimization approaches, with stronger theoretical guarantees. Tests on hotel reviews show that the method produces better responses than leading alternatives in both automated evaluations and human judgments. The findings point to a practical path for using AI to help firms respond faster and more consistently to customers while also underscoring the need for safeguards, human oversight, and domain-specific model alignment in customer-facing AI systems.

Screening human capital based on signals such as job applications or entrepreneurial pitches is crucial for organizations. Signals are often informative insofar as they require differential knowledge and effort to produce. Generative AI (GAI) complicates screening by lowering the cost of producing impressive signals. We model the informational effects of GAI, showing that applicants’ access to GAI can increase—and also decrease—an evaluator’s screening mistakes. This result depends on how GAI affects experts’ signals compared with nonexperts’. Using experiments in hiring and start-up investing, we estimate that senders’ access to GAI (ChatGPT) lowers screening accuracy by 4%–9% for employers and start-up investors. Consistent with our model, senders’ access to GAI also improves screening accuracy in some settings, in our case, among senders from non–English-speaking countries. These results show that GAI can profoundly shape screening accuracy. This paper was accepted by Anindya Ghose, information systems. Funding: We are grateful for the Columbia Business School Digital Future Initiative Grant for helping fund this project. B. Cowgill thanks the Kauffman Foundation Emerging Scholars Program, the Columbia Center for Political Economy, the NET Institute, and the Stellar Development Foundation. P. Hernandez-Lagos thanks the Yeshiva University Sy Syms Dean’s Research Fund. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2024.07027 .