Sigal Oren

AI systems are increasingly used to support human decision-making. In many cases, despite the algorithm's superior performance, the final decision remains in human hands. For example, an AI may assist doctors in determining which diagnostic tests to run, but the doctor ultimately makes the diagnosis. Focusing on these scenarios, this paper studies AI-assisted decision-making where the human learns through repeated interactions with the algorithm. In our framework, the algorithm - designed to maximize decision accuracy according to its own model - determines which features the human can consider. The human then makes a prediction based on their own, less accurate model. Additionally, we consider the possibility of a constraint on the number of features that can be taken into account.
We observe that the discrepancy between the algorithm's model and the human's model creates a fundamental trade-off. Should the algorithm prioritize recommending more informative features, encouraging the human to recognize their importance, even if it results in less accurate predictions in the short term until learning occurs? Or is it preferable to forgo educating the human and instead select features that align more closely with their existing understanding, minimizing the immediate cost of learning? This trade-off is shaped by both the algorithm's patience (the time-discount rate of its objective function over multiple periods) and the human's willingness and ability to learn.
Our results show that optimal feature selection has a surprisingly clean combinatorial characterization, reducible to a stationary sequence of feature subsets that is tractable to compute. As the algorithm becomes more patient or the human's learning improves, the algorithm increasingly selects more informative features, enhancing both prediction accuracy and the human's understanding of the world. Notably, early investment in learning leads to the selection of more informative features compared to a later investment. We complement our analysis by showing that the impact of errors in the algorithm's knowledge is limited as it does not make the prediction directly.