README.org

#+TITLE: README #+DATE: <2018-01-05 Fri> #+AUTHOR: Jiaxuan Wang #+EMAIL: jiaxuan@umich #+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline author:t c:nil #+OPTIONS: creator:comment d:(not "LOGBOOK") date:t e:t email:nil f:t inline:t #+OPTIONS: num:t p:nil pri:nil stat:t tags:t tasks:t tex:t timestamp:t toc:nil #+OPTIONS: todo:t |:t #+CREATOR: Emacs 24.5.1 (Org mode 8.2.10) #+DESCRIPTION: #+EXCLUDE_TAGS: noexport #+KEYWORDS: #+LANGUAGE: en #+SELECT_TAGS: export

Code for paper [[https://arxiv.org/pdf/1711.03190.pdf][Learning Credible Models]] (presented at KDD 2018). The idea is extended to image input in [[https://openreview.net/pdf?id=yKYCwTvl8eU][Learning Concept Credible Models for Mitigating Shortcuts]] (presented at NeuRIPS 2022, [[https://github.com/nathanwang000/ConceptCredibleModel][link to code]])

In many settings, it is important that a model be capable of providing reasons for its predictions (i.e., the model must be interpretable). However, the model’s reasoning may not conform with well-established knowledge. In such cases, while interpretable, the model lacks credibility. In this work, we formally define credibility in the linear setting and focus on techniques for learning models that are both accurate and credible. In particular, we propose a regularization penalty, expert yielded estimates (EYE), that incorporates expert knowledge about well-known relationships among covariates and the outcome of interest. We give both theoretical and empirical results comparing our proposed method to several other regularization techniques. Across a range of settings, experiments on both synthetic and real data show that models learned using the EYE penalty are significantly more credible than those learned using other penalties. Applied to two large-scale patient risk stratification task, our proposed technique results in a model whose top features overlap significantly with known clinical risk factors, while still achieving good predictive performance.

Implementation of the EYE penalty can be found in [[lib/regularization.py]]. The synthetic data is generated via code in [[synthetic_data_generation.py]].

MIMIC2 dataset result is attained by running: #+BEGIN_SRC bash bash run.sh python eval_MIMIC2.py #+END_SRC

We provide a small example of using this package, please look through code in [[example.py]].

Install dependencies with #+BEGIN_SRC bash pipenv install #+END_SRC

One can load the trained final model by #+BEGIN_SRC python torch.load("models/test*.pt") # replace * with the actual model name #+END_SRC