LIME is supposed to help you explain your model's predictions — but the explanations themselves are wildly unstable. Run explain_instance() on the exact same data point twice in a row, and you'll get different feature importances both times. Sometimes dramatically different. A feature that was ranked #1 in explaining a prediction one run might barely show up in the next.
This isn't just a minor annoyance. If you're using LIME in a regulated environment — healthcare, finance, legal — and your explanation changes every time a compliance officer runs it, that's a serious problem. You literally cannot give a consistent answer for why the model made a particular decision. The explanation tool becomes untrustworthy, which defeats the entire point of using it.
LIME is supposed to help you explain your model's predictions — but the explanations themselves are wildly unstable. Run explain_instance() on the exact same data point twice in a row, and you'll get different feature importances both times. Sometimes dramatically different. A feature that was ranked #1 in explaining a prediction one run might barely show up in the next.
This isn't just a minor annoyance. If you're using LIME in a regulated environment — healthcare, finance, legal — and your explanation changes every time a compliance officer runs it, that's a serious problem. You literally cannot give a consistent answer for why the model made a particular decision. The explanation tool becomes untrustworthy, which defeats the entire point of using it.