Ah, sorry, I was working on mobile and confused you with the author of the article, thanks for the clarification!

Now reading you both as separate people, I think I totally agree with you. I think the 1% error rate is really good evidence for statistical rather than causal reasoning.

However, I also wonder how often humans actually do causal reasoning rather than statistical extrapolation. My curiosity is based on the fact that

1. Humans also make mistakes.

2. If the network's model is "correct" in the sense that it allows it to make the right prediction in many various cases, isn't it doing causal reasoning?

3. If the answer to (2) is "no", then what if that representation is sufficiently small such that it mathematically couldn't be doing the Chinese room because it doesn't have the space to store a lookup table that big? *Then* is it doing reasoning?

In short, it seems like it's the probabilistic nature of it that bothers you, but it also seems like there's a probabilistic nature to human understanding as well, and I'm curious as to how you differentiate between the two.