In Philosophical Investigations and even in the Remarks on the Foundations of Mathematics, Wittgenstein talks about the contradictions in the "Rule Following" paradigm of human knowledge, epistemology and linguistics. The idea he is trying to address here is the idea that says that language is essentially a set of rules that try to communicate the intention of the speaker geared towards an action. The extreme interpretation of this idea is the "Language is pure syntax" interpretation of the computational paradigm of language spearheaded by Chomsky. It is agreed upon by most philosophers that this paradox is at the heart of the later Wittegenstein. No one is sure though, what Wittgenstein intends to communicate by the paradox.
Here is what Wittegenstein actually says: "This was our paradox: no course of action could be determined by a rule, because any course of action can be made out to accord with the rule"
The most famous interpretation of this is given by Kripke who interprets this in the skeptical tradition. To extend a rule to a novel situation, you have to assume that the rule applies to that situation. But this assumption has no logical basis. So you have to apply the procedure in an intuitive manner. So an unlimited application of the rule across a domain becomes an illogical device. An example would be to question the application of gravity throughout the universe. Since you can not test the fact about gravity beyond a certain physical boundary, it is illogical to assume the laws works throughout the universe, yet intuitively one knows that it is more illogical to assume the converse. Even in a simple domain like addition, as Kripke has postulated, it is difficult to apply a rule throughout the entire number series. In a way, this is extension of the Humean tradition of skepticism.
There are multiple interpretations of the paradox but most of them point towards a more intuitive, communal and traditional interpretation of human knowledge. In a way, Wittgenstein points towards the role of tradition, community and intuition in a process of learning of language for the child. I till try and continue this conversation in a series of posts, scattered over time. If you want to read more, here are some links for you: