By pointing to deep philosophical confusions endemic to cognitive science, Wittgenstein might seem an enemy of computational approaches. We agree (with Mills 1993) that while Wittgenstein would reject the classicist’s symbols and rules approach, his observations align well with connectionist or neural network approaches. While many connectionisms that dominated the later twentieth century could fall prey to criticisms of biological, pedagogical, and linguistic implausibility, current connectionist approaches can resolve those problems in a Wittgenstein-friendly manner. We (a) present the basics of a Vector Symbolic Architecture formalism, inspired by Smolensky (1990), and indicate how high-dimensional vectors can operate in a context-sensitive and object-independent manner in biologically plausible time scales, reflecting Wittgenstein’s notions of language-games and family resemblance; we (b) show how “soft” symbols for such a formalism can be formed with plausible learning cycles using Sparse Distributed Memory, resolving disputes surrounding Wittgenstein’s private language argument; and © show how connectionist networks can extrapolate meaningful patterns to solve problems, providing “ways to go on” without explicit rules, which indicates linguistic plausibility. Connectionism thus provides a systematicity and productivity that is more than a mere implementation of a classical approach, and provides Wittgenstein-friendly and Wittgenstein-illuminating models of mind and language for cognitive science.