Beyond Machine Learning

Imagination, Continuous Learning, and Instincts

2016-01-11

We live in dangerous times for us humans. Technology is moving so fast, and there is lots of noise, an ocean of noise. Although isolated noise can be good because it generates awareness, diffused noise makes it hard to think clearly. The new generation of Machine Learning tools works so well that the discipline is being pushed into almost every industry. Unfortunately it has also generated misplaced expectations, and is commonly associated with Full / Strong /General Artificial Intelligence. Because of structural reasons, Machine Learning won’t lead there. Only a better understanding of the structures of intelligence will allow us to create Thinking Machines. How did we get here, and how do we jump beyond this wall?

Words matter. Associations matter. Otherwise it’s just noise.

Noise [nɔɪz] noun: 2. a series or combination of loud, confused sounds.

It is important to use appropriate context, words, and linguistic structures to avoid confusion; Confusion in collaboration leads to ︎↔️ instead of ⏩. The trouble is that the more technology advances, the more difficult it becomes to explain. Because technology affects all civilizations —all of us—, common and correct explanations must be established (Up Goer Five, for instance, is a valid approach. You can try it out here).

There is currently confusion in the context of Machine Learning, as its second word is overloaded:

Learning []ˈləːnɪŋ] noun: 1. the acquisition of knowledge or skills through study, experience, or being taught.

Learning is associated with intelligence, because of the implied capability to adapt to new situations, and to avoid repeating mistakes. This in turn begs for the shortcut to Artificial Intelligence, especially of the type seen in Science Fiction.

As visions go, this could be justified. If the conversation, however, is about the current state of the art, then the association between Machine Learning and Artificial General Intelligence is a misleading stretch.

What is then a better, more truthful description of the current state of the art of Machine Learning?

Current Machine Learning = Complex Pattern Recognition

Just a few years ago, Machine Learning was mostly confined to research, where it meant a “field of study that gives computers the ability to learn without being explicitly programmed”. More in detail,

Machine Learning is a subfield of computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence.

Now the vast majority of current (beginning of 2016) efforts in Machine Learning are still in Pattern Recognition, although the performance and complexity of the patterns being recognized has increased tremendously.

Pattern recognition is a branch of machine learning that focuses on the recognition of patterns and regularities in data, although in some cases it is considered to be nearly synonymous with machine learning*.

So why did Machine Learning impose itself as a default term over Pattern Recognition? Likely because it sounds better; more hopeful in a brighter future that holds the promises of the Jetsons or Futurama. To be clear, there are efforts that already go beyond complex pattern recognition, but they are very few, and it isn’t their results that led to the hype. They still need to clarify that they do or intend to go beyond Complex Pattern Recognition to show their value.

Why go beyond Complex Pattern Recognition?

But why go beyond Machine Learning? After all, it produces great accuracy in recognition (images, natural language, etc.), better results in online search, it is pushing fields beyond the state of the art (e.g., robotics), and it is even opening up new, mesmerising fields (e.g., DeepDreaming. See also samim’s excellent round-up of Assisted Drawing). When the current hype of Machine Learning will be over, we’ll have witnessed a paradigm shift in engineering, manufacturing, and even art. The artistic aspect of semi-handmade objects like a Ferrari will disappear when not supported by strong marketing (even then!). Art will likely be the field changing the most, with new currents and dominant figures emerging —Learnism and a 21st Century Picasso.

'Guernica' by Pablo Picasso, la exposición del Reina-Prado, Madrid, Spain. The original is overwhelmingly huge.

What makes us marvel now, tomorrow will be just another tool. Today no architect would design a house without CAD, and in 6 years it will be routine to train multi-layered models to solve specific tasks (non-Computer Science/Math/… students beware: you’ll need to be fluent in these technologies because you’ll define the processes, where to use which ML-system, etc. . University Deans beware: the future of your Institutions depends on curricula that include platform technical capabilities for all departments). We will have a set of incredible tools that provide super-human results.

But these tools won’t be able to think. Albeit counter-intuitive, this is more dangerous.

Machine Learning lacks imagination

Thinking [θɪŋkɪŋ] noun: 1. The action of using your mind to produce ideas, decisions, memories, etc. 2. The process of considering or reasoning.

Thinking encompasses many capabilities, including that of using its own results to produce new results. The multitude of capabilities and its internal feedback growth model is one of the key elements that it sets it apart from most of current Machine Learning systems. ConvNets and alike are trained with a large number of samples in view of a specific task, and then deployed to produce results. In the vast majority of cases, the training sets are of one type only: image, sound,… up to the point where it doesn’t really matter what type of data.

The structural inability of Complex Pattern Recognition systems to learn continuously (even from a single observation), make thought experiments, and change its own results (i.e. interpretation, even at an early processing stage) is what prevents them, at a profound level, from developing into thinking machines. If the situation changes and new data becomes available, the interpretation of the original data might be different. In some cases, it needs to change.

Tools that are trained to reach a certain objective without the possibility of reasoning are often the worst in terms of human survival. Especially in humorous situations.

What this means in the development is that Machine Learning will soon fall from hype, and start being productive as toolbox. Unfortunately, that will also constitute the next wall of AI research. This time having a few stubborn “Canadians” spending another winter knocking on this wall, answering people who ask: “Why bother with Artificial General Intelligence, when we have all these wonderful tools?”, will be too dangerous because the machines that we will have built will be powerful, competing for resources, and lacking imagination.

So, Students of all ages, use the current gold rush to make time to think and tinker with different architectures. Architectures that can actually think.

Going Beyond Complex Pattern Recognition

Thinking Machines will have imagination, learn continuously using multimodal (possibly active) feedback, and still be driven by fundamental instincts.

Imagination: A whole set of potential outcomes (thought experiments) are generated continuously. Imagination has its place in short- and long-term memory. The concept of sequences and consequences is central.

Continuous learning: A single observation can be sufficient to alter the outcome, enabled by strong and weak (dreaming) Feedback Activation.

Multimodal (possibly active) feedback: Different data types are the basis for interpretation. Feedback allows to re-evaluate situations. Active feedback allows carrying out experiments (e.g., using different inputs).

Be driven by fundamental instincts: Counter-intuitive element that directs the intelligence and, unintentionally, provides personality. Not subject to learning. Where to place “AI doomsday” antidotes.

One possible architecture is composed of modules that are semi-task-independent. The input data type specialises the modules, which are capable of carrying out Bayesian spatio-temporal sequence convolutions. The modules are arranged in hierarchical fashion to obtain abstraction. Feedback connections are numerous, allowing for sub-threshold activation, possibly by means of a different communication medium.

This architecture can be implemented on a smartphone.