The field of Machine Learning research is divided into subject areas, whereeach area tries to solve a specific problem, using specific methods. In recentyears, borders have almost been erased, and many areas inherit methods fromother areas. This trend leads to better results and the number of papers in thefield is growing every year. The problem is that the amount of information isalso growing, and many methods remain unknown in a large number of papers. Inthis work, we propose the concept of inheritance between machine learningmodels, which allows conducting research, processing much less information, andpay attention to previously unnoticed models. We hope that this project willallow researchers to find ways to improve their ideas. In addition, it can beused by researchers to publish their methods too. Project is available by link:https://www.infornopolitan.xyz/backronym
Quick Read (beta)
The field of Machine Learning research is divided into subject areas, where each area tries to solve a specific problem, using specific methods. In recent years, borders have almost been erased, and many areas inherit methods from other areas. This trend leads to better results and the number of papers in the field is growing every year. The problem is that the amount of information is also growing, and many methods remain unknown in a large number of papers. In this work, we propose the concept of inheritance between machine learning models, which allows conducting research, processing much less information, and pay attention to previously unnoticed models. We hope that this project will allow researchers to find ways to improve their ideas. In addition, it can be used by researchers to publish their methods too. Project is available by link: https://www.infornopolitan.xyz/backronym
Arip Asadulaev INFORNOPOLITAN [email protected]
August 8, 2019
Keywords Machine Learning 3D visualization
The knowledge’s of Machine Learning researchers is limited, we know a small part of the total amount that can be very useful. Researchers analyzed hundreds of articles each year, but it is rather difficult to structure such type of information, and it is necessary to write down dependencies and connections between the methods.
Today, almost any new method is the development of an older method or compilation of a set of methods. It is important to note that some method may not include another method fully, but only some part of it. For example, GAN consist of Generator(GEN), Discriminator (DIS), Adversarial Autoencoder (AAE) based on Autoencoder (AE) and DIS. Unfortunately, the citation does not reflect very well which methods are directly used in the architecture.
To represent such dependencies as a table or a list has its drawbacks. In our opinion, a strictly fixed graph, with edges between methods and their components is the best way. In such graph every component is a separated node, so for example for AAE we will have an edge to AE and DIS.
During papers analyzing, we realized how many methods with excellent results are still not used, due to the fact that they were lost among many other papers. There are a lot of models being developed, and often research’s on a new conference do not inherit most of the improvements proposed a year ago.
There are many amazing services that allow you to monitor the emergence of new articles in the field. But all these methods were aimed primarily at the acquisition of knowledge, but not for help in research. Our platform aims to diversify research. Creativity is more important than the experience, and tools that can help you to be more creative should be an integral part of the research process.
We have tools that make it easy to conduct experiments, run and evaluate models, but we do not have methods that allow us to extend our model in a ideas level. In Machine learning, and Artificial Intelligence research as a whole, fresh and elegant ideas play a key role.
Other fields of science cannot boast of such tools too, but we are absolutely sure that Machine Learning and other areas that solve difficult problems, needs an extraordinary approach for generating ideas.
2.1 Tech details
In graph each paper can be presented as one single method, for example, Autoencoder (AE), or separately, Autoencoder (AE) -> Encoder (ENCDR), Decoder (DCDR), That is, one model can consist of several elements that can be used separately.
In just a few months, we took out about 250 articles from the NeurIPS 2019 conference and 250 other papers on which new are based. Graph has been built based on a table with 10 columns:
Link to paper
Names of authors
Place of publication
Subject area (Using For)
Acronym of method name
A brief description of the method
"Based on":the list of methods on which this method is based (list of acronyms which are available in table).
The graph is built using "Based on" column, where each row consists of a list of methods on which the method is based, Fig. 1.
The way we analyzed papers is very far from ideal. Most of the areas were completely unfamiliar to us, to understand them took several days. Sometimes we could not find the right description for some methods, and the abstract of the article was used for this. The graph is interactive, clicking on the node opens a list with meta-information from columns, Fig. 2.
The “Subject area (Using For)” column allows creating subgraphs with inheritances inside the field, where the method is marked in red if it using in other areas, see Fig. 3.
2.2 How to use?
Now the simplest way to use the graph it skips connections. For example, you use CNN in your model, and in a graph, you can see another method that inherited from CNN. You can make little research and try just to replace your CNN with the advanced version.
It is very difficult to create an inheritance graph that would be fully consistent with the truth. Based on this, we created an opportunity for authors to add their own methods and make changes to existing ones in the graph. After all, no one except the authors of the article knows how to disassemble and describe their method in the best way. https://www.infornopolitan.xyz/add-research
Nevertheless, I believe that the main thing that the graph can give is associations. I was excited to know how GAN idea occurred to Ian Goodfellow. Talking on Artificial Intelligence podcast by Lex Fridman, Ian said that it was motivated by Boltzmann machine “positive” and “negative” training phases. This is a great example when a model inherits the properties of another not directly but very abstractly.
I would very much like to see this type of connections in this graph, and it depends solely on authors of papers, will or not they share what was motivated them to create such type of model to solve some problem. I think that the story not only about what methods were used specifically in their architecture but also ideologically on which concepts and knowledge the solution was formed, can allow us to be much more resourceful.
This project is community-driven. We want to make it better and motivate more peoples to add their models to the graph. More accurate information about methods and better visualization technologies can really make it a very useful tool.
3 Discussion and future work
In the future, with community support, we can level up BACKRONYM and built the system which will recommend us how to extend research. For this scenario, we plan to give to users the ability to visualize their own knowledge, publicly or privately, irrespective of the main graph. For example, user can build a graph of all methods which he knows or even graph of one model components, and the system will recommend him the most useful paper or method. Also, we plan to add: 1) 2D visualization. 2) Ability to build and save individual subgraphs. 3) Search by method name.
It seems that even today this graph take a place to be because it probably allows someone to get associations that may help to create some new method or extend current. Today, it’s just a graph, but tomorrow we will have the tools to objectively get machine recommendations and automatically evaluate the impact the proposed idea on the way to General Machine Intelligence systems.
-  Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. Generative Adversarial Nets. https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf.
-  Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, Brendan Frey. Adversarial Autoencoders. https://arxiv.org/abs/1511.05644
-  Dana H. Ballard. Autoencoder. https://www.aaai.org/Papers/AAAI/1987/AAAI87-050.pdf.
-  Yann Lecun, Patrick Haffer, Leon Bottou and Yoshua Bengio. Convolutional Neural Network. http://yann.lecun.com/exdb/publis/pdf/lecun-99.pdf
-  Ian Goodfellow: Artificial Intelligence podcast at MIT. https://www.youtube.com/watch?v=Z6rxFNMGdn0