Monday, November 28, 2011

Big problems on Computer Science and Machine Learning

One of the most intriguing questions that Richard Hamming raised in his talk : "You and your research" (which I'm sure most of you've red) is : What are the most important problems in your area?
In one hand it is obvious that one can only focus on single well defined thing to do "great" stuff but in the other, and as Feynman states:

"... You have to keep a dozen of your favorite problems constantly present in your mind, although by and large they will lay in a dormant state. Every time you hear or read a new trick or a new result, test it against each of your twelve problems to see whether it helps. ..."

 So I decided to do a quick search about this and found a Wikipedia entry, a StackOverflow entry and a very interesting blog from Andreas Zwinkau, who describes an experiment : "Fantasy Research Lab" proposed by Philip Greenspun.
The Fantasy Research Lab consists of

 " ... [you] pretending that you are the lab director for computer science at a brand-new research university and to come up with a plan for how you'd populate your lab with projects."

 I thought it was an interesting experiment so I asked prof. Fabio to try it out, he rapidly came up with the interesting problem of Automatic Programming.
Now I'm interested in knowing the point of view of some very bright people like you,  but not only regarding the entire Computer Science field (which by itself could extend to Operating Systems, Software Engineering, Security, Databases, Hardware and so on as Microsoft Academic Search unveils) but also the more specific Machine Learning, Information Retrieval and HPC view which is central to this blog.

Thanks for reading ;)

3 comments:

  1. Cool! All right, let me think...

    A computer vision system that is able to recognize objects even from very small and difficult cues. I mean, a system that works better than the human eye in recognizing stuff... that'd be awesome.

    Actually, you made me remember the quote "Any sufficiently advanced technology is indistinguishable from magic"... that's for real.

    ReplyDelete
  2. I found a phrase, don't really remember where, but it said something like " What would you do if you had an amazing computing power?" ... What would you do?

    ReplyDelete
  3. This is a very interesting topic, and I agree with Santiago, you must focus to be able to get a deeper understanding and eventually be able to make a contribution, but also you must keep your mind open. It is interesting that must of the problem listed in Wikipedia and StackOverflow are related to theoretical computer science (TCS), but there are not problems in other areas. A reason could be the fact that problems in TCS are very well defined (they are actually mathematical problems), whilst problems in other areas are more general and less formal, sort of "ill-posed" problems. Computer vision and automatic programming are good examples.

    I want to elaborate a little bit about automatic programming, since I thing the definition in Wikipedia is somehow restricted. When I think of automatic programming, I think of computers that can program themselves, starting from a very general specification. Something like telling the computer "HAL, please write a matrix factorization program". Obviously, this is a very general problem and could involve complex tasks ranging from language understanding to machine learning.

    There are many areas of computing which are related to this general problem:

    1. Programming languages. The development of high level programming languages (and the corresponding compilers) was motivated by the fact that direct programming of computers using machine code was a very hard task. However, programming is still a hard task, and we still would like to have higher level programming languages, and, in someway, software engineering works on the direction of making the task of programming easier.

    2. Program derivation and formal methods. The idea is to start from a formal specification of the task and then, automatically, generate (or derive) a program that satisfy the specification. This is tightly related to automatic theorem problem and has had important advances in last years. A good amount of the microcode in current processors is generated in this way using automatic verification systems. Also, functional programming languages are examples of the development of this research area.

    3. Language understanding. If the idea is to start from a high level, possibly-ambiguous and informal specification, the we will need some sort of natural language understanding capabilities. A good example of this is Watson, the new invention of IBM that combines natural language processing, an extensive knowledge base and inference algorithms.

    4. Machine learning. It is natural to use the same mechanism that humans use to program themselves, learning. This is in fact a different way to address a computational problem, instead of carefully coding a solution, provide enough samples and the computer must look for a solution (a program) to the problem. In its current form, machine learning could be seem as very restrictive to be considered automatic programming. However, the general framework applies very well, the hypothesis space corresponds to the set of possible programs, which could be as general as wanted, and the loss function corresponds to how well a particular program satisfies a given specification.

    I'm not sure whether this problem is equivalent to the one of building and artificial intelligent system, but it is clearly better defined. Even if we don't foresee a strategy to solve the general problem, we're clearly advancing towards its solution.

    ReplyDelete