Offensive Squid

cross posting from the philosophy thread we had a while back:

sheila said:

Yes, could someone expand on the idea of intelligence in itself. Can we measure how much intelligence exists in the same way that we can look at information and see how much noise it contains? can one think of intelligence analogously as someone thinks about information theory? I don't know about this topic at all, so I'd like if someone would help here.

and if one can, is that a way better than the turing test to see if something is in intelligent in a way that we would have a difficult time to identify since we only dimly recognize intelligence?

My goodness! I am catching up with metafilter and saw a FPP, Clever /monkey/human/robot/alien, in reference to an article to be published that helps me with the question.

Measuring universal intelligence: Towards an anytime intelligence test

Abstract:

In this paper, we develop the idea of a universal anytime intelligence test. The meaning of the terms “universal” and “anytime” is manifold here: the test should be able to measure the intelligence of any biological or artificial system that exists at this time or in the future. It should also be able to evaluate both inept and brilliant systems (any intelligence level) as well as very slow to very fast systems (any time scale). Also, the test may be interrupted at any time, producing an approximation to the intelligence score, in such a way that the more time is left for the test, the better the assessment will be. In order to do this, our test proposal is based on previous works on the measurement of machine intelligence based on Kolmogorov complexity and universal distributions, which were developed in the late 1990s (C-tests and compression-enhanced Turing tests). It is also based on the more recent idea of measuring intelligence through dynamic/interactive tests held against a universal distribution of environments. We discuss some of these tests and highlight their limitations since we want to construct a test that is both general and practical. Consequently, we introduce many new ideas that develop early “compression tests” and the more recent definition of “universal intelligence” in order to design new “universal intelligence tests”, where a feasible implementation has been a design requirement. One of these tests is the “anytime intelligence test”, which adapts to the examinee's level of intelligence in order to obtain an intelligence score within a limited time.

The project's website has a preprint and many other links of interest. Anytime Universal Intelligence (ANYNT)

Kolmogorov complexity wiki page.