All the most important things boil down to correlations. High intakes of olive oil correlate with lower heart disease; use of condoms correlates with lower rates of HIV; greater use of coal correlates with increased erosion of masonry. Everything is a correlation, until the underlying cause/effect relationship is established.

So.

MaxCo is launching the Database of Correlatable Information.

Users can upload any data they like, rather like YouTube, but in a limited number of standard formats. For example, you might have records of the rainfall in Leeds for the last two decades. Someone else might have data on the frequency of mobile phone ownership across Africa. Someone else else might have counted the number of swallows nesting in a barn in Tashkent over the last decade. Everyone knows something.

Each dataset has to contain a minimum of 30 values (to make it worthwhile). Standard forms will be available for values distributed over time (eg, swallow nests per year), space (eg, mobile phone density versus geographic location) or both (eg, lightning strikes on aircraft divided by location and year).

Users will, of course, be able to browse this atticful of data as they wish, using a variety of graphical representations. I can imagine many happy hours looking at the annual cycle of Accident and Emergency admissions in Stoke on Trent, or the region-by-region breakdown of otter sightings.

This, however, is mere frippery.

The main purpose of the database is to unearth correlations. Algorithms will regularly trawl the data, looking for all possible correlations between datasets. Positive correlations (maybe between rainfall and road accidents), negative correlations (maybe between fluorescent light-bulb sales and sunburn), or correlations offset in time or space. Day and night, the algorithms will trawl and sift, reporting all significant correlations.

Users can then browse these correlations. Does anything correlate with the number of coffee shops per square mile? Yes - ownership of mobile phones correlates positively, whilst sales of tractor tyres correlates negatively. How about otter sightings? They correlate with local rainfall positively by time, but negatively with sales of hand-cream in space. And strangely, cheese exports from Menorca show a significant (p<0.02) correlation with typhoons.

A million datasets, a trillion possible correlations....Many will be spurious (like otters and hand-cream), but some (such as between television sales and influenza, or cheese and typhoons) will reveal new connectednesses in this otherwise uncorrelated and confusing universe.
-- MaxwellBuchanan, Feb 10 2009

Superstition in the pigeon http://en.wikipedia...ition_in_the_pigeon
A bit of B.F. Skinner's work that this reminds me of. [Spacecoyote, Feb 10 2009]

The Journal of Spurious Correlations http://www.jspurc.org/
[phoenix, Feb 10 2009]

House of Stairs http://en.wikipedia...liam_Sleator_novel)
A novel about conditioning similar to Skinner's pigeons, but with humans [simonj, Feb 10 2009]

Helvetica Scenario http://www.youtube....watch?v=aY7XH2ulTEU
Can be managed by homoeopathic Hanley - note the comment by amishrobots. [nineteenthly, Feb 11 2009]

... So, the inevitable buns for this idea will clearly correlate to the global economic downturn.

... Soooooo. hmmm.

Start posting bad ideas, yes?
-- Custardguts, Feb 10 2009


Ben Elton's 'Blind Faith' did something like this in a branch of NatDat (National Database) called DegSep (Degrees of Separation).
To quote:
"Trafford's department had recently been astonished to discover that the DegSep computers were not linking preference in pre-cooked meals with parental star signs; hence the fact that an individual with at least one Taurean parent had a very slight statistical likelihood to eat lasagne more often than a person with two Sagittarian parents had lain completely hidden."

I am sure that you didn't know this, of course!
-- gnomethang, Feb 10 2009


Aha - it seems that Phoenix and Gnomethang have found some prior art...
-- MaxwellBuchanan, Feb 10 2009


As long as the algorithms can actually assign a stastical signifigance to the correlation, I think this would be useful. Obviously it would only work for things that are readily quantitized, and of course correlation is not causation, but it could provide interesting areas of research (and quite a few doctoral thesis regardless of research validity).
-- MechE, Feb 10 2009


To be fair, you have taken things a bit further.
My first thought was the works of the Daily Mail:
'Drinking Wine Causes Cancer!'
'Drinking Wine Can Help Fight Cancer!'
'Can Drinking Wine affect your House Price?!'
Repeat Ad Nauseum.....
-- gnomethang, Feb 10 2009


sounds like a fun website [+]
-- simonj, Feb 10 2009


//As long as the algorithms can actually assign a stastical signifigance to the correlation// Should be no problem, at least in a one-size-fits-all way. There are standard algorithms to handle correlations of quantitative data, and they assign a probability of the apparent correlation being due to chance alone. Hence, p=0.01 means that the data are so correlated that there's only a 1/100 chance of that degree of correlation being seen by chance alone.
-- MaxwellBuchanan, Feb 10 2009


Hey! I've been to Stoke on Trent !
-- normzone, Feb 10 2009


We know.
-- MaxwellBuchanan, Feb 10 2009


It was closed.
-- gnomethang, Feb 10 2009


Finding correlations is like playing billiards. To have it count you need to call your shot before you make it.
-- bungston, Feb 10 2009


Sounds rather like my entire life.

[Custardguts], doesn't this mean we should stop posting _good_ ideas?
-- nineteenthly, Feb 11 2009


//Hey! I've been to Stoke on Trent //
Counselling is available to help you get over it.
-- coprocephalous, Feb 11 2009


Excellent - and I think it's jolly fortunate you managed to post this idea before we all posted these annotations on it. Imagine if you'd posted it a day later!
-- hippo, Feb 11 2009


I've been to Stoke on Trent too. Is there a correlation between annotators on here and going to Stoke on Trent?
-- nineteenthly, Feb 11 2009


//sales of hand-cream in space//
I always assumed that the astronauts got it for free.
-- coprocephalous, Feb 11 2009


//Is there a correlation between annotators on here and going to Stoke on Trent?//

It seems increasingly likely. Not only did I live in Stoke-on-Trent for a while but I was also an emergency admission on one occassion.

Post-hoc ergo propter hoc.
-- DrBob, Feb 11 2009


<someone had to say it>
//Cheese causes typhoons// No, just nasty wind sometimes.
</shtsi>
-- pertinax, Feb 11 2009


So [DrBob], is mere presence in Stoke On Trent itself therapeutic? I have often heard the opposite. Are homoeopathic doses of Stoke On Trent good for geodermic granititis?
On a slightly related note, did anyone know that the ghost of Tchaikovsky's star sign is the Croissant?
-- nineteenthly, Feb 11 2009


Well that depends upon which of the five towns you are in and at what precise time. For example, the currently worst time & place to be in Stoke, spiritually speaking, is at the Brittania Stadium at around about twenty to five on a Saturday afternoon.

I'd rely on Anti-Cobble cream* for the granititis though.


*Thank you for another credibility save Wikipaedia!
-- DrBob, Feb 11 2009


//correlations offset in time or space// So 25 years before and 1800 miles away from a spike in coffee shop openings in Stoke on Trent, there was a rapid decrease in otter numbers. I think that's a good reason to ban coffee shops.

I went by narrow boat to Stoke-on-Trent.
-- MadnessInMyMethod, Feb 11 2009


Yes, both those annos make a lot of sense. I tend to equate the Potteries with the Medway Towns, which makes Newcastle under Lyme into Gillingham. Maybe there are subtle distinctions between the precise actions of different Potteries, so that Stoke is more directly for granititis, whereas Hanley is good at preventing the Helvetica Scenario. Certainly the calcium content of the ceramics would seem to suggest that.
-- nineteenthly, Feb 11 2009



random, halfbakery