h a l f b a k e r yRenovating the wheel
add, search, annotate, link, view, overview, recent, by name, random
news, help, about, links, report a problem
browse anonymously,
or get an account
and write.
register,
|
|
|
Distributed computing market
Have a market in which you decide how to allocate your processing credit but not what is processed on your computer. | |
You are probably familiar with Seti at home (Search for Extra Terrestrial Inteligence) It is one of the more popular of the distributed computing programs and received awards for making a really neat screen saver. People had may have had a dream of being the one to have their computer process a bundle
of radiowave data and have it be the first indication of intelegent life broadcasting their existance.
There are many other projects that would love for you to volunteer time on your computer for their cause. They are all running in parrallel and are doing what some people consider important work. Distributed computing is good for many types of computer processing. Things that can be broken down into individual steps such as seti's radio data or Computer graphics rendering. It is not an architeture generally favorable to Testing neuclear bombs or predicting weather patterns, but mathematicians can think of many uses.
The problem with the current volunteer assemblage method is that it is just a plain inefficient use of computing. Why run a project that has a finite runtime in parallel with others of finite runtime? If you work on them first come first serve you will spit out the answer to the first problem earlier. You may be just as likely to get that one done and out of the way before the next one is even started as to get in the next one's way at all. If they all run in parrallel some answers are needlessly slowed.
In distributed computing just like in Highlander "There can be only one." The whole purpose of computing is to get an answer fast. The faster you get your answer the faster you can move on the more valuable the system. Yet it relies on individual decisions to volunteer computer time. Your interest in volunteering is not the next person's interest in volunteering and probably never will be. I would probably be interested in getting money for my computer time and someone else might be interested in science and someone else might be interested in donating to a charity that has nothing to do with needing computer time. While I doubt there will be much more money in this than recycling alluminum cans, this would be big to the people who need computer time and people who have large processing capacity. and people who wish to trade long periods of computer time for short gluts of it.
People in general need to feel useful. There was a scandel with Seti at home. They were accused of running chunks of data multiple times and not bothering to run efficiently processing programs. It is perfectly reasonable to want to keep people busy rather than tell them that you haven't got enough work lined up to use them, if you don't have a marketplace and competition for computer time.
There are few types of programs that you would run this way:
Finite: like movie cells in which the runtime can be properly estimated and can be farmed out and processed in a given time
Finite ongoing: Like Seti while we may never find an answer to the question we only receive so much data to process
Undetermined: Programs for which the runtime has not yet been proven to be finite (turing halting problem)
known to be Infinite: Mathematical problems like finding the next prime number is a never ending quest and is perfect for distributed computing. It will always be necessary for coded transmissions and helps keep ahead of code cracking supercomputers.
Finite programs should probably be run one at a time and finished on demand, maximizing value and would best be suited for the early morning hours. Everything else should be run in parallel to get the job done (if possible). Individual project managers who have farmed out their computers while they are not in use will probably want to collect their runtime from the system while people are awake to use it. There will probably be a multi teired marked for different computer archatectures. Distributed computing may indeed get into weather prediction, though this would not be optimal for personal computer type of archetexure.
While all this somewhat complicated allocation process is going on You simply are in charge of your runtime. You can allocate your time earned to any project you desire and because your processor time is bought by some people it will have a value that can be donated to favorite charities. It probably won't be enought to make it economical to send you a check every month(you would probably be lucky to pay for your electricity) You could put your money into a non-taxed lottery of any denomination above the minimum check amount. I imagine if you were the one to fill up the amount to above $100 then you would be the one to get a $100 dollar check. Whatever floats your boat. The focus will be on meeting your demands and you will be given the option to veto the use of your computer for any give project, though this will poorly serve your favorite project by possibly reducing the value of the computing that you do. On the other hand you could just raise your price if you think they will pay and you want to stick it to a particular project. This will have to be allowed to keep people in the market.
All of this would probably be controled by the newly formed fifth branch of the military for which I will need help with a creative name. Due to the Prime number/ code breaking part of big computing this will be of strategic military significance and they will have a vital interest in soaking up loose computing time that could be used improperly, They will therefore work to acheive safe, efficient and continuous usage of all computer time in a way that accomodates a free and open market for all legal distributed computing aplications. They will probably have at least two programs of major significance to their aims; The next Prime and code breaking. Which as I said can be opted out of if you choose since their goal is congruent with the fact that "There can be only one".
prime numbers and coded transmissions
http://www.beyonddi...iew.page.asp?I=3428
something that distributed computing is addept at both cracking and making uncrackable codes. [MercuryNotMars, Feb 07 2007]
Current distributed computing projects
http://www.distribu....info/projects.html
[MercuryNotMars, Feb 07 2007]
The Fifth Column
http://en.wikipedia.org/wiki/Fifth_column
Now with your computer feeding it via a network - no thanks ! [xenzag, Feb 07 2007]
Acrimonious overtones
Right_20to_20agree
In case you missed my //seeds of Hegemony// and were wondering why all my ideas are bad. [MercuryNotMars, Feb 07 2007]
A summariser
http://www.senseproject.com/
A good tool for the loquacious [DenholmRicshaw, Feb 07 2007]
1.500+ Big Macs
http://www.apple.co...nce/profiles/colsa/
I want to see this re-built using valves. [xenzag, Feb 08 2007]
China-Tiawan Information warfare
http://www.global-d...00/pages/china.html
asymetric warfare [MercuryNotMars, Feb 10 2007]
Please log in.
If you're not logged in,
you can see what this page
looks like, but you will
not be able to add anything.
Destination URL.
E.g., https://www.coffee.com/
Description (displayed with the short name and URL.)
|
| |
People already are in charge of what they run on their computers, and choose to participate or not participate in distributed computing projects as they like. |
|
| |
I know, but they run in parallel defeating part of the purpose and promise of distributed computing. This is running what is useful on your computer and applying a credit to whatever you want. Did you catch that? |
|
| |
Read up on coding Jutta. It uses large primes that are hard to factor out. |
|
| |
Edit: This comment has been getting me some flak because it got left high and dry. There was more to Jutta's statement. I think she abandoned her position, I don't know for sure. Not my preferred method of retracting if that is the case but, I thought I would put it here because evidently No one reads everything! shock! Gasp, and bewilderment. |
|
| |
Come to think of it that might be why this is her method she chose; understandable. |
|
| |
That running in parallel defeats "part of the purpose and promise of distributed computing" is technically just plain wrong (i.e., that finite/infinite taxonomy you explain doesn't make sense to me), but I don't feel like arguing with you about that - it's material for a whole class. |
|
| |
There are algorithms that are harder to parallelize than others, but that distinction has nothing to do with whether or not the algorithm terminates. What you do with those algorithms is - you don't parallelize them. |
|
| |
I have studied program runtime, That is what computer scientist do. I think you have recently got into the habit of putting me down for no reason. |
|
| |
You have studied program runtime!
Well, then, that completely changes things. |
|
| |
I am glad you commented it does point out what I did a poor job of communicating. That helps. |
|
| |
//Read up on coding Jutta.//
Does anyone else think that's really funny? |
|
| |
A sort of commodity market for spare computer capacity is not a bad idea, though there might be problems making the commodity itself sufficiently uniform. (Mind you, I don't really know how the Chicago Board of Trade guarantees that one ton of May wheat is interchangeable with any other ton of May wheat, so maybe the problem is not that hard). |
|
| |
Putting spare computer capacity under the control of an anonymous branch of the U.S. Military is a less attractive idea, if you care about privacy, accountability, checks & balances, etc. |
|
| |
[jutta] is not infallible <looks nervously over shoulder/>, but you are advised to think carefully about what her point was before dismissing it. |
|
| |
//I think you have recently got into the
habit of putting me down for no
reason// - well I have reasons - your
last two postings have within them the
seeds of a hegemony. eg |
|
| |
//All of this would probably be
controled by the newly formed fifth
branch of the military// I have the
perfect name for this - The Fifth
Column. |
|
| |
At some point quit changing your posts and simply annotate. You leave my comments high and dry in that they are not responding to you. Running infinite programs in parrallel and running possibly infinite programs in parallel does not make them change. It does however assure that if there is a finite solution to one of them that it will be found in a finite amount of time. |
|
| |
Finding the next prime is a finite problem and an infinite one as well since primes are infinite. It has to have a low priority. It simply needs to stay ahead of the game slightly. It does not need to occupy every computer from now till the end of time. It can be relegated to simply a portion of the total computing power. Now that I have said this please do not erase your comment. |
|
| |
Finite runtime programs do not waste time with the doubt that it will never end. They are the bread and butter since they can attract investors with an actually realizable promise. Everything else can be suspended and you can hit the project hard and finish it. |
|
| |
My Foundations of CS III class taught me that if there was a finite solution to an atomata then it could be found by taking every path in parallel. You might get there quicker without going every path but you might go on indefinitely if you do not find it. Kind of like taking turns in an arguement. It is frustrating letting an idiot speak but if you get your turn you can feel like you are making progress. I am therefore dealing with possibly infinite problems like they are possibly infinite. This plan is for efficiency in the finite problems, and it is to increase market liquidity, Maximize utilization of your computer, Facilitate easy management by a responsible party to manage the resourced that you allocate them, all without scareing people off. |
|
| |
I have Identified the party who has a stake in facilitating this. |
|
| |
That is a pretty sweet name xenzag since I don't know what it is refering to at the moment, I'll research it. I was thinking along the likes of Air force only, computer force. This might be something like the Signal core |
|
| |
//Read up on coding Jutta// - I'd like to nominate that for a Halfbakery tagline |
|
| |
//Read up on coding jutta// was an actual response to something she said until she erased it. It was in context of prime numbers and RSC code breaking. (not writing in a computer language which is often referred to as coding) |
|
| |
I think I just figured out part of her confusion though. Running programs in parallel is running each program on less than all they computers available or in turns. I think she misunderstood that to mean running them on all the computers available at once till they are done(absurd of course and right to point out if that is what she thinks), kind of like "Parallel processing". I meant processing in parallel like multitasking on possibly infinite problems. |
|
| |
Yeah, Yeah! I have no idea what you just said but I am guessing this is probably covered under the second and third "fallacy of Distributed Computing". I did find 12864 processor as an idea that Vernon dreamed up. |
|
| |
I imagine packages will probably be processed for an hour and then lost packages will be redundantly processed in a second round to give an overnight answer to many problems that distributed can solve. This is probably about the limit of what you can expect as far as speed. A virtual processor sounds slow like virtual memory. Please explain if I have missed your point, I am guessing this is an inside joke though. |
|
| |
No matter how good or bad this idea is,
the part about putting a fifth column of
the military in charge of it attracts my
fishbone. |
|
| |
If I name it the Highlanders can I have a bun? Maybe the Imortals which references the highlander series and is a group of elite body guards that lost at Thermopoly. I know you like your country's military to lose so would that name work? Fifth collumn was the good guys in V. I was going to suggest the fifth array. |
|
| |
I am curious who you want to trust with that much processing power? I don't trust the UN. |
|
| |
[MercuryNotMars]: Sorry for not quite understanding, could you please, in a couple of sentences focus on what idea/invention you specifically propose? |
|
| |
I couldn't face reading this so I used a text summariser (link) to yield this... |
|
| |
[They were accused of running chunks of data multiple times and not bothering to run efficiently processing programs. |
|
| |
Individual project managers who have farmed out their computers while they are not in use will probably want to collect their runtime from the system while people are awake to use it. |
|
| |
On the other hand you could just raise your price if you think they will pay and you want to stick it to a particular project. |
|
| |
All of this would probably be controled by the newly formed fifth branch of the military for which I will need help with a creative name. |
|
| |
They will probably have at least two programs of major significance to their aims; The next Prime and code breaking. |
|
| |
Which as I said can be opted out of if you choose since their goal is congruent with the fact that "There can be only one".] |
|
| |
That's made the whole situation clear to me. |
|
| |
//I have Identified the party who has a stake in facilitating this.// |
|
| |
It makes sense that the U.S. government would be a significant buyer of this commoditized computing power. I don't think anyone has a problem with that. |
|
| |
However, if one of the major players in a market is also administering the market, there's a conflict of interest. For comparison, you might like to research the relationship between the bank JP Morgan and the eurobond clearing house Euroclear, which is kept carefully at arm's length, even though I think Morgan set up Euroclear in the first place. |
|
| |
Also, whenever you start extending the remit of the armed forces beyond its traditional scope, you are doing something which is characteristic of fascism. I'm not saying that you *are* a fascist, just that this would represent an incremental step in that direction. To understand how this can turn nasty, consider the example of Indonesia. (Although Indo is now becoming more democratic, the army is still reluctant to give up its business interests, and has sometimes seemed to act 'above the law' in defence of those interests). |
|
| |
The text summarizer missed a major theme: |
|
| |
"There can be only one" It somehow missed a theme repeated 3 times. |
|
| |
There can be only one distributed computer because it needs to focus to be able to complete projects as they become ready to run. |
|
| |
Currently there are many parallel projects This is fine if they are all ongoing projects, but they are not. |
|
| |
All computers together put out 1000 units of processing in 1 year, If we have 10 imaginary projects that require 100 units per project. Should we complete 1 every one tenth of a year or should we complete them all in one year? |
|
| |
There is a way to support your project even better cooperatively even if you don't use your computer to run your favoite project on it. I sumarize that approach. |
|
| |
One of the things The military will be buying is the proper use of Distributed computing. They will pay a premium, even subsudizing the system to accomodate their primacy as the provider of Distributed computing. They will be sitting on the hub of the internet. That is a unique commodity more akin to extortion, they will be forced into making it work properly. |
|
| |
They will have to eliminate competitors by allowing people freedom to not participate in objectionable projects. They are buying the non-use of this technology by hostle powers. I am sure they will be a buyer but may chose to pay more than market price simply because of what you have pointed out as a perceived conflict of interest. |
|
| |
If you have the marked controled you don't suddenly have people assembeling tons of computers for a one time hack job. |
|
| |
I am very opposed to Government control of any industry, I Think roads, schools, and charity should be privatized. This is a matter of protecting life and property. |
|
| |
I think this is kind of like calling a spade a spade, People are not going to keep their hands off this one there is going to be Government regulation. This is probably as good or better than a two teared Private system with Government second guessers over them. Regulators generally function by slowing instead of helping, and Being Military is a constant reminder of the need to contribute to the protection of life and property instead of regulating it out of your jurisdiction and pushing the center out of your grasp. |
|
| |
OK, it looks as though there are two distinguishable ideas emerging here. One is for a market in spare computing power, for reasons of efficiency, and the other is for U.S. Military control of spare computing power (or of the internet generally?), for reasons of security. |
|
| |
Regarding the first, I see a mismatch between the '1000 processing units' example you quoted to [DenholmRicshaw] and the market mechanism. |
|
| |
In an open market, I would expect the ten projects in your example to be completed at random intervals over the year, based on how much money their sponsors were willing to put into them at any given time. If the project sponsors had similar budgets and planning cycles, then the market mechanism would probably lead to all the projects being completed in a bunch at the end of the year, which is not the outcome you want. |
|
| |
On the other hand, if the regulator is routinely going to override the market mechanism in the interests of algorithmic efficiency, then it will be more difficult to make the system work as a market at all, and instead you will have a socialist-style 'command economy' (albeit only in this particular economic niche). This is not necessarily a fatal flaw in the idea; I accept that there may exist certain economic niches which might be better served by a more technocratic approach (rather than a classical market approach), and maybe this is one of them. However, you may want to re-name the idea in that case. |
|
| |
Regarding the second, the thing you are trying to prevent is //people assembeling tons of computers for a one time hack job//. Currently, people already assemble tons of computers for hack jobs as 'bot nets'. I don't see a way that this idea will prevent that. |
|
| |
Also, I don't see why the case you make for military control of spare computing power could not be extended to military control of other things. Schools and charities have been used extensively for terrorist ends (if you believe the news reports), and yet you don't want military control of these. Distributed computing has not been (yet), and yet you do want military control of that. |
|
| |
I'm finding it hard to warm to your ideas given the acrimonious tinge to your annos (less the case with this idea, mind you, aside from your sharp retort to [jutta]). When you're right, you don't need to be acrimonious; when you're wrong, you look sillier if you are. |
|
| |
As for the idea, what [pertinax] said, particularly his last paragraph. |
|
| |
[pertinax], you give me a great idea! What we should do is create malware that creates a back-door to (depending on your information source) between 30 and 80% of Windows running machines. BUT, instead of using them to nefarious means, we should use them to enslave the computers to run distributed computing projects, prioritising according to what ever whim dictates (technocratic, market drive, megalomaniac ambition, percieved benefit to mankind), ensuring that the computer is only used during screensaver mode. We should also find users bank details to reimburse them anonymously for their computer time, according to a rate set by our whim. We should then set up an organisation to administer the system, populated by idealists who are beyond corruption but don't mind invading people's privacy, are also well versed in evading detection by internet policing bodies, and can write ever more subtle and devious malware, probably mostly inspired by code already out there.
Tadaaaaaa! |
|
| |
So, am I correct in saying that the actual idea/invention (when distilled down) is to promote the distributed computing of a small number of projects at any one time, so that these problems may be focussed on? |
|
| |
That is one aspect Jinbish. |
|
| |
Pertanax, To finish a project will take approximatly as much computing power if you take turns or if you do not take turns. You can bid on whatever type of lumps you want. I am sure there will be a premium for that last bits of processor width and I am sure it will cost in loading time to involve more computers than necessary, but there is also a premium on time that is a marketable commodity If you go second or go at the same time as someone who is running a program parrallel to you and all you need is the final answer then you have no advantage you might as well let the other guy go first. This is no more a command economy that facilitates this aspect than any other type of bidding system. There is simply a premium in processing width and the last person in my example would have no loss of advantage in letting the other 9 go first. The ninth whould have no disadvantage in letting the other 8 go first and so on. People use third party sellers all the time and as I said they would have a vested interest in getting the best price for sellers because that is their job, to attract them to the market and keep them. |
|
| |
I know there are hackers with distributed computing systems at their disposal. They don't have what I am proposing. They might have one thousanth of the width of all distributed computing. Eventually one computer can hack any such code that is hackable meaning that a code is time sensitive information to begin with. The time sensitive nature of codes is all I am talking about here. There is a difference between 1/1000 and 1 when we are talking about time sensitive information. If there is a system that the military has that can exclude hostile interest's bids from the only distributed computing system then that is their incintive whether they personally use it or not. |
|
| |
They do not have the ability to instantly assemble computers for a project like they would if they could bid on a glut of processor width. We know who they are and we can check what they are hacking if we sign up and have the code already. |
|
| |
If you are looking for the command economy part of what I am proposing it will be in programming end of things. The algorithm will probably have to be packaged by one central authority to make it interchangeable. One standard encapsulation that probably will not be open source will be a prerequisite. but their success will still be measured in eliminating competition by dirrect competion(unless we find them hacking military codes which might involve bombs). |
|
| |
So what else is your idea/invention - is it a marketplace for computational resources? |
|
| |
There are some details of making that goal work. and there are some advantages besides time from start to stop but not much else. |
|
| |
I think that is the main point. Computer time is more valuable when organized in this manner. So I wanted to empasize stuff like that. |
|
| |
But seriously, why the military? Why not an independent regulatory commission, like the Atomic Energy Commission, the NYSE, or the electricity market? There are plenty of examples where a large market made up of the output of small(er), privately-owned enterprises, is commoditised and regulated. |
|
| |
Inferno OS would be ideal. |
|
| |
I know very little about actual programs, just algorithms. You might be right about inferno. |
|
| |
This is not just buying and selling things that have been bought and sold since forever, This is infastructure to build a market that will end up being regulated to peices. |
|
| |
The fact is that war is really something that should be planned ahead so you don't have to catch up later when you find you are vulnerable. China is putting a lot into it, and it is a vulnerability. |
|
| |
It is the fastest way to get it organized and serves many purposes to have military in it. If they have any kind of strategic planning it should be a model of public relations. It is a place to put and train experts on cryptology and viral warfare. It allows oversight and maintains the proper use of a powerful system. |
|
| |
I am surprised by some comments, it is like you only trust enemies with guns. This is a weapon that needs watched. I am curious who do you trust with this? Atomic energy commision is close, but no one is going to be testing neuclear bombs on this thing or fluid dynamics even weather. You can break down hacking into independantly processable components fairly easily. and the computers that can and do simulate bombs are probably already managed to the hilt. |
|
| |