Cyclic redundancy checks

Re: http://arstechnica.com/news.ars/post/20080312-study-amount-of-digital-info-global-storage-capacity.html

I wanted to just link it up in del.icio.us but hey this is something I’m passionate about and I’ve been wanting to write on this for quite a long time.

Yes, the future of the internet is filesharing. This was the utopian ideal that was never even dreamt of when RAID systems continually get optimised and web technologies become riches and software becomes a service, thanks to extensible AJAX libraries.

And I’m not denying that redundancy is key to information integrity on the internet. This new paradigm is nothing like our oldest word-of-mouth modes of information transmission. Previously, age-old information was preserved in some way by generational story-telling, if you will. The problem? Baudrillard’s nightmare. Languages evolved. Languages died. Story-tellers forgot some details, exaggerated others. And that’s why we have at least 3 versions of a certain epic that began maybe 4000years ago.
But now, we have the unprecedented ability to not only preserve the information for eons without any changes [I heart thee PDF], we can now accomodate all the modifications that would have been present traditionally too. Of course it can be argued that these mods will be more sterile, less organic than previously. It’s a tradeoff. I’m ok with that. The new data doesn’t decay, doesn’t fall apart with age. And if it can, there’s always backups! [If you don’t backup regularly, shame on you. Leopard and Time Machine ftw.]

But I’ve always been paranoid about machine limitations. My laptop’s always on power-save even on a fresh charge. My priority in choosing electronics is always battery life, hence no PSP or iPods. Hard-drive is meticulously maintained, clutter is promptly organised and unnecessary files are deleted from the main drive and backed up religiously, even though half the HDD is free. Stuff like that. So I have always been a bit critical about the new generation of webspam.

I don’t want to judge [alone?], but how many of you feel that web content proliferation has taken a turn for the worse in this Youtube/Facebook generation? I mean, for every good video on the tubes, there are hundreds of lame random crap that people upload for almost no reason other than to exercise the right to upload. Are you with me? For every cool informative page on the internet, there are millions of parked domains, millions of pages with nothing but adverts in them, tonnes of pornographic content, and of course those oh-so-annoying-popups-thank-god-for-Opera-&-Firefox. For every important email, tens of hundreds of spam mails and chain letters [those should count as spam too!]. For every internet service, tens of phishing sites. For every essential download, hundreds of viruses, trojans, worms. For every legitimate server request, unlimited potential for getting hacked, for getting DOD ping of death attacks. For every wonderfully elegant web interface, hundreds of wannabe flashy [pun intended] ones, hundreds of badly coded pages. All these are real flaws, eyesores and they take up space. Mucho space senor. It’s a cumulative disaster.

And you wonder. I really don’t want to judge. One man’s meat is another’s poison. What I think is crappy information may be entertainment at least or even essential to other people. Right?
A case study. I don’t want to seem insensitive here. My friend had a friend who passed away [rest easy brother]. And to hear it from my friend, this person, let’s call him R, sounds like a great guy. A model [participated in Mister World], a national-level dragonboater, a accomplished musician [he even composed the OST for a local production], and a game sprite designer [with a trademarked charcter that has its own merchandise] and only 20: wow. This guy had a string of achievements. But my friend and I had a disagreement about whether R deserves a Wikipedia page. Irony is, I refer to it as we speak, to list his details. Yes he was a great guy, but is popularity the key to web celebrity? I’d hoped at least in an intellectual interactive medium like the Internet, intelligence would have free reign.
Yes, R was a great guy and I respect his credentials, but this biography wasn’t on his homepage, which is still ok in my book.
It’s on Wikipedia for crying out loud. My impression of Wikipedia is the professional version of E2. A low cost, vastly scalable, instantly and universally accessible repository of information should traditional sources fail. In the case of a catastrophic event that only Wikipedia is available for information, would anyone access R’s biography? Did R change the world, or will his details help to change the world? And given my paranoia, there’s R and he has a page, and that’s one unnecessary Wikipedia page I actually *know* about. What about the unknown legion that I don’t know about? Porn star biographies, the category of “exploding animals” among others? Seriously.
Again, no disrespect meant. I come from a purely logical standpoint.

And yes, the irony is not lost on me; that it is this freedom of the internet that has allowed gems that would’ve otherwise gone unnoticed or inaccessible to be available to the public. I still don’t agree that all and any content on the internet is justified of course. Maybe some self-censorship can be carried out, people? If you have a message, by all means, carry on. If it’s lame, and you know it’s lame [like game replays and analysis of accidental frags…] then don’t post it up. Share it among yourselves, torrent it even, but don’t upload it on public servers. Is that too much to ask?

But it’s not enough. More governments continue to move their records to secure intranets. More corporations launch a staggeing number of online services each year. The amount of social data that’s going to be floating around is unimaginable. Consider the multiplier effect when the nature of these online transactions transcend the simple one to one functions and encompass more than one person’s details.

The Net *must* evolve. The software is immensely scalable thanks to LAMP, but the hardware WILL run into problems. So what can we do? Definitely a paradigm shift in information storage is in order. Tapes and magnetic drives have come in handy, but they’re old school. I’m NOT talking about alternative storage. We don’t want to have to deal with tracks, and pits and sectors anymore. They corrupt relatively easily [hence the title] no thanks to mechanical constraints of the HDD/OD technolog. The head is so close to the surface of the platter, it’s a wonder they don’t touch. A fingerprint residue is at least five times thicker than the gap between head and platter in a HD.

Well these are a couple of new technologies I know about offhand. There’s holographic data storage, where highly polished crystals are “burnt” with lasers along different crystallographic planes as a medium of data storage. It’s condensed, it’s highly reliable [can’t get scratched], pretty hard to destroy and pretty damn cool. Imagine your wedding ring also has your unique signature embedded in it. The cool part about this tech is that you can burn different data on the same crstal in the same region, but along different planes. This is rewrite capability, this is layer capability without having to turn over the crystal.

Then there’s microdots. Not very practical for online services definitely, but good for static information storage. You know, things like….um, I can’t think of anything that doesn’t change. Ok fine, microdots are pretty stupid unless you’re watching an ALIAS rehash.

But ultimately, we need look no further to find a information storage system than within ourselves. Not our brains, mind you. These special things have stored millenia worth of information, cumulatively even. And they’re capable of modification and self-correction as well, while preserving crucial data. They even destroy rogues amongst their population in a fully regulated process. I refer to the humble wunder-molecule: DNA.
Like seriously. It’s being done you know.
What do we have here? Bacterial DNA modified to include engineering bits within its genome, that don’t change AND get reproduced every generation without taking too much space [heh].
Why is this cool? Firstly, you must understand that in the genome, the DNA strands do code for proteins, but the bits that code for proteins are few and far apart, and there’s a LOT of junk in between. This arguably minimises the risk of mutation in essential regions by providing infinitely many other possibilities. So we’re really not going to hurt the bacterium by arbitrarily changing the biologically inactive regions. Not only does the data become hhighly redundant, the information will FIGHT to survive and stay intact. Now that’s the best kinda storage solution. The kind that won’t get deleted without a fight to the bitter end.
And the more codons we use per bit, the better the error correction will be when we finally do “read” the data. Of course now we have to come up with a way to make he information easily re-accessible, without going through 3billion bacteria and counting to get last semester’s lab report out.

Now the speculative part. We can now use DNA to store our information and we know crystals can too. And recently there’s the talk of engineered biocrystals using complementary DNA strands, one strand of which has extra codons for linking up with other strands. Hmmmm.

Infant tech, will take time to mature.
So until then, stop spamming in the ph4t p1p3s n00bs!

Advertisements

About this entry