Encoding Text in Images
So I've been thinking lately about some free file storage and backup systems that could be used to backup my site's files and whatnot. Ah...the hacker in me is still alive. There's a pretty interesting solution I thought of that I would love to try out...but probably wont because it's not of much use to me.
Kia Kroas :=: 14 Mar, 2009 06:52:12 last updated: 17 Mar, 2009 09:47:27
1. The obvious
The obvious solution is to use the systems built especially for file backups and storage. Their fault tolerance and precautions are not something you get from a free ghetto service. But of course, the good ones aren't free. On the free ones, there are limits on file size and length of storage. Usually there's something in the FAQ about logging in every so and so, or at least one download every 30 to 60 days or else they kill the files. It doesn't make sense to me, and I really doubt they actually delete the files, but well, it's their business not mine.
[side note 1]
But you don't want arbitrary limits in a backup system. First, you don't want other people downloading your sensitive backups (if they're important, don't be so cheap, pay for the good stuff). Second, when disaster strikes, do you have time to login to the site to save your files?
If anyone reading this has a free file backup/storage site that keeps files indefinitely without any limits, please feel free to link me. It has to be viable though, I don't want to upload files and see the company die within two months.
2. Use free email systems.
Most people have heard about Gmail. With an offer of 7305MB as of this writing (and still counting) it's hard to not grab that storage potential. If you think about it, who in the right mind is going to get 7 GB of email in the first place? And in terms of time...I don't think there's enough time in a person's life to read all that. Maybe if there's a lot of attachments...and that's where the virtual drives come in. They let you use gmail for storage:
GMail Drive a shell extension for Windows and
GmailFS for Linux and
gDisk Mac OSX.
Note: Although Google doesn't explicitly ban this type of activity, it's still against their Terms of Service (access of service without using interface provided and using it in a manner not intended) so you risk losing your entire account.
[side note 2]
3. This is the fun one...
I'm probably not the first to think of this. I'm too lazy to go around searching for precedents, but I did think of this myself and I'm really proud of myself for this. It's a pretty simple concept. You change the file to something else. Kinda like cryptography and steganography, but not really encrypting or hiding it. Not just a simple rename (most upload sites have some sort of heuristic for that sort of thing) but encoding the data inside other filetypes, particularly images, video, and audio. That's the word, encoding.
3A. Change the file to an image
An image is a set of pixels on a grid. So right off I can think of two quick and easy solutions to encode data in images.
First for black and white. You can use white as a counter and black as a placeholder. So say you have a text file of the alphabet in uppercase. The first pixel is black. 'A' is ASCII 65, so that means you have a line of 65 white pixels and then another black dot. Then a line of 66 pixels (for 'B'), and another dot, etc etc.
But this would cause for a pretty big image. Since there are 255 shades of gray, you could just do each shade for each letter (this would mean you could only do ASCII, unless you roll out some wicked bit-encoding wizardry). Then each letter would be one pixel. But why stop there?
With the full rgb, you could encode 3 letters per pixel. Then, you can store a relatively large textfile as a small image:
100x100 = 10000 pixels (method 2) *3 = 30000
so a 30kb text file can be stored as a 100x100 image. The resulting filesize may (probably will) be larger than the initial text file. So, most likely this wont be a great compression system. Also, doing it pixel by pixel like this might lose some data depending on the program, filetype and compression algorithm. So you should check the decoding before believing that it's all safe.
There's no shortage of image hosting providers and I don't think they have a system out there to capture this type of "file smuggling". So this is going to be interesting.
There are also a number of good barcoding systems that could do something similar, and implementations of those are already out. The downside is, many of them are already patented.
[side note 3]
If you go with the barcode route, since most of them are black and white, what you can do is get an image and darken/lighten areas corresponding to the barcode's black/white regions. Then when you want to remake the image, compare the darkened/lightened image with the original. Kinda like XORing bytes.
3B. Change the file to Video
And, if you can do it with images...what's to stop you from doing it with video? When you really think about it, video is just a series of frames (images) put together in a timed series. So add in the list of video providers to image hosters and you're set for a very good run.
Keep in mind though, you may lose content on some of those video providers. They resize and export the video into flv format. To compensate, you might have to increase the blocks/size of pixels.
3C. Add in Audio
A series of beeps, bangs, booms, or pops at intervals that could be unencoded to the file.
Leaving message
Now I originally thought of this when doing file backups, but this could be useful to people trying to mangle information. People under corporate firewalls who want to send out information without it being blocked could find some use for this. Since there's been a lot of talk these days of government firewalls--it's not only China anymore--there may be more of a need for this later.
Encryption has the downside of requiring a key and/or knowing the algorithm. It requires effort and secret knowledge which makes it a barrier when trying to disseminate information to the masses quickly. So there's the conundrum of requiring knowledge of a secret to get knowledge of another secret. Encoding is less secretive and can be used to spread data through barriers quickly even if those firewalls are reading data. (They can't analyze every packet right?)
Of course, for people who don't want to go to the trouble of using a program to do the pixeling or barcoding, there's always the option of writing the text directly on the image and saving. Any decent image editor nowadays allows the writing of text onto images, so there you go...might not be so easy to change it back to text automatically though.
Side Note 1: It doesn't make sense to me infrastructure-wise why they delete files. I'm almost 100% sure most large-scale sites (Facebook, Myspace, Flickr, Google, Youtube, and Yahoo, etc) don't really "delete" their stuff when you click the delete button/link. Most filesystems don't do a complete delete. Sometimes they save it in the Recycle bin or .trash folder, but that's not what I'm talking about. When you actually delete the file, the Operating System just deletes the filename, its link to the data blocks, sets those blocks as free for reuse/rewrite. Everybody who has ever played with filesystems knows it. (However, I have seen some that do a scramble+null overwrite of the data just to be overly paranoid.)
Facebook ran into some trouble back in 2008 about not actually deleting data, but only removing the database link to the data. Facebook and MySpace have too many users and the aggregate data for each user is relatively small, mostly stuck in individual (small) database rows. They're not going to go to the trouble of deleting every row. They just mark the user as "gone" so all references to the user id are ignored.
side side note, if you want to really delete the facebook account instead of just deactivating here's the link. It's a full delete. All photos, comments, videos, messages, items, etc so be sure if you actually want to do it. Of course, the info used by user-contributed facebook apps are a different story; facebook doesn't control the data those guys already have. http://www.facebook.com/help/contact.php?show_form=delete_account
Flickr, Google, Youtube and Yahoo have enormous servers with over hundreds of terabytes of storage. They wont go to the trouble of lagging their servers with a search and delete operation to free up a few megabytes of data. Plus, if they do that, they'll start chunking their storage space. I don't think that's what they want. Any overhead at all, no matter how small, is bad. GMail and Yahoo Mail (I think) use some sort of Bayesian filter for spam. They'll have those spams marked and indexed whether deleted or not.
So really, unless the online storage systems have really high turn over, then I don't see why they would delete it. Maybe they have some system to handle that sort of thing. Or maybe it's just to remove the info from their database for faster search and retrieval? If so, it means they have bad search and retrieval systems. But meh, that's just what I think. I could be wrong.
Dang...that was a long side note.
Side note 2: But of course, one of the disadvantages of being a big company is that there are a lot of users. So there's no reason they would go to the overhead of checking every single user (or even the majority). There have been a lot of users that tried this without any problem I'm sure. As long as you don't trip the overuse/overload sensors, you should be fine.
Then again, I have heard of an incident where a program of this type went rogue and had a backdoor that sent out the passwords to some email address. I'm not sure if it's any of those two that I linked up there, but still...I wouldn't be giving away my password to some program I didn't code myself. It's your password, not mine. (And if by some odd chance we had the same password, well, it's your account not mine.)
Most of these aren't really made for lots of data though. The data matrix implementations would work if you chunk it. Check out these established systems:
Code 93 can encode only 26 uppercase, 10 digit and 7 symbols. For the Perl coders out there, there's GD::Barcode::Code93
Code 128 can encode the full ASCII 128. There's also a CPAN module for this one, Barcode::Code128
Data Matrix and its buddies
Aztec Code and
QR Code
And here's a quick introduction to a bunch more from
TALtech.