Ruby on Rails, Paperclip, Uploadify and S3
I've been doing lots of work with Heroku. Since Heroku only deals with with application side and the filesystem is treated as though it were read-only, file storing has been done through Amazon S3. thoughtbot's Paperclip is being used to make file maintenance easier.
However, since clients nowadays want the fanciness of Web 2.0, the uploads were to have progress bars. So, because the sites were using jQuery the applications ended up being integrated with Uploadify.
Kia Kroas :=: 16 Mar, 2010 04:06:47
I'm under NDA and confidentiality agreements so I won't give any details about the applications themselves. But, here's how to integrate Ruby on Rails with Paper clip with Uploadify with Amazon S3.
Now keep in mind I'm not going to do a step by step tutorial on how to get it to work--I don't have the time for that and this blog isn't exactly targetted toward the beginner programming class anyway. (I don't even have a target audience, this is mostly just for me. If anyone wants to read it, great, if not no loss.) I'll just give you the steps to handle yourself.
Steps (in no particular order):
- Set up Rails, jQuery, Uploadify, S3 Bucket, etc. Duh!
- (optional) Install and set up Paperclip.
- Generate an upload signature for use with Uploadify.
- Set Uploadify with options to work with S3.
- Set up a callback action to listen to the return from S3.
(optional) Setting up Paperclip. If you want to do image manipulations, you'll most likely want to just upload the image and use paperclip's S3 storage system to handle everything. These links may help:
http://timmyc.posterous.com/uploadify-on-rails-with-paperclip
http://jimneath.org/2008/05/15/swfupload-paperclip-and-ruby-on-rails/
Generate an upload signature for use with Uploadify
S3 uses access control policies to handle uploads. Since generally you want your users to upload to your bucket, you'll be doing this over HTTP or HTTPS. S3 has an access control signing system to allow uploads by the public; they call it
Query String Authentication. The easy way to get around this is to just set the bucket you want to be public write, but there's lots of security issues that come out of that, so we'll generate a signature.
def s3_upload
# s3.yml is a file in the config folder, it's formatted for paperclip
s3config = YAML.load_file("#{RAILS_ROOT}/config/s3.yml")
bucket = s3config['production']['bucket']
access_key = s3config['production']['access_key_id']
secret = s3config['production']['secret_access_key']
key = "uploads/"
expiration = 10.hours.from_now.utc.strftime('%Y-%m-%dT%H:%M:%S.000Z')
max_filesize = 30.megabytes
sas = '201' # Tells amazon to redirect after success instead of returning xml
policy = Base64.encode64(
"{'expiration': '#{expiration}',
'conditions': [
{'bucket': '#{bucket}'},
['starts-with', '$key', '#{key}'],
{'success_action_status': '#{sas}'},
['content-length-range', 1, #{max_filesize}],
['starts-with', '$filename', ''],
['starts-with', '$folder', ''],
['starts-with', '$fileext', '']
]}
").gsub(/\n|\r/, '')
signature = Base64.encode64(OpenSSL::HMAC.digest(OpenSSL::Digest::Digest.new('sha1'), secret, policy)).gsub(/\n| |\r/, '')
{:access_key => access_key, :key => key, :policy => policy, :signature => signature, :sas => sas, :bucket => bucket}
end
With this, the bucket stays privately writable, but users can upload if the key starts with 'uploads/'. So if you want to see all user uploads, look for keys with that as a prefix. Notice that it returns a hash. I put it in the application controller as a utility function that's used like so:
uploadify = s3_upload
@javascript = "var S3_BUCKET='http://#{uploadify[:bucket]}.s3.amazonaws.com/';\n
var UPLOADIFY = { 'AWSAccessKeyId' : encodeURIComponent(encodeURIComponent('#{uploadify[:access_key]}')),
'key': encodeURIComponent(encodeURIComponent('#{uploadify[:key]}${filename}')),
'policy': encodeURIComponent(encodeURIComponent('#{uploadify[:policy]}')),
'signature': encodeURIComponent(encodeURIComponent('#{uploadify[:signature]}')),
'success_action_status': '#{uploadify[:sas]}',
'folder': '',
'Filename': '' };"
Then I output @javascript into the page that needs the uploadify, usually in the top header. The two javascript variables become global constants, so I can call them from external js files since with jQuery I generally have external javascript files hooked with $(document).ready() rather than inline js. Next, using these options in uploadify.
Set Uploadify with options to work with S3
$("#uploadify_source").uploadify({
'uploader' : '/flash/uploadify.swf',
'buttonImg' : '/images/upload.png',
'cancelImg' : '/images/cancel.png',
'script' : S3_BUCKET,
'fileDataName': 'file',
'scriptData' : TRACK_UPLOADIFY,
'fileDesc' : 'Music',
'fileExt' : '*.mp3;', //list of uploadable filetypes
'onComplete' : uploadify_complete,
'onError': uploadify_error,
'scriptAccess' : 'always',
'queueID' : 'uploadifyQueue',
'auto' : true,
'folder' : '',
'sizeLimit' : 31457280,
'multi' : true
});
function uploadify_error(event, queueID, fileObj, errorObj){
if(errorObj.info == 201){
$.post(CALLBACK_URL, {'_method': 'put', 'file[size]': fileObj.size, 'file[name]': fileObj.name}, function(data){
// do stuff with data
$('#uploadify_source').uploadifyCancel(queueID);
}, 'json');
}
return true;
}
function uploadify_complete(event, queueID, fileObj, response){
$.post(CALLBACK_URL, {'_method': 'put', 'file[size]': fileObj.size, 'file[name]': fileObj.name}, function(data){
// do stuff with data
}, 'json');
}
We need to catch errors because on some platforms, Flash reads a 201 as an error, and Uploadify treats it as an error. The $.post call is sent to the CALLBACK_URL, which I didn't set in the example, but you should when you do. Next up, setting the callback action.
Set up a callback action to listen to the return from S3
Ok, so since we're uploading the file over to S3, our server doesn't even know the file exists. We'll need a way to tell the webserver and database that the file is there. That's what the $.post() call earlier was for. My callback url was just a create action.
def create
@file = SomeModel.new(params[:file])
if @file.save
# render :json => {:success => :data}
else
# render :json => {:failure => :data}
# redirect_to ...
end
end
That's it. Good luck trying to find a more comprehensive article!
Uploadify forum on the topic (This was the only thing I had to work with when I was trying to do it.)
rails-uploadify-s3 example Found it on a quick google for the forum link above. Haven't looked at it, but it looks like an example to get you started.