marcell mars's Library tagged amazon → View Popular

Filter:
All
Annotated

05 Sep 09

Converting 11 million articles from TIFF to PDF-s on amazon EC2 & S3: Self-service, Prorated Super Computing Fun!

"I was ready to deploy Hadoop and my code on a cluster of EC2 machines. For deployment, I created a custom AMI (Amazon Machine Image) for EC2 that was based on a Xen image from my desktop machine. Using some simple Python scripts and the boto library, I booted four EC2 instances of my custom AMI. [..] thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3."

open.blogs.nytimes.com/...e-prorated-super-computing-fun - Preview

amazon distribution compute conversion programming linux java python storage

I was ready to deploy Hadoop and my code on a cluster of EC2 machines. For deployment, I created a custom AMI (Amazon Machine Image) for EC2 that was based on a Xen image from my desktop machine. Using some simple Python scripts and the boto library, I booted four EC2 instances of my custom AMI. I logged in, started Hadoop and submitted a test job to generate a couple thousands articles — and to my surprise it just worked.

I then began some rough calculations and determined that if I used only four machines, it could take some time to generate all 11 million article PDFs. But thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3.

13 Apr 09

tehdely: On Amazon Failure, Meta-Trolls, and Bantown

"It's obvious Amazon has some sort of automatic mechanism that marks a book as "adult" after too many people have complained about it. It's also obvious that there aren't too many people using this feature, as indicated by the easy availability (and search ranking) of pornography and sex toys and other seemingly "objectionable" materials, otherwise almost all of those items would have been flagged by this point. So somebody is going around and very deliberately flagging only LGBT(QQI)/feminist/survivor content on Amazon until it is unranked and becomes much more difficult to find. To the outside world, this looks like deliberate censorship on the part of Amazon, since Amazon operates the web application in question. To me, this looks like one of two things: 1. Some "Family"-type organization astroturfing Amazon in an attempt to rid the world of EVIL PRO-HOMOSEXUAL FILTH!! 2. Bantown"

tehdely.livejournal.com/88823.html - Preview

amazon posthuman politics distribution bizarre book collection search fem sex

21 Jan 09

Cheap, Easy Audio Transcription with Mechanical Turk - Waxy.org

Here's how to do it yourself, with no programming knowledge required. Mechanical Turk workers are all working in parallel, so the more discrete tasks, the faster the job gets done. This also diminishes the risk of one bad worker ruining your whole job. (Though you're always allowed to reject bad submissions, and you'll never have to pay for those.)

waxy.org/...scription_with_mechanical_turk - Preview

business distribution amazon audio webservice social

Here's how to do it yourself, with no programming knowledge required.
Here's how to do it yourself, with no programming knowledge required.
1 more annotations...
- Mechanical Turk workers are all working in parallel, so the more discrete tasks, the faster the job gets done. This also diminishes the risk of one bad worker ruining your whole job. (Though you're always allowed to reject bad submissions, and you'll never have to pay for those.)