Home
Joel Liu's Library tagged → View Popular
19 Jul 09
Cliff | Performance Followup from NoSQL
-
Jon Travis has already started a project for testing out Project Voldemort. It is called vpork. This is a step in the right direction. As a community we should come together and start talking about how we test performance because it is important. We owe it to our users and we owe it to ourselves to be honest about the capabilities of our projects. Only through our commitment to quality, even at a cost to our own egos, will we build great systems.
03 Jul 09
Gojko Adzic » QCon London 2009: Upgrading Twitter without service disruptions
-
A very interesting observation during the talk was that Twitter started up with a CMS model and that they gradually moved towards a messaging model. I’ve seen this in a few applications so far, including a casino system, where the messaging model seems to fit best an application intended to power massive community of online users, it seems regardless of what the application actually does business wise. Applications start out completely different, but then more and more functionality gets bolted on top of user messaging capabilities that the whole architecture on the end gets refactored to utilise the messaging channels as the core information transport. With Twitter, I’d expect this to be more obvious from the start as it was intended to help people notify each other.
-
The interesting thing, however, was that all the upgrades were done live, without shutting down the system. The changes were always introduced to one node, then regression issues were sorted out, and then the software would be rolled out to the whole cluster. They went as far as building a whole messaging system based on memcached APIs in order to be able to slot in such changes.
- 3 more annotations...
InfoQ: Twitter, an Evolving Architecture
-
Most of the tools used by Twitter are open source. The stack is made up of Rails for the front side, C, Scala and Java for the middle business layer, and MySQL for storing data. Everything is kept in RAM and the database is just a backup.
-
- 1 more annotations...
16 Feb 09
Search Y Combinator - Blog
-
Some have speculated that we've moved on. And while that's true to some extent, we are still actively maintaining this website as a service to the YC community. Call it a personality flaw, or whatever you want, but there's just something about letting a website die that doesn't sit well with me.
17 Dec 08
After Credentials
-
In a world of small companies, performance is all anyone cares about. People
hiring for a startup don't care whether you've even graduated from college, let
alone which one. All they care about is what you can do. Which is in fact all
that should matter, even in a large organization. The reason credentials have
such prestige is that for so long the large organizations in a society tended to
be the most powerful. But in the US at least they don't have the monopoly on
power they once did, precisely because they can't measure (and thus reward)
individual performance. Why spend twenty years climbing the corporate ladder
when you can get rewarded directly by the market?
23 Oct 08
Agile Testing: Performance vs. load vs. stress testing
-
However, testers also take a black-box approach in running the load tests against the system under test. For a Web application, testers will use tools that simulate concurrent users/HTTP connections and measure response times. Some lightweight open source tools I've used in the past for this purpose are ab, siege, httperf. A more heavyweight tool I haven't used yet is OpenSTA. I also haven't used The Grinder yet, but it is high on my TODO list.
05 Jun 08
Lukas Biewald » Scaling Fast
-
After working so hard to get users to come to your site, it’s amazingly frustrating to see hundreds of thousands of people suddenly locked out. Unbelievably, our webserver (nginx) couldn’t even reliably show that static page… Brendan discovered that we were exceeding the system’s open file limit — set at 100,000 — because connections were counting as open files.
02 Jun 08
Twitter Blog: It's Not Rocket Science, But It's Our Work
Q: Is that why most of your major outages can be traced to periods of time when [a system administrator] was there to sit and monitor the system?
A: There are a number of reasons for our past outages. Everything from faulty process, environment, configuration, and just plain load. Our system must be designed for peaks; currently we're tightly coupled which means that massive traffic on one part affects all. We're addressing this by breaking the stack into small lightweight pieces which are designed for failure.
06 Mar 08
Re: Mass Mailout Performance Tips
-
There is a qmail patch (big concurrency) that will allow running more
remotes. The default kernal in RedHat 6.1 limits you to 512 total processes
and half that per user. I have recompiled the kernel for 2560 total
processes and 2048 per user. See the file
/usr/src/linux/include/linux/tacks.h. -
I can get about 50K emails per hour using 400 remotes. That would take 10
hours with your list. The qmail queue size seems to stabilaze between 10K
and 15K during the run. There is usually between 300-400 remotes running. - 1 more annotations...
Selected Tags
Related Tags
Top Contributors
Groups interested in Performa...
Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »
Join Diigo