This link has been bookmarked by 28 people . It was first bookmarked on 26 Aug 2008, by Tristan Rivoallan.
-
18 Dec 09
-
04 May 09
Tim Lossen"Over the course of building out a number of large, distributed systems I've had the opportunity to observe (and implement) some worst practices. Most of these worst practices start innocently enough, but if neglected they will jeopardize the growth and s
-
17 Apr 09
-
21 Mar 09
-
29 Oct 08
-
24 Oct 08
-
06 Sep 08
-
Scalability Worst Practices
-
Introduction
-
if all you have is a hammer, everything looks like a nail.
-
common solution to the problem of persisting key-value pairs is to use a database.
-
The problems arise when the features of a database (relational integrity, locking, joins and schemas) become bottlenecks or inhibit scaling
-
As the access rate of the key-value store increases, the concurrency model of a database can begin to degrade performance while the advanced features that the database provides are not used.
-
Another common Hammer is always using threads to program for concurrency. While threads do address concurrency, they come at the cost of increased code complexity and an inherent failure for real composability with the locking and access models currently available. Thousands of lines of code contain race conditions, potential deadlocks and inconsistent data access management because the most popular programming languages today manage concurrency with threads. There is a growing community demonstrating alternatives methods for concurrency without the scaling issues of threads, namely the concurrency models promoted by Erlang or Stackless Python. Even if alternative languages are not chosen for production code, studying them for their concepts, such as message passing or asynchronous I/O, is good practice.
-
Resource Abuse
-
Not everyone, however, is as adept understanding the constraints of a larger system such as identifying the performance characteristics of a shared resource, knowing the clients of services or understanding database access patterns.
-
However, an often overlooked aspect of this build-out, in my experience, is not addressing the shared resources that are utilized by the additional services.
-
The use of the pool is a good practice and one which helps shield the database from excessive connection management
-
Continually increasing the number of services while not decreasing the maximum pool sizes
-
ncreasing the individual pool sizes without decreasing the number of services
-
Big Ball of Mud
-
- Compile the entire codebase together
- Pick and choose components and services based on known versions
- Publish models and services comprised of only backwards-compatible changes
-
Adopting a service architecture with backwards-compatible changes ensures the dependencies issues are minimized.
-
Everything or Something
-
The Hero Pattern can work in a small environment when one individual has the talent and capacity to understand an entire system, including the many nuances required to keep it functioning properly. For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions.
-
However, it also helps simply to rotate individuals from team to team if the organization has the capacity. In banking, it's sometimes mandatory to take vacation so any "I'll just run this from my workstation" activities can be brought to light.
-
As described above, the activities of the Hero are ripe for automation to ensure that it is possible for one person (or many people) to replace another with relative ease. An alternative approach to automation is additional process -- Clay Shirky offers an entertaining definition of process:
-
Process is an embedded reaction to prior stupidity.
Prior stupidity is inevitable – automation should capture the education.
-
Monitoring, like testing, is often one of the first items sacrificed when time is tight. Sometimes when I ask specific questions about the runtime characteristics of a component, no answer exists. The lack of visibility into the internals of a running system and the ability to quickly dive into answers to such questions jeopardizes the ability to make accurate and critical decisions about where to look and what to look for.
-
-
03 Sep 08
Marcos PereiraOver the course of building out a number of large, distributed systems I've had the opportunity to observe (and implement) some worst practices. Most of these worst practices start innocently enough, but if neglected they will jeopardize the growth and sc
-
27 Aug 08
-
26 Aug 08
-
Tristan RivoallanA system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome. It's important that a particular build, deployment or environment be reproducible and automation with explicit metadata is a key to successful reproducibility.
-
A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome. It's important that a particular build, deployment or environment be reproducible and automation with explicit metadata is a key to successful reproducibility.
-
-
25 Aug 08
Drew SudellCollection of Scalability anit-patterns based on Brian Zimmer's experiences at Orbitz
architecture scalability antipatterns for:trumant for:cherryr
-
Alan DeanOver the course of building out a number of large, distributed systems I've had the opportunity to observe (and implement) some worst practices. Most of these worst practices start innocently enough, but if neglected they will jeopardize the growth and sc
-
jordimOver the course of building out a number of large, distributed systems I've had the opportunity to observe (and implement) some worst practices. Most of these worst practices start innocently enough, but if neglected they will jeopardize the growth and sc
-
22 Aug 08
-
19 Aug 08
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.