This link has been bookmarked by 184 people . It was first bookmarked on 15 Sep 2007, by CK Lee.
-
28 Mar 16
-
11 Dec 15
-
03 Sep 15
-
28 Aug 15
-
The big architectural change that Amazon made was to move from a two-tier monolith to a fully-distributed, decentralized, services platform serving many different applications.
-
The databases were split into small parts and around each part and created a services interface that was the only way to access the data.
-
Their architecture is loosely coupled and built around services. A service-oriented architecture gave them the isolation that would allow building many software components rapidly and independently.
-
Grew into hundreds of services and a number of application servers that aggregate the information from the services. The application that renders the Amazon.com Web pages is one such application server. So are the applications that serve the Web-services interface, the customer service application, and the seller interface.
-
Amazon wanted to build an open community around their services. Web services were chosed because it's simple. But hat's only on the perimeter. Internally it's a service oriented architecture. You can only access the data via the interface. It's described in WSDL, but they use their own encapsulation and transport mechanisms.
-
When a customer submits an order you favor consistency because several services--credit card processing, shipping and handling, reporting--are simultaneously accessing the data.
-
Switch to a deep services-based architecture (http://webservices.sys-con.com/read/262024.htm).
-
-
12 Aug 15
-
15 May 15
-
Teams are Small and are Organized Around Services
- Services are the independent units delivering functionality within Amazon. It's also how Amazon is organized internally in terms of teams.
- If you have a new business idea or problem you want to solve you form a team. Limit the team to 8-10 people because communication hard. They are called two pizza teams. The number of people you can feed off two pizzas.
- Teams are small. They are assigned authority and empowered to solve a problem as a service in anyway they see fit.
- As an example, they created a team to find phrases within a book that are unique to the text. This team built a separate service interface for that feature and they had authority to do what they needed.
- Extensive A/B testing is used to integrate a new service . They see what the impact is and take extensive measurements. -
State Management is the Core Problem for Large Scale Systems
- Internally they can deliver infinite storage.
- Not all that many operations are stateful. Checkout steps are stateful.
- Most recent clicked web page service has recommendations based on session IDs.
- They keep track of everything anyway so it's not a matter of keeping state. There's little separate state that needs to be kept for a session. The services will already be keeping the information so you just use the services.
-
-
17 Feb 15
euler eulerExplication architecture géant du web
-
02 Sep 14
-
Amazon Architecture
-
-
-
Amazon Architecture
-
People's side projects, the one's they follow because they are interested, are often ones where you get the most value and innovation. Never underestimate the power of wandering where you are most interested
-
-
28 Aug 14
yngwiemHigh Scalability - database
-
27 Aug 14
-
11 Aug 14
-
14 May 14
-
23 Jan 14
-
A service-oriented architecture gave them the isolation that would allow building many software components rapidly and independently.
-
Not stuck with one particular approach. Some places they use jboss/java, but they use only servlets, not the rest of the J2EE stack.
-
The SOAP web stack seems to want to solve all the same distributed systems problems all over again.
-
Offer both SOAP and REST web services. 30% use SOAP. These tend to be Java and .NET users and use WSDL files to generate remote object interfaces. 70% use REST. These tend to be PHP or PERL users.
-
- If you have a new business idea or problem you want to solve you form a team. Limit the team to 8-10 people because communication hard. They are called two pizza teams. The number of people you can feed off two pizzas.
-
Force developers to focus on value delivered to the customer instead of building technology first and then figuring how to use it.
-
- Start with a press release of what features the user will see and work backwards to check that you are building something valuable.
-
End up with a design that is as minimal as possible. Simplicity is the key if you really want to build large distributed systems.
-
Eric Brewer's CAP Theorem or the Three properties of Systems
-
- Three properties of a system: consistency, availability, tolerance to network partitions.
-
- You can have at most two of these three properties for any shared-data system.
-
- To scale you have to partition, so you are left with choosing either high consistency or high availability for a particular system. You must find the right overlap of availability and consistency.
-
-
22 Oct 13
-
28 Aug 13
-
10 Jun 13
-
09 Jun 13
-
18 Nov 12
-
04 Sep 12
-
23 May 12
-
14 May 12
-
12 Apr 12
-
12 Nov 11
-
Amazon doesn't like middleware because it tends to be framework and not a tool. If you use a middleware package you get lock-in around the software patterns they have chosen. You'll only be able to use their software. So if you want to use different packages you won't be able to. You're stuck. One event loop for messaging, data persistence,
AJAX, etc. Too complex. If middleware was available in smaller components, more as a tool than a framework, they would be more interested. -
Offer both SOAP and REST web services. 30% use SOAP. These tend to be Java and .NET users and use WSDL files to generate remote object interfaces. 70% use REST. These tend to be PHP or PERL users.
-
Web services were chosed because it's simple
-
Work From the Customer Backwards to Verify a New Service is Worth Doing
- Work from the customer backward. Focus on value you want to deliver
for the customer.
- Force developers to focus on value delivered to the customer instead of building technology first and then figuring how to use it.
- Start with a press release of what features the user will see and work backwards to check that you are building something valuable.
- End up with a design that is as minimal as possible. Simplicity is the key if you really want to build large distributed systems. -
State Management is the Core Problem for Large Scale Systems
-
Eric Brewer's CAP Theorem or the Three properties of Systems
-
probabilistic sense
-
Only way to manage as large distributed system is to keep things as simple as possible.
-
Use measurement and objective debate to separate the good from the bad
-
Avinash Kaushik calls this getting rid of the influence of the HiPPO's, the highest paid people in the room
-
Innovation can only come from the bottom. Those closest to the problem are in the best position to solve it. any organization that depends on innovation must embrace chaos. Loyalty and obedience are not your tools.
-
-
02 Oct 11
-
Get big fast. The big guys like Barnes and Nobel are on your tail. Amazon wasn't even the first, second, or even third book store on the web, but their vision and drive won out in the end.
-
-
27 Sep 11
-
06 Jul 11
-
13 Feb 11
-
What is it that we really mean by scalability? A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added. Increasing performance in general means serving more units of work, but it can also be to handle larger units of work, such as when datasets grow.
-
Internally it's a service oriented architecture. You can only access the data via the interface
-
Services are the independent units delivering functionality within Amazon. It's also how Amazon is organized internally in terms of teams.
-
As an example, they created a team to find phrases within a book that are unique to the text. This team built a separate service interface for that feature and they had authority to do what they needed.
-
Force developers to focus on value delivered to the customer instead of building technology first and then figuring how to use it.
-
Start with a press release of what features the user will see and work backwards to check that you are building something valuable.
- End up with a design that is as minimal as possible. Simplicity is the key if you really want to build large distributed systems. -
Eric Brewer's CAP Theorem or the Three properties of Systems
- Three properties of a system: consistency, availability, tolerance to network partitions.
- You can have at most two of these three properties for any shared-data system.
- Partitionability: divide nodes into small groups that can see other groups, but they can't see everyone.
- Consistency: write a value and then you read the value you get the same value back. In a partitioned system there are windows where that's not true.
- Availability: may not always be able to write or read. The system will say you can't write because it wants to keep the system consistent. -
For the checkout process you always want to honor requests to add items to a shopping cart because it's revenue producing. In this case you choose high availability. Errors are hidden from the customer and sorted out later.
- When a customer submits an order you favor consistency because several services--credit card processing, shipping and handling, reporting--are simultaneously accessing the data.
-
For example, go more with a fast reboot and fast recover approach. With a decent spread of data and services you might get close to 100%. Create self-healing, self-organizing lights out operations.
-
A service oriented architecture allows the creation of a parallel and isolated development process that scales feature development to match your growth.
-
Open up you system with APIs and you'll create an ecosystem around your application.
-
Only way to manage as large distributed system is to keep things as simple as possible. Keep things simple by making sure there are no hidden requirements and hidden dependencies in the design. Cut technology to the minimum you need to solve the problem you have. It doesn't help the company to create artificial and unneeded layers of complexity.
-
Avinash Kaushik calls this getting rid of the influence of the HiPPO's, the highest paid people in the room. This is done with techniques like A/B testing and Web Analytics. If you have a question about what you should do code it up, let people use it, and see which alternative gives you the results you want.
-
People's side projects, the one's they follow because they are interested, are often ones where you get the most value and innovation. Never underestimate the power of wandering where you are most interested.
-
Look for three things in interviews: enthusiasm, creativity, competence. The single biggest predictor of success at Amazon.com was enthusiasm.
-
Hire a Bob. Someone who knows their stuff, has incredible debugging skills and system knowledge, and most importantly, has the stones to tackle the worst high pressure problems imaginable by just leaping in.
-
Innovation can only come from the bottom. Those closest to the problem are in the best position to solve it. any organization that depends on innovation must embrace chao
-
Creativity must flow from everywhere.
-
You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service.
-
- Developers should spend some time with customer service every two years. Their they'll actually listen to customer service calls, answer customer service e-mails, and really understand the impact of the kinds of things they do as technologists.
- Use a "voice of the customer," which is a realistic story from a customer about some specific part of your site's experience. This helps managers and engineers connect with the fact that we build these technologies for real people. Customer service statistics are an early indicator if you are doing something wrong, or what the real pain points are for your customers.
- Infrastructure for Amazon, like for Google, is a huge competitive advantage. They can build very complex applications out of primitive services that are by themselves relatively simple. They can scale their operation independently, maintain unparalleled system availability, and introduce new services quickly without the need for massive reconfiguration.
- Developers should spend some time with customer service every two years. Their they'll actually listen to customer service calls, answer customer service e-mails, and really understand the impact of the kinds of things they do as technologists.
-
-
19 Jan 11
-
03 Nov 10
-
Between 100-150 services are accessed to build a page.
-
two-tier monolith to a fully-distributed, decentralized, services platform
-
The databases were split into small parts and around each part and created a services interface that was the only way to access the data.
-
The databases became a shared resource that made it hard to scale-out the overall business
-
The application that renders the Amazon.com Web pages is one such application server. So are the applications that serve the Web-services interface, the customer service application, and the seller interface.
-
- framework, they would be more interested.
- The SOAP web stack seems to want to solve all the same distributed systems problems all over again.
-
Offer both SOAP and REST web services. 30% use SOAP. These tend to be Java and .NET users and use WSDL files to generate remote object interfaces. 70% use REST. These tend to be PHP or PERL users.
-
-
23 Aug 10
-
23 Apr 10
-
12 Mar 10
-
08 Mar 10
-
10 Feb 10
-
12 Oct 09
Tim Lossen"Their architecture is loosely coupled and built around services. A service-oriented architecture gave them the isolation that would allow building many software components rapidly and independently. -- Grew into hundreds of services and a number of appli
-
05 Oct 09
-
09 Sep 09
-
08 Sep 09
-
14 Aug 09
-
30 Jun 09
-
16 Jun 09
Will CritchlowThis is an interesting read on how Amazon grew
-
26 May 09
-
22 Apr 09
-
16 Apr 09
-
05 Mar 09
-
27 Feb 09
-
26 Jan 09
-
15 Oct 08
-
04 Sep 08
-
03 Sep 08
-
26 Aug 08
-
15 Jul 08
-
-
03 Jul 08
-
22 Jun 08
-
13 Jun 08
-
01 May 08
-
hard to scale-out the overall business
-
If you use a middleware package you get lock-in around the software patterns they have chosen. You'll only be able to use their software.
-
You're stuck
-
gave them the isolation
-
many software components rapidly and independently
-
aggregate the information from the services
-
hundreds of services
-
use only servlets
-
you get lock-in
-
framework and not a tool
-
software patterns they have chosen
-
only be able to use their software
-
70% use REST
-
Teams are Small
-
Organized Around Services
-
two pizza teams
-
anyway they see fit
-
design that is as minimal as possible
-
deliver infinite storage
-
divide nodes into small groups
-
can't see everyone
-
either high consistency or high availability
-
parallel and isolated development
-
create an ecosystem around your application
-
simple as possible
-
expose real customers to a choice and see which one works best
-
getting rid of the influence of the HiPPO's
-
get the most value and innovation
-
-
11 Dec 07
-
06 Dec 07
-
06 Nov 07
-
24 Oct 07
-
04 Oct 07
-
03 Oct 07
-
01 Oct 07
-
27 Sep 07
-
26 Sep 07
-
24 Sep 07
-
21 Sep 07
-
20 Sep 07
-
Avinash Meetoo# More than 55 million active customer accounts.
# More than 1 million active retail partners worldwide.
# Between 100-150 services are accessed to build a page.architecture software amazon scalability web web2.0 internet performance perl java database development distributed enterprise infrastructure j2ee linux oracle programming server soa soap rest statistics technology
-
19 Sep 07
-
18 Sep 07
-
Marqs ShortBuilding bigger, faster, more reliable websites
software web development architecture scalability amazon soa performance delicious
-
17 Sep 07
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.