This link has been bookmarked by 46 people . It was first bookmarked on 30 Mar 2008, by Scott Moody.
-
04 Jun 15
-
14 Mar 12
-
- Limited to applications whose data can be easily partitioned.
- Performance: we are still constrained by disk I/O performance
- Limited to applications whose data can be easily partitioned.
-
-
14 Apr 10
-
12 Oct 09
-
24 Jul 09
-
MySQL becomes a scalability bottleneck because it introduces contention among the distributed application components.
-
Scale your database through database clustering
2. Scale your application, while leaving your existing database untouched by front-ending the database with In-Memory-Data-Grid (IMDG) or caching technologies. The database acts as a persistence store in the background. I refer to this approach as Persistence as a Service (PaaS).
-
Database replication is used to address concurrent access to the same data. Database replication enables us to load-balance the access to the shared data elements among multiple replicated database instances
-
Limited to "read mostly" scenarios: when it comes to inserts and updates, replication overhead may be a bigger constraint than working with a single server (especially with synchronous replication)
-
asynchronous replication leads to inconsistency
-
replication assumes that all nodes hold the entire data set. This creates two problems:.1) each table holds a large amount of data, which increases query/index complexity. 2) We need to provision (and pay for) more storage capacity with direct proportion to the number of replicated database instances
-
database shards/partitions enable the distribution of data on multiple nodes. In other words, each node holds part of the data. This is a better approach for scaling both read and write operations, as well as more efficient use of capacity, as it reduces the volume of data in each database instance.
-
constrained by disk I/O performance
-
Many database implementations require that knowledge of which partition the data resides in is exposed to the application code
-
The fundamental problems with both database replication and database partitioning are the reliance on the performance of the file system/disk and the complexity involved in setting up database clusters.
-
These database patterns evolved under the assumption that memory is scarce and expensive, and that network bandwidth is a bottleneck. Today, memory resources are abundant and available at a relatively low cost. So is bandwidth.
-
its protocol-based approach requires the application to be explicitly exposed to the cache topology, i.e., it needs to be aware of each server host, and explicitly map operations to a specific node. These limitations prevent us from fully exploiting the memory resources available to us.
-
In-memory data grids (IMDG) provide object-based database capabilities in memory, and support core database functionality, such as advanced indexing and querying, transactional semantics and locking
-
IMDGs also abstract data topology from application code.
-
The IMDG loads its initial state from the database and from that point on acts as the "system of record" for our application. In other words, all updates and queries are handled by the IMDG. The IMDG is also responsible for keeping the database in sync.
-
To reduce performance overhead, synchronization with the database is done asynchronously
-
Data can be accessed by reference
-
The database is not used for high availability purposes. This means that In-flight transactions are not stored in the database, only the end result of the business transactions
-
Also keep in mind that queries don't really hit the database, only updates and inserts. All this together means that the IMDG acts as a smart buffer to the database
-
The database and the application are now decoupled, giving you more options for optimization
-
there are scenarios where writing to the database is required to ensure the durability of the data. In this scenario, you can store the data directly in a persistent log (to ensure durability). The log can be updated at a relatively high rate
-
when we use in-memory storage in an IMDG, we don't need the O/R mapping layer. In addition, we don't need separate languages to perform data manipulation. We can use the native application code, or dynamic languages, for that purpose.
-
-
03 Nov 08
-
19 Jul 08
-
02 Jul 08
-
03 Jun 08
-
23 May 08
-
20 May 08
-
19 May 08
-
deanburgeDiscussions about middleware and distributed technologies
-
18 May 08
-
28 Apr 08
-
20 Apr 08
-
19 Apr 08
-
13 Apr 08
-
05 Apr 08
-
04 Apr 08
-
02 Apr 08
-
01 Apr 08
-
31 Mar 08
-
30 Mar 08
Brent SordylThere are basically two approaches to this challenge that I'll touch in this post:1. Scale your database through database clustering 2. Scale your application, while leaving your existing database untouched by front-ending the database
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.