Kafka suffers significant bottlenecks as the number of queues increase: leader reassignment latency increases dramatically, memory requirements increase significantly, and many of the metadata APIs become noticeably slower. As the queues become smaller and more numerous, the workload begins to resemble that of a generic random-access datastore.
- High-throughput writes (or puts, or produces) and reads (or gets, or consumes)
- Reliable persistence of data to multiple nodes, so data isn’t lost if failures occur
- Detecting and handling node failures with minimal disruption
- Rebalancing data as nodes are added to avoid hotspots
- Scaling, both in terms of number of nodes but also number of objects
Alternative install options¶
Install using pip¶
Compose can be installed from pypi using
pip. If you install using
pipit is highly recommended that you use a virtualenv because many operating systems have python system packages that conflict with docker-compose dependencies. See the virtualenv tutorial to get started.
$ pip install docker-compose
Note: pip version 6.0 or greater is required
Install as a container¶
Compose can also be run inside a container, from a small bash script wrapper. To install compose as a container run:
$ curl -L https://github.com/docker/compose/releases/download/1.8.0/run.sh > /usr/local/bin/docker-compose $ chmod +x /usr/local/bin/docker-compose