Home
Joel Liu's Library tagged → View Popular, Search in Google
-
The action
ssize
sets the maximum file size for subsequent dir actions. multilog will decide that current is big enough if current has size bytes. (multilog will also decide that current is big enough if it sees a newline within 2000 bytes of the maximum file size; it tries to finish log files at line boundaries.) size must be between 4096 and 16777215. The default maximum file size is 99999. -
The action
nnum
sets the number of log files for subsequent dir actions. After renaming current, if multilog sees num or more old log files, it removes the old log file with the smallest timestamp. num must be at least 2. The default number of log files is 10.
Here at Facebook, we're constantly facing scaling challanges because of our enormous growth. One particular problem we encountered a couple of years ago was collection of data from our servers. We were collecting a few billion messages a day (which seemed like a lot at the time) for everything from access logs to performance statistics to actions that went to News Feed. We used a variety of different technologies for the different use cases, and all of them were bursting at the seams. We decided to build a unified system (called Scribe) to handle all of these cases, and do it in a way that would scale with Facebook's growth. The system we built turned out to be enormously useful, handling over 100 use cases and tens of billions of messages a day. It has also been battle tested by just about anything that can go wrong, so I encourage you to take a look at the newly opened Scribe source and see if it might be useful for you. To give the code some context, I'm going to go through the major design decisions we made to allow the system to scale.
-
When you back up the slave's data, you should back up these two status files as well, along with the relay log files. They are needed to resume replication after you restore the slave's data. If you lose the relay logs but still have the
relay-log.infofile, you can check it to determine how far the SQL thread has executed in the master binary logs. Then you can useCHANGE MASTER TOwith theMASTER_LOG_FILEandMASTER_LOG_POSoptions to tell the slave to re-read the binary logs from that point. Of course, this requires that the binary logs still exist on the master server. -
Relay logs have the same format as binary logs and can be read using mysqlbinlog. The SQL thread automatically deletes each relay log file as soon as it has executed all events in the file and no longer needs it. There is no explicit mechanism for deleting relay logs because the SQL thread takes care of doing so. However,
FLUSH LOGSrotates relay logs, which influences when the SQL thread deletes them.
-
Logmerge contains two utilities logmerge and ip2name. logmerge merges Apache access logs into one log ordered by date. ip2name performs DNS lookups on Apache access logs using multiple threads and Ruby's DNS resolver library to speed through log files.
-
The Production Log Analyzer examines Rails log files and gives a report showing you the worst speed offendors. It also includes action_grep which will give you all log results for a particular action.
Selected Tags
Related Tags
Top Contributors
Groups interested in log
-
Learning Log 2
Resources for Learning Log 2...
Items: 64 | Visits: 34
Created by: Carl Davis
-
log in pages
Items: 7 | Visits: 27
Created by: Denise Simon
Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »
Join Diigo