Skip to content

Add section on logging aggregators to the logging page. #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattmakai merged 5 commits into from
Mar 10, 2014
Merged

Add section on logging aggregators to the logging page. #12

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7e6f73a
WIP on logging aggregator information
Feb 21, 2014
9a7573c
Finish up the writeup on logging aggregators
Mar 10, 2014
2a7d726
Merge with upstream
Mar 10, 2014
d30e8c8
Move content to new logging section and update formatting
Mar 10, 2014
56fd8af
Remove personal page added to store logging framework information
Mar 10, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions source/content/pages/19-logging/1901-logging.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,31 @@ log data to Sentry for aggregation. Sentry provides a clean web application
interface for viewing the exceptions. Sentry can also be configured with a
mail plugin to send emails when exceptions occur.

## Logging Aggregators

When you are running your application on several servers, it is helpful
to have a monitoring tool called a "logging aggregator". You can configure your
application to forward your system and application logs to one location that
provides tools such as to viewing searching, and monitoring logging events across your cluster.

Another advantage of log aggregatortion tools is they allow you to set up custom alerts
and alarms so you can get notified when error rates breach a certain threshold.

### Log Aggregator Third Party Services
* [loggly](https://www.loggly.com/) Loggly is a third party cloud based application that
aggregates logs. They have instructions for every major language, including python. It includes email
alerting on custom searches.
* [papertrail](https://papertrailapp.com/) Paper trail is similar to both loggly and splunk and provides
integration with S3 for
* [splunk](http://www.splunk.com/) Splunk offers third party cloud and self hosted solutions
for event aggregation. It excells at searching and data mining any text based data.

###Open Source Projects
* [Graylog2](http://graylog2.org/) Provides a central server for log aggregation as well as a GUI for
browsing and searching through log events. There are libraries for most major languages, including python.
Saves data in elasicache.
* [Logstash](http://logstash.net/) Similar to Graylog2, logstash offers features to programatically
configure log data workflows.
* [Scribe](https://github.com/facebook/scribe) A project written by facebook to aggregate logs. It's designed
to run on multiple servers and scale with the rest of your cluster. Uses the Thrift messagaing format so it can
be used with any language.