Skip to content

Commit beb33ca

Browse files
mattmakaimattmakai
committed
building pull request of logging aggregators for FSP
1 parent 15ea244 commit beb33ca

4 files changed

Lines changed: 41 additions & 14 deletions

File tree

‎feeds/all.atom.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
<?xml version="1.0" encoding="utf-8"?>
2-
<feed xmlns="http://www.w3.org/2005/Atom"><title>Matt Makai</title><link href="http://www.fullstackpython.com/" rel="alternate"></link><link href="http://www.fullstackpython.com/feeds/all.atom.xml" rel="self"></link><id>http://www.fullstackpython.com/</id><updated>2014-03-09T12:17:12Z</updated></feed>
2+
<feed xmlns="http://www.w3.org/2005/Atom"><title>Matt Makai</title><link href="http://www.fullstackpython.com/" rel="alternate"></link><link href="http://www.fullstackpython.com/feeds/all.atom.xml" rel="self"></link><id>http://www.fullstackpython.com/</id><updated>2014-03-09T22:50:25Z</updated></feed>

‎logging.html‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,34 @@ <h1>Logging</h1>
9797
log data to Sentry for aggregation. Sentry provides a clean web application
9898
interface for viewing the exceptions. Sentry can also be configured with a
9999
mail plugin to send emails when exceptions occur.</p>
100+
<h2>Logging Aggregators</h2>
101+
<p>When you are running your application on several servers, it is helpful
102+
to have a monitoring tool called a "logging aggregator". You can configure your
103+
application to forward your system and application logs to one location that
104+
provides tools such as to viewing searching, and monitoring logging events across your cluster. </p>
105+
<p>Another advantage of log aggregatortion tools is they allow you to set up custom alerts
106+
and alarms so you can get notified when error rates breach a certain threshold.</p>
107+
<h3>Log Aggregator Third Party Services</h3>
108+
<ul>
109+
<li><a href="https://www.loggly.com/">loggly</a> Loggly is a third party cloud based application that
110+
aggregates logs. They have instructions for every major language, including python. It includes email
111+
alerting on custom searches. </li>
112+
<li><a href="https://papertrailapp.com/">papertrail</a> Paper trail is similar to both loggly and splunk and provides
113+
integration with S3 for </li>
114+
<li><a href="http://www.splunk.com/">splunk</a> Splunk offers third party cloud and self hosted solutions
115+
for event aggregation. It excells at searching and data mining any text based data. </li>
116+
</ul>
117+
<h3>Open Source Log Aggregators</h3>
118+
<ul>
119+
<li><a href="http://graylog2.org/">Graylog2</a> Provides a central server for log aggregation as well as a GUI for
120+
browsing and searching through log events. There are libraries for most major languages, including python.
121+
Saves data in elasicache.</li>
122+
<li><a href="http://logstash.net/">Logstash</a> Similar to Graylog2, logstash offers features to programatically
123+
configure log data workflows.</li>
124+
<li><a href="https://github.com/facebook/scribe">Scribe</a> A project written by facebook to aggregate logs. It's designed
125+
to run on multiple servers and scale with the rest of your cluster. Uses the Thrift messagaing format so it can
126+
be used with any language. </li>
127+
</ul>
100128
<br/>
101129
Next read the
102130
<a href="/monitoring.html">monitoring</a> section.

‎source/content/pages/19-logging/1901-logging.markdown‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ interface for viewing the exceptions. Sentry can also be configured with a
2929
mail plugin to send emails when exceptions occur.
3030

3131
## Logging Aggregators
32-
3332
When you are running your application on several servers, it is helpful
3433
to have a monitoring tool called a "logging aggregator". You can configure your
3534
application to forward your system and application logs to one location that
@@ -47,7 +46,7 @@ integration with S3 for
4746
* [splunk](http://www.splunk.com/) Splunk offers third party cloud and self hosted solutions
4847
for event aggregation. It excells at searching and data mining any text based data.
4948

50-
###Open Source Projects
49+
### Open Source Log Aggregators
5150
* [Graylog2](http://graylog2.org/) Provides a central server for log aggregation as well as a GUI for
5251
browsing and searching through log events. There are libraries for most major languages, including python.
5352
Saves data in elasicache.

‎wsgi-servers.html‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -91,30 +91,30 @@ <h1>WSGI Servers</h1>
9191
Nginx should handle static assets (such as images, JavaScript, and CSS
9292
files) under the /static directory and pass all other requests to the WSGI
9393
server running on port 8000:</p>
94-
<div class="codehilite"><pre><span class="c"># this specifies that there is a WSGI server running on port 8000</span>
94+
<div class="codehilite"><pre><span class="cp"># this specifies that there is a WSGI server running on port 8000</span>
9595
<span class="n">upstream</span> <span class="n">app_server_djangoapp</span> <span class="p">{</span>
96-
<span class="n">server</span> <span class="n">localhost</span><span class="p">:</span>8000 <span class="n">fail_timeout</span><span class="p">=</span>0<span class="p">;</span>
96+
<span class="n">server</span> <span class="n">localhost</span><span class="o">:</span><span class="mi">8000</span> <span class="n">fail_timeout</span><span class="o">=</span><span class="mi">0</span><span class="p">;</span>
9797
<span class="p">}</span>
9898

99-
<span class="c"># Nginx is set up to run on the standard HTTP port and listen for requests</span>
99+
<span class="cp"># Nginx is set up to run on the standard HTTP port and listen for requests</span>
100100
<span class="n">server</span> <span class="p">{</span>
101-
<span class="n">listen</span> 80<span class="p">;</span>
101+
<span class="n">listen</span> <span class="mi">80</span><span class="p">;</span>
102102

103-
<span class="c"># nginx should serve up static files and never send to the WSGI server</span>
103+
<span class="err">#</span> <span class="n">nginx</span> <span class="n">should</span> <span class="n">serve</span> <span class="n">up</span> <span class="k">static</span> <span class="n">files</span> <span class="n">and</span> <span class="n">never</span> <span class="n">send</span> <span class="n">to</span> <span class="n">the</span> <span class="n">WSGI</span> <span class="n">server</span>
104104
<span class="n">location</span> <span class="o">/</span><span class="k">static</span> <span class="p">{</span>
105105
<span class="n">autoindex</span> <span class="n">on</span><span class="p">;</span>
106106
<span class="n">alias</span> <span class="o">/</span><span class="n">srv</span><span class="o">/</span><span class="n">www</span><span class="o">/</span><span class="n">assets</span><span class="p">;</span>
107107
<span class="p">}</span>
108108

109-
<span class="c"># requests that do not fall under /static are passed on to the WSGI</span>
110-
<span class="c"># server that was specified above running on port 8000</span>
109+
<span class="err">#</span> <span class="n">requests</span> <span class="n">that</span> <span class="k">do</span> <span class="n">not</span> <span class="n">fall</span> <span class="n">under</span> <span class="o">/</span><span class="k">static</span> <span class="n">are</span> <span class="n">passed</span> <span class="n">on</span> <span class="n">to</span> <span class="n">the</span> <span class="n">WSGI</span>
110+
<span class="err">#</span> <span class="n">server</span> <span class="n">that</span> <span class="n">was</span> <span class="n">specified</span> <span class="n">above</span> <span class="n">running</span> <span class="n">on</span> <span class="n">port</span> <span class="mi">8000</span>
111111
<span class="n">location</span> <span class="o">/</span> <span class="p">{</span>
112-
<span class="n">proxy_set_header</span> <span class="n">X</span><span class="o">-</span><span class="n">Forwarded</span><span class="o">-</span><span class="n">For</span> $<span class="n">proxy_add_x_forwarded_for</span><span class="p">;</span>
113-
<span class="n">proxy_set_header</span> <span class="n">Host</span> $<span class="n">http_host</span><span class="p">;</span>
112+
<span class="n">proxy_set_header</span> <span class="n">X</span><span class="o">-</span><span class="n">Forwarded</span><span class="o">-</span><span class="n">For</span> <span class="err">$</span><span class="n">proxy_add_x_forwarded_for</span><span class="p">;</span>
113+
<span class="n">proxy_set_header</span> <span class="n">Host</span> <span class="err">$</span><span class="n">http_host</span><span class="p">;</span>
114114
<span class="n">proxy_redirect</span> <span class="n">off</span><span class="p">;</span>
115115

116-
<span class="k">if</span> <span class="p">(</span><span class="o">!-</span><span class="n">f</span> $<span class="n">request_filename</span><span class="p">)</span> <span class="p">{</span>
117-
<span class="n">proxy_pass</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">app_server_djangoapp</span><span class="p">;</span>
116+
<span class="k">if</span> <span class="p">(</span><span class="o">!-</span><span class="n">f</span> <span class="err">$</span><span class="n">request_filename</span><span class="p">)</span> <span class="p">{</span>
117+
<span class="n">proxy_pass</span> <span class="n">http</span><span class="o">:</span><span class="c1">//app_server_djangoapp;</span>
118118
<span class="k">break</span><span class="p">;</span>
119119
<span class="p">}</span>
120120
<span class="p">}</span>

0 commit comments

Comments
 (0)