tag:blogger.com,1999:blog-84605682695559193082025-11-06T04:44:05.551+11:00Beaver notesbavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-8460568269555919308.post-13772706892006942442016-07-19T14:44:00.000+10:002016-07-19T22:26:02.831+10:00<h1 class="title">Release of dsq-0.6. Task queue using redis</h1>
<p>I tired from celery/kombu bugs and wrote simple task queue <a class="reference external" href="https://github.com/baverman/dsq">https://github.com/baverman/dsq</a></p>
<p>Why? I can do better. <a class="reference external" href="http://dsq.readthedocs.io/en/latest/faq.html">http://dsq.readthedocs.io/en/latest/faq.html</a></p>
<p>Simple design, small code base with 100% test coverage and library-style
(not framework-style) guarantees stability for production usage.</p>
<p>All features are done, documentation fixes and questions are welcome!</p>
<pre class="code python literal-block">
<span class="comment single"># tasks.py</span>
<span class="keyword namespace">import</span> <span class="name namespace">dsq</span>
<span class="name">manager</span> <span class="operator">=</span> <span class="name">dsq</span><span class="operator">.</span><span class="name">create_manager</span><span class="punctuation">()</span>
<span class="name decorator">@manager.task</span><span class="punctuation">(</span><span class="name">queue</span><span class="operator">=</span><span class="literal string single">'normal'</span><span class="punctuation">)</span>
<span class="keyword">def</span> <span class="name function">add</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">):</span>
<span class="keyword">print</span> <span class="name">a</span> <span class="operator">+</span> <span class="name">b</span>
<span class="keyword">if</span> <span class="name">__name__</span> <span class="operator">==</span> <span class="literal string single">'__main__'</span><span class="punctuation">:</span>
<span class="name">add</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">)</span>
</pre>
<pre class="code bash literal-block">
$ python tasks.py
$ dsq worker -bt tasks normal
</pre>
<p>See full <a class="reference external" href="http://dsq.readthedocs.io/">DSQ documentation</a>.</p>
<div class="section" id="features">
<h1>Features</h1>
<ul class="simple">
<li>Low latency.</li>
<li>Expiring tasks (TTL).</li>
<li>Delayed tasks (ETA).</li>
<li>Retries (forever or particular amount).</li>
<li>Dead letters.</li>
<li>Queue priorities.</li>
<li>Worker lifetime.</li>
<li>Task execution timeout.</li>
<li>Task forwarder from one redis instance to another.</li>
<li>HTTP interface.</li>
<li>Inspect tools.</li>
<li>Supports 2.7, 3.4, 3.5 and PyPy.</li>
<li>100% test coverage.</li>
</ul>
</div>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com0tag:blogger.com,1999:blog-8460568269555919308.post-66327870205634730222013-02-14T21:34:00.000+11:002013-02-14T21:36:36.502+11:00How to change Skype default browser<pre>
xdg-mime default opera.desktop x-scheme-handler/http
xdg-mime default opera.desktop x-scheme-handler/https
</pre>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com2tag:blogger.com,1999:blog-8460568269555919308.post-26767631735677885682010-11-22T18:23:00.002+11:002010-11-22T22:41:08.316+11:00Snaked 0.4<p>Next big Snaked's release. It features:</p>
<ul>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#external-tools">External tool plugin</a>.</p>
</li>
<li><p class="first">Browser mode in quick open dialog.</p>
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#pygtk-hints">Pygtk hint provider</a> allows to
complete attributes and signal handlers parameters based on glade file.</p>
<img alt="http://packages.python.org/snaked/_images/pygtk-hints.png" src="http://packages.python.org/snaked/_images/pygtk-hints.png" />
</li>
<li><p class="first">Regex support in search plugin. Replace is also implemented.</p>
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#snaked.plugins.python.dochints.DocStringHintProvider">Doc string hint provider</a>.
One can place type hints directly in python code.</p>
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#django-hints">Django hint provider</a>. Supported ORM
fields, managers (objects and related sets) and query sets.</p>
<img alt="http://packages.python.org/snaked/_images/django-hints.png" src="http://packages.python.org/snaked/_images/django-hints.png" />
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#snippets">Snippets plugin</a>.</p>
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/start.html#spot-navigation">Spot navigation</a>.</p>
</li>
<li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#spell-check">Spell check</a></p>
</li>
</ul>
<p>Also some minor improvements:</p>
<ul class="simple">
<li>Hide tab bar on <tt class="docutils literal"><Alt>F11</tt>.</li>
<li>Switch tabs on additional "standard" <tt class="docutils literal"><Ctrl>PageUp/PageDown</tt> keys.</li>
<li>Automatic character encoding detection.</li>
<li>Highlight selection on <tt class="docutils literal"><Ctrl>H</tt>.</li>
<li>All search dialogs allow activate the one item by Enter key in search entry.</li>
<li>Hints config format changed. Now it is ordinary python module.</li>
<li>Show cursor column on <tt class="docutils literal"><span class="pre"><Ctrl><Alt>O</span></tt></li>
<li>Editor's line spacing option.</li>
<li>Hints for <tt class="docutils literal">re.RegexObject</tt> and <tt class="docutils literal">re.MatchObject</tt>.</li>
<li>Auto remove trailing space on file save.</li>
<li>Disable left mouse button click in text view window.</li>
<li>Rewrap text on right margin with <tt class="docutils literal"><Alt>F</tt>.</li>
</ul>
<p>What will be in next version?</p>
<ul class="simple">
<li>Multiple snippet contexts.</li>
<li>Variable autocomplete in Django/Jinja templates.</li>
<li>Editing capabilities improving.</li>
</ul>
<p>You can install:</p>
<pre class="literal-block">
sudo pip install snaked
</pre>
<p>Upgrade:</p>
<pre class="literal-block">
sudo pip install -U snaked
</pre>
<p>Or download package directly from <a class="reference external" href="http://pypi.python.org/pypi/snaked/">PyPI</a>.
Also check <a class="reference external" href="http://packages.python.org/snaked/install.html">install instructions</a>.</p>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com4tag:blogger.com,1999:blog-8460568269555919308.post-50328917735776084452010-11-04T18:58:00.008+11:002010-11-04T23:18:44.828+11:00Bad servers, chunked encoding and IncompleteRead<p>Have you ever got IncompleteRead exception on trying to fetch chunked data with
urllib2? I did. Look at the snippet:</p>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">urllib2</span><span style="color: #666666">,</span> <span style="color: #0000FF; font-weight: bold">httplib</span>
<span style="color: #008000; font-weight: bold">try</span>:
data <span style="color: #666666">=</span> urllib2<span style="color: #666666">.</span>urlopen(<span style="color: #BA2121">'http://some.url/address'</span>)<span style="color: #666666">.</span>read()
<span style="color: #008000; font-weight: bold">except</span> httplib<span style="color: #666666">.</span>IncompleteRead:
<span style="color: #408080; font-style: italic"># Ahtung! At this point you lose any fetched data except last chunk.</span>
</pre></div>
<p>IRL most bad servers transmit <em>all</em> data, but due implementation errors they wrongly
close session and urllib raise error and bury your precious bytes.</p>
<p>What you have to do to handle such situation?</p>
<p>I don't like any solutions which involve manual data reading loop, so I prefer
to patch read function.</p>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">httplib</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">patch_http_response_read</span>(func):
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">inner</span>(<span style="color: #666666">*</span>args):
<span style="color: #008000; font-weight: bold">try</span>:
<span style="color: #008000; font-weight: bold">return</span> func(<span style="color: #666666">*</span>args)
<span style="color: #008000; font-weight: bold">except</span> httplib<span style="color: #666666">.</span>IncompleteRead, e:
<span style="color: #008000; font-weight: bold">return</span> e<span style="color: #666666">.</span>partial
<span style="color: #008000; font-weight: bold">return</span> inner
httplib<span style="color: #666666">.</span>HTTPResponse<span style="color: #666666">.</span>read <span style="color: #666666">=</span> patch_http_response_read(httplib<span style="color: #666666">.</span>HTTPResponse<span style="color: #666666">.</span>read)
</pre></div>
<p>It allows you to deal with defective http servers.</p>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com15tag:blogger.com,1999:blog-8460568269555919308.post-18024546358957907722010-10-22T16:16:00.012+11:002010-10-22T17:05:47.210+11:00First Snaked releaseI'm glad to announce first realease of <a href="http://github.com/baverman/snaked">Snaked</a> -- minimalist editor for python developers. Some features:
<ul><li>Python code navigation (goto definition and quick outline) and completion based on <a href="http://rope.sourceforge.net/">Rope refactoring library</a>. Also Snaked can provide additional type information for rope static object infer system to allow you to complete every object in the project.</li>
<li>Light UI without unnecessary widgets.</li>
<li>Keyboard oriented control.</li>
<li>Tabbed or windowed interface on your choice.</li></ul>
<br/>
<h3>Installing</h3>
<pre>
sudo pip install snaked
</pre>
<br/>
<h3>Documentation</h3>
<p>
You can find detailed editor documentation in the folowing page:
<a href="http://packages.python.org/snaked/">Snaked user manual</a>
</p>
<p>
Feel free to post any issues, ideas and suggestions on <a href="http://github.com/baverman/snaked/issues">Github page</a> or in comments.
</p>
<br/><br/>
<img src="http://packages.python.org/snaked/_images/complete.png" />bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com2