tag:blogger.com,1999:blog-84605682695559193082025-11-06T04:44:05.551+11:00Beaver notesbavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-8460568269555919308.post-13772706892006942442016-07-19T14:44:00.000+10:002016-07-19T22:26:02.831+10:00<h1 class="title">Release of dsq-0.6. Task queue using redis</h1> <p>I tired from celery/kombu bugs and wrote simple task queue <a class="reference external" href="https://github.com/baverman/dsq">https://github.com/baverman/dsq</a></p> <p>Why? I can do better. <a class="reference external" href="http://dsq.readthedocs.io/en/latest/faq.html">http://dsq.readthedocs.io/en/latest/faq.html</a></p> <p>Simple design, small code base with 100% test coverage and library-style (not framework-style) guarantees stability for production usage.</p> <p>All features are done, documentation fixes and questions are welcome!</p> <pre class="code python literal-block"> <span class="comment single"># tasks.py</span> <span class="keyword namespace">import</span> <span class="name namespace">dsq</span> <span class="name">manager</span> <span class="operator">=</span> <span class="name">dsq</span><span class="operator">.</span><span class="name">create_manager</span><span class="punctuation">()</span> <span class="name decorator">&#64;manager.task</span><span class="punctuation">(</span><span class="name">queue</span><span class="operator">=</span><span class="literal string single">'normal'</span><span class="punctuation">)</span> <span class="keyword">def</span> <span class="name function">add</span><span class="punctuation">(</span><span class="name">a</span><span class="punctuation">,</span> <span class="name">b</span><span class="punctuation">):</span> <span class="keyword">print</span> <span class="name">a</span> <span class="operator">+</span> <span class="name">b</span> <span class="keyword">if</span> <span class="name">__name__</span> <span class="operator">==</span> <span class="literal string single">'__main__'</span><span class="punctuation">:</span> <span class="name">add</span><span class="punctuation">(</span><span class="literal number integer">1</span><span class="punctuation">,</span> <span class="literal number integer">2</span><span class="punctuation">)</span> </pre> <pre class="code bash literal-block"> $ python tasks.py $ dsq worker -bt tasks normal </pre> <p>See full <a class="reference external" href="http://dsq.readthedocs.io/">DSQ documentation</a>.</p> <div class="section" id="features"> <h1>Features</h1> <ul class="simple"> <li>Low latency.</li> <li>Expiring tasks (TTL).</li> <li>Delayed tasks (ETA).</li> <li>Retries (forever or particular amount).</li> <li>Dead letters.</li> <li>Queue priorities.</li> <li>Worker lifetime.</li> <li>Task execution timeout.</li> <li>Task forwarder from one redis instance to another.</li> <li>HTTP interface.</li> <li>Inspect tools.</li> <li>Supports 2.7, 3.4, 3.5 and PyPy.</li> <li>100% test coverage.</li> </ul> </div>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com0tag:blogger.com,1999:blog-8460568269555919308.post-66327870205634730222013-02-14T21:34:00.000+11:002013-02-14T21:36:36.502+11:00How to change Skype default browser<pre> xdg-mime default opera.desktop x-scheme-handler/http xdg-mime default opera.desktop x-scheme-handler/https </pre>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com2tag:blogger.com,1999:blog-8460568269555919308.post-26767631735677885682010-11-22T18:23:00.002+11:002010-11-22T22:41:08.316+11:00Snaked 0.4<p>Next big Snaked's release. It features:</p> <ul> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#external-tools">External tool plugin</a>.</p> </li> <li><p class="first">Browser mode in quick open dialog.</p> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#pygtk-hints">Pygtk hint provider</a> allows to complete attributes and signal handlers parameters based on glade file.</p> <img alt="http://packages.python.org/snaked/_images/pygtk-hints.png" src="http://packages.python.org/snaked/_images/pygtk-hints.png" /> </li> <li><p class="first">Regex support in search plugin. Replace is also implemented.</p> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#snaked.plugins.python.dochints.DocStringHintProvider">Doc string hint provider</a>. One can place type hints directly in python code.</p> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/python.html#django-hints">Django hint provider</a>. Supported ORM fields, managers (objects and related sets) and query sets.</p> <img alt="http://packages.python.org/snaked/_images/django-hints.png" src="http://packages.python.org/snaked/_images/django-hints.png" /> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#snippets">Snippets plugin</a>.</p> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/start.html#spot-navigation">Spot navigation</a>.</p> </li> <li><p class="first"><a class="reference external" href="http://packages.python.org/snaked/plugins.html#spell-check">Spell check</a></p> </li> </ul> <p>Also some minor improvements:</p> <ul class="simple"> <li>Hide tab bar on <tt class="docutils literal">&lt;Alt&gt;F11</tt>.</li> <li>Switch tabs on additional &quot;standard&quot; <tt class="docutils literal">&lt;Ctrl&gt;PageUp/PageDown</tt> keys.</li> <li>Automatic character encoding detection.</li> <li>Highlight selection on <tt class="docutils literal">&lt;Ctrl&gt;H</tt>.</li> <li>All search dialogs allow activate the one item by Enter key in search entry.</li> <li>Hints config format changed. Now it is ordinary python module.</li> <li>Show cursor column on <tt class="docutils literal"><span class="pre">&lt;Ctrl&gt;&lt;Alt&gt;O</span></tt></li> <li>Editor's line spacing option.</li> <li>Hints for <tt class="docutils literal">re.RegexObject</tt> and <tt class="docutils literal">re.MatchObject</tt>.</li> <li>Auto remove trailing space on file save.</li> <li>Disable left mouse button click in text view window.</li> <li>Rewrap text on right margin with <tt class="docutils literal">&lt;Alt&gt;F</tt>.</li> </ul> <p>What will be in next version?</p> <ul class="simple"> <li>Multiple snippet contexts.</li> <li>Variable autocomplete in Django/Jinja templates.</li> <li>Editing capabilities improving.</li> </ul> <p>You can install:</p> <pre class="literal-block"> sudo pip install snaked </pre> <p>Upgrade:</p> <pre class="literal-block"> sudo pip install -U snaked </pre> <p>Or download package directly from <a class="reference external" href="http://pypi.python.org/pypi/snaked/">PyPI</a>. Also check <a class="reference external" href="http://packages.python.org/snaked/install.html">install instructions</a>.</p>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com4tag:blogger.com,1999:blog-8460568269555919308.post-50328917735776084452010-11-04T18:58:00.008+11:002010-11-04T23:18:44.828+11:00Bad servers, chunked encoding and IncompleteRead<p>Have you ever got IncompleteRead exception on trying to fetch chunked data with urllib2? I did. Look at the snippet:</p> <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">urllib2</span><span style="color: #666666">,</span> <span style="color: #0000FF; font-weight: bold">httplib</span> <span style="color: #008000; font-weight: bold">try</span>: data <span style="color: #666666">=</span> urllib2<span style="color: #666666">.</span>urlopen(<span style="color: #BA2121">&#39;http://some.url/address&#39;</span>)<span style="color: #666666">.</span>read() <span style="color: #008000; font-weight: bold">except</span> httplib<span style="color: #666666">.</span>IncompleteRead: <span style="color: #408080; font-style: italic"># Ahtung! At this point you lose any fetched data except last chunk.</span> </pre></div> <p>IRL most bad servers transmit <em>all</em> data, but due implementation errors they wrongly close session and urllib raise error and bury your precious bytes.</p> <p>What you have to do to handle such situation?</p> <p>I don't like any solutions which involve manual data reading loop, so I prefer to patch read function.</p> <div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">httplib</span> <span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">patch_http_response_read</span>(func): <span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">inner</span>(<span style="color: #666666">*</span>args): <span style="color: #008000; font-weight: bold">try</span>: <span style="color: #008000; font-weight: bold">return</span> func(<span style="color: #666666">*</span>args) <span style="color: #008000; font-weight: bold">except</span> httplib<span style="color: #666666">.</span>IncompleteRead, e: <span style="color: #008000; font-weight: bold">return</span> e<span style="color: #666666">.</span>partial <span style="color: #008000; font-weight: bold">return</span> inner httplib<span style="color: #666666">.</span>HTTPResponse<span style="color: #666666">.</span>read <span style="color: #666666">=</span> patch_http_response_read(httplib<span style="color: #666666">.</span>HTTPResponse<span style="color: #666666">.</span>read) </pre></div> <p>It allows you to deal with defective http servers.</p>bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com15tag:blogger.com,1999:blog-8460568269555919308.post-18024546358957907722010-10-22T16:16:00.012+11:002010-10-22T17:05:47.210+11:00First Snaked releaseI'm glad to announce first realease of <a href="http://github.com/baverman/snaked">Snaked</a> -- minimalist editor for python developers. Some features: <ul><li>Python code navigation (goto definition and quick outline) and completion based on <a href="http://rope.sourceforge.net/">Rope refactoring library</a>. Also Snaked can provide additional type information for rope static object infer system to allow you to complete every object in the project.</li> <li>Light UI without unnecessary widgets.</li> <li>Keyboard oriented control.</li> <li>Tabbed or windowed interface on your choice.</li></ul> <br/> <h3>Installing</h3> <pre> sudo pip install snaked </pre> <br/> <h3>Documentation</h3> <p> You can find detailed editor documentation in the folowing page: <a href="http://packages.python.org/snaked/">Snaked user manual</a> </p> <p> Feel free to post any issues, ideas and suggestions on <a href="http://github.com/baverman/snaked/issues">Github page</a> or in comments. </p> <br/><br/> <img src="http://packages.python.org/snaked/_images/complete.png" />bavermanhttp://www.blogger.com/profile/14753352519863209882noreply@blogger.com2