danuker | freedom & tech - Englishhttps://danuker.go.ro/2021-10-07T00:10:00+03:00On Saudi Arabia2021-10-07T00:10:00+03:002021-10-07T00:10:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2021-10-07:/on-saudi-arabia.html<p>I have recently seen <a href="https://www.youtube.com/watch?v=V16GdzRvhRU">a video</a> by Wendover Productions about the Kingdom of Saudi Arabia.</p>
<p>It presents that the Kingdom needs foreign investment to be able to thrive past <a href="https://en.wikipedia.org/wiki/Peak_oil">peak oil</a>.
But the leaders <a href="https://www.theguardian.com/commentisfree/2018/nov/02/jamal-khashoggi-justice-unite-hatice-cengiz">take extreme offense and kill people</a> based on what they write about the Kingdom. </p>
<p>So, yours …</p><p>I have recently seen <a href="https://www.youtube.com/watch?v=V16GdzRvhRU">a video</a> by Wendover Productions about the Kingdom of Saudi Arabia.</p>
<p>It presents that the Kingdom needs foreign investment to be able to thrive past <a href="https://en.wikipedia.org/wiki/Peak_oil">peak oil</a>.
But the leaders <a href="https://www.theguardian.com/commentisfree/2018/nov/02/jamal-khashoggi-justice-unite-hatice-cengiz">take extreme offense and kill people</a> based on what they write about the Kingdom. </p>
<p>So, yours truly is fanning the flame.</p>
<p>After this video, I think the most effective steps in Saudi Arabia gaining international trust would be:</p>
<ol>
<li>Leaders admitting they are only human, and make mistakes sometimes. Making a mistake is not a problem, but persisting in failure with no prospect of improvement is a problem.</li>
<li>Do not execute people. Aside from the chilling effects this has on tourism and investment, do you want your people to live in fear?</li>
<li>Ensure equal rights for women, and people of other and no creeds. I will not come to Saudi Arabia with such severe punishments based on religion.</li>
<li>Let press write whatever it wants. If the press insults the Kingdom, it proves the press is cheap and childish, and it will fall into oblivion by itself. But if the press is <a href="https://rsf.org/en/saudi-arabia">saying the truth</a>, perhaps listen to it.</li>
</ol>
<p>I hope that if any Saudi official reads this, they take it seriously, because the wealth of a nation depends on it. With great power comes great responsibility. </p>On surveillance through face recognition2021-06-13T18:05:00+03:002021-06-13T18:05:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2021-06-13:/on-surveillance-through-face-recognition.html<p><a href="https://eu.freep.com/story/news/local/michigan/detroit/2020/06/26/facial-recognition-wrongful-arrest-detroit-police/3265943001/">A year ago</a>, a <a href="https://www.cjr.org/analysis/capital-b-black-styleguide.php">Black</a> man was arrested for theft. The evidence was crummy face-recognition coupled with a security guard's ID who had not seen the person, only a security tape.</p>
<p><a href="https://www.aclu.org/news/privacy-technology/the-computer-got-it-wrong-why-were-taking-the-detroit-police-to-court-over-a-faulty-face-recognition-match/">A few months ago</a>, some civil rights organizations filed a lawsuit against the Detroit Police Department for the wrongful …</p><p><a href="https://eu.freep.com/story/news/local/michigan/detroit/2020/06/26/facial-recognition-wrongful-arrest-detroit-police/3265943001/">A year ago</a>, a <a href="https://www.cjr.org/analysis/capital-b-black-styleguide.php">Black</a> man was arrested for theft. The evidence was crummy face-recognition coupled with a security guard's ID who had not seen the person, only a security tape.</p>
<p><a href="https://www.aclu.org/news/privacy-technology/the-computer-got-it-wrong-why-were-taking-the-detroit-police-to-court-over-a-faulty-face-recognition-match/">A few months ago</a>, some civil rights organizations filed a lawsuit against the Detroit Police Department for the wrongful arrest.</p>
<p>I want to offer my insight into the issue of arrests based on face recognition.</p>
<h1 id="a-game-of-numbers"><a class="toclink" href="#a-game-of-numbers">A game of numbers</a></h1>
<p>Suppose the face recognition system has a one-in-a-million FPR (false positive rate).</p>
<p>Let's also say it has 100% TPR (true positive rate, or "100% recall" - meaning if the guilty person is in the dataset, they will be matched also).</p>
<p>If you search for matches through the 1M faces, and the suspect is among those records, you get on average 1 false positive match, and 1 true positive.</p>
<p>So the people you are arresting only have a 1 in 2 chance of being guilty (though to be completely fair, the police would know something's up if it gets two matches, but here it was apparently just one).</p>
<h1 id="bigger-numbers"><a class="toclink" href="#bigger-numbers">Bigger numbers</a></h1>
<p>If you scale this up ("<a href="https://www.nytimes.com/2019/07/08/us/detroit-facial-recognition-cameras.html">50 million driver’s license photographs and mug shots contained in a Michigan police database</a>"), you get roughly <strong>50 false positives, and maybe one true positive, if it's in the data.</strong> But if the guilty person is not in the dataset, the police only get the false positives.</p>
<p>To be fair, usually you can restrict the suspect pool somehow else. But here, the arrest was <strong>based on</strong> facial recognition.</p>
<h1 id="benefit-of-the-doubt"><a class="toclink" href="#benefit-of-the-doubt">Benefit of the doubt</a></h1>
<p>The Detroit Police themselves admit a misidentification rate of <a href="https://arstechnica.com/tech-policy/2020/06/detroit-police-chief-admits-facial-recognition-is-wrong-96-of-the-time/">96%</a> - which means the false-positive-to-true-positive ratio is not the 50 we guessed above, but 96/(100-96) = 24.</p>
<p><a href="http://www.dataworksplus.com/faceplus.html">The tech supplier does not mention the error rate of their product</a>.</p>
<p>However, <a href="https://web.archive.org/web/20210611140250/https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf">NIST ran some tests</a> in 2019, which are no longer available on the NIST website, but fortunately they are on archive.org (<a href="https://web.archive.org/web/20210611140250/https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf">linked</a>).</p>
<p>(By the way, I find it suspicious that NIST removed this relevant study at a time when this trial begins. In case archive.org deletes it, you can also download it <a href="NIST.IR.8280.pdf">here</a>).</p>
<p>They made roughly 195 billion comparisons between known-differrent people. The False Match Rate (same as our FPR) of algorithms ranges between 1e-6 to 2e-4 (5th to 95th percentiles respectively), or from one-in-a-million to 2-in-10-thousand. This means for each million of comparisons between random people, the algorithms flag about 1 to 200 as false matches.</p>
<p>So, checking a single person against Detroit's 50M database, would yield somewhere between 50 matches to 10k matches. Giving them the benefit of the doubt, and assuming the suspect is indeed in the dataset, that would mean a false-positive-to-true-positive ratio between 50 and 10000.</p>
<p>The value of 24 claimed by the police lies outside this interval, which means it is less than 5% likely that they are telling the truth. But let's assume they do, since technology improves year by year.</p>
<h1 id="more-accurate-numbers-race"><a class="toclink" href="#more-accurate-numbers-race">More accurate numbers: Race</a></h1>
<p>Fortunately, the NIST report breaks down the false-match-rate by country. On page 42 of <a href="https://web.archive.org/web/20210611140250/https://nvlpubs.nist.gov/nistpubs/ir/2019/NIST.IR.8280.pdf">the report</a> shows that while the false match rate within E. Europe is at least 1e-6, that for men within W. Africa is at least 7.5e-6, which is 7.5 times worse. For women it's much worse (>10e-6 vs. 1e-6).</p>
<p>This makes it likely that Black people in the US suffer more false positives (if not 7.5 times more), and seeing that <a href="https://arstechnica.com/tech-policy/2020/06/detroit-police-chief-admits-facial-recognition-is-wrong-96-of-the-time/">"68 out of 70 facial recognition searches were done on Black suspects"</a>, this casts further doubt on their claimed 4% success rate.</p>
<h1 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h1>
<p>The abysmal 4% success rate that Detroit Police claim is very optimistic, even putting into question the cases successfully indicted.</p>
<p>Even if it were correct, it would mean a typical search in their database matches 24 innocent people for every legitimate match, and has very limited statistical power, due to the large datasets.</p>
<p>In addition, eyewitness identification uses the same features as the automated one (namely, the face). As such, it is unfit for use as corroboration, as was done in this case.</p>
<p>I will refer to <a href="https://en.wikipedia.org/wiki/Blackstone%27s_ratio">Blackstone's ratio</a>:</p>
<blockquote>
<p>It is better that ten guilty persons escape than that one innocent suffer.</p>
</blockquote>
<p>The implied performance of one guilty conviction to 24 innocents suffering is far short of Blackstone's ratio (which is itself a minimum; the ideal ratio may be even higher).</p>
<p>Normalizing, we'd get 1/24th of a guilty conviction for every innocent suffering, which is 1/240th of Blackstone's ratio.</p>
<p>As such, a face recognition match could be used as circumstantial evidence at best.</p>SymReg: approximate functions using Symbolic Regression2020-12-16T17:00:00+02:002020-12-16T17:00:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2020-12-16:/symreg-approximate-functions-using-symbolic-regression.html<p>I recently made <a href="https://github.com/danuker/symreg">SymReg</a>. It lets you discover mathematical expressions that explain some data based on some other data.</p>
<p>The code used for this blog post is in <a href="https://github.com/danuker/symreg/blob/master/demo-sine.ipynb">this notebook</a>.</p>
<p>Let's try it on a sine wave, in spite of the fact that it does not have a sine building …</p><p>I recently made <a href="https://github.com/danuker/symreg">SymReg</a>. It lets you discover mathematical expressions that explain some data based on some other data.</p>
<p>The code used for this blog post is in <a href="https://github.com/danuker/symreg/blob/master/demo-sine.ipynb">this notebook</a>.</p>
<p>Let's try it on a sine wave, in spite of the fact that it does not have a sine building block:</p>
<p><img alt="png" class="img-fluid" src="images/sine_fit.png"/></p>
<p>As you can see, the function found is quite a good fit. But a complexity of 337 has its disadvantages: it may be <a href="https://en.wikipedia.org/wiki/Overfitting">overfit</a>. As soon as we go outside the trained interval, there are explosions:</p>
<p><img alt="png" class="img-fluid" src="images/sine_overfit.png"/></p>
<p>Limiting the predictor complexity with <code>max_complexity</code> acts similarly to <a href="https://towardsdatascience.com/how-to-improve-a-neural-network-with-regularization-8a18ecda9fe3">regularization of neural networks</a>. The behavior on the outside is a bit tamer now - +/- 6 instead of +/- 15.</p>
<p><img alt="png" class="img-fluid" src="images/sine_overfit_regularized.png"/></p>
<p>Sadly, Python has no multithreading due to the Global Interpreter Lock, which is a significant limitation for CPU-bound code like this. Also, multiprocessing is very slow, taking more to launch a fork than it takes a whole generation to evolve.</p>
<p>If you know a way around the GIL, please let me know! But it seems the only way is to change the programming language.</p>
<p>Still, this tool might be useful for you. And if you are optimizing multiple functions, you could do it in parallel quite efficiently. Try it out and share your thoughts!</p>The Grand Unified Theory of Software Architecture2020-09-05T15:37:00+03:002020-09-05T15:37:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-09-05:/the-grand-unified-theory-of-software-architecture.html<p>Take <strong>Uncle Bob's</strong> Clean Architecture and map its correspondences with <strong>Gary Bernhardt's</strong> thin imperative shell around a functional core, and you get an understanding of how to cheaply maintain and scale software!</p>
<p>This is what <a href="https://rhodesmill.org/brandon/">Mr. Brandon Rhodes</a> did. It's not every day that I find such clear insight.</p>
<p>I …</p><p>Take <strong>Uncle Bob's</strong> Clean Architecture and map its correspondences with <strong>Gary Bernhardt's</strong> thin imperative shell around a functional core, and you get an understanding of how to cheaply maintain and scale software!</p>
<p>This is what <a href="https://rhodesmill.org/brandon/">Mr. Brandon Rhodes</a> did. It's not every day that I find such clear insight.</p>
<p>I am honored to have found his <a href="https://rhodesmill.org/brandon/talks/#clean-architecture-python">presentation</a> and <a href="https://rhodesmill.org/brandon/slides/2014-07-pyohio/clean-architecture/">slides</a> explaining <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">Uncle Bob's Clean Architecture</a> and Gary Bernhardt's PyCon talks of <a href="https://archive.org/details/pyvideo_422___units-need-testing-too">2011</a>, <a href="https://pycon-2012-notes.readthedocs.io/en/latest/fast_tests_slow_tests.html">2012</a>, and <a href="https://www.destroyallsoftware.com/talks/boundaries">2013</a>.</p>
<p>Mr. Rhodes offers such a distilled view, that he can show you these crucial concepts in 3 slides of code. I will go ahead and summarize what he said and add a tiny bit of my insight.</p>
<p>Copyright of all Python code on this page belongs to <a href="https://rhodesmill.org/brandon/">Mr. Brandon Rhodes</a>, and copyright of the diagram belongs to <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">Robert C. Martin (Uncle Bob)</a>. Edit 2020-11-17: I use these with permission from the authors - they are not under the same CC-BY license as the rest of my blog.</p>
<p>Mr. Rhodes warmly recommends the Harry Percival and Bob Gregory book <strong>Architecture Patterns with Python</strong>, also known as <a href="https://www.cosmicpython.com/">"<strong>Cosmic Python</strong>"</a>. It also talks about the Clean Architecture. You can do any or all of the following:</p>
<ul>
<li>Buy it as a DRM-free ebook <a href="https://www.ebooks.com/en-us/book/209971850/architecture-patterns-with-python/harry-percival/">on ebooks.com</a></li>
<li>Order it from <a href="https://www.amazon.com/Architecture-Patterns-Python-Domain-Driven-Microservices/dp/1492052205/">Amazon.com</a> or <a href="https://www.amazon.co.uk/Enterprise-Architecture-Patterns-Python-Adapters/dp/1492052205/">Amazon.co.uk</a></li>
<li>Read it for free on the <a href="https://www.cosmicpython.com/">Cosmic Python website</a> (it is under a CC-By-NC-ND license).</li>
<li>Hack the source code or fix typos on <a href="https://github.com/cosmicpython/book">GitHub</a></li>
</ul>
<p>I can't wait to read it myself.</p>
<div class="toc">
<ul>
<li><a href="#glossary">Glossary</a></li>
<li><a href="#listing-1-complex-code">Listing #1: Complex code</a></li>
<li><a href="#listing-2-hiding-io-at-the-bottom">Listing #2: Hiding I/O at the bottom</a></li>
<li><a href="#listing-3-io-at-the-top">Listing #3: I/O at the top</a></li>
<li><a href="#functional-core-imperative-shell-vs-clean-architecture">Functional Core / Imperative Shell vs. Clean Architecture</a></li>
<li><a href="#benefits">Benefits</a></li>
</ul>
</div>
<h1 id="glossary"><a class="toclink" href="#glossary">Glossary</a></h1>
<p>First of all, we need to be on the same page, in order to be able to understand each other. Here are the words I'll use:</p>
<ul>
<li>Function: I use "function" or "pure function" to refer to a Python "function" that only uses its parameters for input, returns a result as output, and does not cause any other side-effects (such as I/O). <ul>
<li>A pure function returns the same output given the same inputs.</li>
<li>A pure function may be called any number of times without changing the system state - it should have no influence on DB, UI, other functions or classes.</li>
<li>This is very similar to a mathematical function: takes you from <em>x</em> to <em>y</em> and nothing else happens.</li>
<li>Sadly we can't have only pure functions; software has a <strong>purpose</strong> of causing side-effects.</li>
</ul>
</li>
<li>Procedure, Routine, or Subroutine: A piece of code that executes, that may or may not have side effects. This is a "function" in Python, but might not be a "pure function".</li>
<li>Tests: automated unit tests. By "unit" I mean not necessarily just a class, but a behavior. If you want, see more details in <a href="tdd-revisited-pytest-updated-2020-09-03.html#update-2020-09-03-keep-coupling-low">the coupling chapter of my previous post</a>.</li>
</ul>
<h1 id="listing-1-complex-code"><a class="toclink" href="#listing-1-complex-code">Listing #1: Complex code</a></h1>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">requests</span> <span class="c1"># Listing 1</span>
<span class="kn">from</span> <span class="nn">urllib</span> <span class="kn">import</span> <span class="n">urlencode</span>
<span class="k">def</span> <span class="nf">find_definition</span><span class="p">(</span><span class="n">word</span><span class="p">):</span>
<span class="n">q</span> <span class="o">=</span> <span class="s1">'define '</span> <span class="o">+</span> <span class="n">word</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://api.duckduckgo.com/?'</span>
<span class="n">url</span> <span class="o">+=</span> <span class="n">urlencode</span><span class="p">({</span><span class="s1">'q'</span><span class="p">:</span> <span class="n">q</span><span class="p">,</span> <span class="s1">'format'</span><span class="p">:</span> <span class="s1">'json'</span><span class="p">})</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="c1"># I/O</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span> <span class="c1"># I/O</span>
<span class="n">definition</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="sa">u</span><span class="s1">'Definition'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">definition</span> <span class="o">==</span> <span class="sa">u</span><span class="s1">''</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'that is not a word'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">definition</span>
</code></pre></div>
<p>Here, we have a piece of code that prepares a URL, then gets some data over the network (I/O), then validates the result (a word definition) and returns it.</p>
<p>This is a bit much: a procedure should ideally do one thing only. While this small-ish procedure is quite readable still, it is a metaphor for a more developed system - where it could be arbitrarily long.</p>
<p>The current knee-jerk reaction is to <em>hide</em> the I/O operations somewhere far away. Here is the same code after extracting the I/O lines:</p>
<h1 id="listing-2-hiding-io-at-the-bottom"><a class="toclink" href="#listing-2-hiding-io-at-the-bottom">Listing #2: Hiding I/O at the bottom</a></h1>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">find_definition</span><span class="p">(</span><span class="n">word</span><span class="p">):</span> <span class="c1"># Listing 2</span>
<span class="n">q</span> <span class="o">=</span> <span class="s1">'define '</span> <span class="o">+</span> <span class="n">word</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://api.duckduckgo.com/?'</span>
<span class="n">url</span> <span class="o">+=</span> <span class="n">urlencode</span><span class="p">({</span><span class="s1">'q'</span><span class="p">:</span> <span class="n">q</span><span class="p">,</span> <span class="s1">'format'</span><span class="p">:</span> <span class="s1">'json'</span><span class="p">})</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">call_json_api</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">definition</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="sa">u</span><span class="s1">'Definition'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">definition</span> <span class="o">==</span> <span class="sa">u</span><span class="s1">''</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'that is not a word'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">definition</span>
<span class="k">def</span> <span class="nf">call_json_api</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="c1"># I/O</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span> <span class="c1"># I/O</span>
<span class="k">return</span> <span class="n">data</span>
</code></pre></div>
<p>In Listing #2, the I/O is extracted from the top-level procedure. </p>
<p>The problem is, the code is still <strong>coupled</strong> - <code>call_json_api</code> is called whenever you want to test anything - even the building of the URL or the parsing of the result.</p>
<p><strong>Coupling kills software.</strong></p>
<p>A good rule of thumb to spot coupling is this: Can you test a piece of code without having to mock or dependency inject like Frankenstein?</p>
<p>Here, we can't test <code>find_definition</code> without somehow replacing <code>call_json_api</code> from inside it, in order to avoid making HTTP requests.</p>
<p>Let's find out what a better solution looks like.</p>
<h1 id="listing-3-io-at-the-top"><a class="toclink" href="#listing-3-io-at-the-top">Listing #3: I/O at the top</a></h1>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">find_definition</span><span class="p">(</span><span class="n">word</span><span class="p">):</span> <span class="c1"># Listing 3</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">build_url</span><span class="p">(</span><span class="n">word</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">json</span><span class="p">()</span> <span class="c1"># I/O</span>
<span class="k">return</span> <span class="n">pluck_definition</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">build_url</span><span class="p">(</span><span class="n">word</span><span class="p">):</span>
<span class="n">q</span> <span class="o">=</span> <span class="s1">'define '</span> <span class="o">+</span> <span class="n">word</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'http://api.duckduckgo.com/?'</span>
<span class="n">url</span> <span class="o">+=</span> <span class="n">urlencode</span><span class="p">({</span><span class="s1">'q'</span><span class="p">:</span> <span class="n">q</span><span class="p">,</span> <span class="s1">'format'</span><span class="p">:</span> <span class="s1">'json'</span><span class="p">})</span>
<span class="k">return</span> <span class="n">url</span>
<span class="k">def</span> <span class="nf">pluck_definition</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="n">definition</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="sa">u</span><span class="s1">'Definition'</span><span class="p">]</span>
<span class="k">if</span> <span class="n">definition</span> <span class="o">==</span> <span class="sa">u</span><span class="s1">''</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s1">'that is not a word'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">definition</span>
</code></pre></div>
<p>Here, the procedure at the top (aka. the <span class="text-success"><strong>imperative shell</strong></span> of the program) is handling the I/O, and everything else is moved to <span class="text-danger"><strong>pure functions</strong></span> (<code>build_url</code>, <code>pluck_definition</code>). The <span class="text-danger"><strong>pure functions</strong></span> are easily testable by just calling them on made-up data structures; no Frankenstein needed.</p>
<p>This separation into an <span class="text-success"><strong>imperative shell</strong></span> and <span class="text-danger"><strong>functional core</strong></span> is an encouraged idea by Functional Programming.</p>
<p>Ideally, though, in a real system, you wouldn't test elements as small as these routines, but integrate more of the system. See <a href="tdd-revisited-pytest-updated-2020-09-03.html#update-2020-09-03-keep-coupling-low">the coupling chapter of my previous post</a> to understand the trade-offs.</p>
<h1 id="functional-core-imperative-shell-vs-clean-architecture"><a class="toclink" href="#functional-core-imperative-shell-vs-clean-architecture">Functional Core / Imperative Shell vs. Clean Architecture</a></h1>
<p>Look at <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">Uncle Bob's Clean Architecture chart</a> (Copyright Robert C. Martin aka. Uncle Bob) :
<img alt="The Clean Architecture" class="img-fluid" src="images/CleanArchitecture.jpg"/></p>
<p>Uncle Bob's <span class="text-danger"><strong>Use Cases</strong></span> and <span class="text-warning"><strong>Entities</strong></span> (red and yellow circles of the chart) map to the <span class="text-danger"><strong>pure functions</strong></span> we saw earlier - <code>build_url</code> and <code>pluck_definition</code> from Listing 3, and the <span class="text-warning"><strong>plain objects</strong></span> they receive as parameters and send as outputs. <em>(updated 2020-10-28)</em></p>
<p>Uncle Bob's <span class="text-success"><strong>Interface Adapters</strong></span> (green circle) map to the top-level <span class="text-success"><strong>imperative shell</strong></span> from earlier - <code>find_definition</code> from Listing 3, handling only I/O to the outside (Web, DB, UI, other frameworks).</p>
<p><a href="https://www.reddit.com/r/programming/comments/jj7ave/the_grand_unified_theory_of_software_architecture/gabst6z/?context=3">Update 2020-10-28</a>: A "Model" object in today's MVC frameworks is a poisoned apple: it is not a <a href="https://khanlou.com/2014/12/pure-objects/">"pure" object</a> or <a href="http://xunitpatterns.com/Humble%20Object.html">"humble" object</a>, but one that can produce side effects like saving or loading from the database. Their "save" and "read" methods litter your code with untestable side-effects all over. Avoid them, or confine them to the periphery of your system and reduce their influence accordingly (they are actually a hidden <span class="text-success"><strong>Interface Adapter</strong></span>) due to interacting with the DB.</p>
<p>Notice the arrows on the left side of the circles, pointing inwards to more and more abstract parts. These are procedure or function calls. Our code is called by the outside. <strong>This has some exceptions. Whatever you do, the database won't call your app. But the web can, a user can through a UI, the OS can through STDIN, and a timer can, at regular intervals (such as in a game).</strong> <em>(updated 2020-10-28)</em></p>
<p>The top-level procedure:</p>
<ol>
<li>gets the input, </li>
<li>adapts it to simple objects acceptable to the system,</li>
<li>pushes it through the functional core,</li>
<li>gets the returned value from the functional core,</li>
<li>adapts it for the output device,</li>
<li>and pushes it out to the output device.</li>
</ol>
<p>This lets us easily test the functional core. Ideally, most of a production system should be pure-functional.</p>
<h1 id="benefits"><a class="toclink" href="#benefits">Benefits</a></h1>
<p>If you reduce the <span class="text-success"><strong>imperative shell</strong></span> and move code into the <span class="text-danger"><strong>functional core</strong></span>, each test can verify almost the entire (now-functional) stack, but stopping short of actually performing external actions.</p>
<p>You can then test the imperative shell using <strong>fewer integration tests</strong>: you only need to check that it is <strong>correctly connected</strong> to the functional core.</p>
<p>Having two users for the system - the real user and the unit tests - and listening to both, lets you guide your architecture so as to <strong>minimize coupling</strong> and build a more <strong>flexible system</strong>.</p>
<p>Having a flexible system lets you implement new features and change existing ones <strong>quickly and cheaply</strong>, in order to <strong>stay competitive as a business</strong>.</p>
<p>Comments are much appreciated. I am yet to apply these insights, and I may be missing something!</p>
<p><strong>Edit 2020-10-28:</strong> I have tried out this methodology in some small TDD Katas, and together with TDD, it works great. But I am not employed right now, so I can't say I've <em>really</em> tried it.</p>Installing Isso on Debian & Apache2020-09-01T12:15:00+03:002020-09-01T12:15:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-09-01:/installing-isso-on-debian-apache.html<p><a href="https://posativ.org/isso/">Isso</a> is a self-hosted alternative to Disqus (comments on websites). </p>
<p>I installed it. You can comment on my blog without loading third party sites or creating any accounts now. You can even comment anonymously. Please don't abuse this, or I will turn it off.</p>
<div class="toc">
<ul>
<li><a href="#installing-the-server">Installing the server</a><ul>
<li><a href="#importing-old-comments">Importing old comments …</a></li></ul></li></ul></div><p><a href="https://posativ.org/isso/">Isso</a> is a self-hosted alternative to Disqus (comments on websites). </p>
<p>I installed it. You can comment on my blog without loading third party sites or creating any accounts now. You can even comment anonymously. Please don't abuse this, or I will turn it off.</p>
<div class="toc">
<ul>
<li><a href="#installing-the-server">Installing the server</a><ul>
<li><a href="#importing-old-comments">Importing old comments</a></li>
<li><a href="#apache">Apache</a></li>
</ul>
</li>
<li><a href="#installing-the-client">Installing the client</a></li>
<li><a href="#rant-on-js">Rant on JS</a></li>
<li><a href="#no-e-mail-notification-of-reply">No e-mail notification of reply</a></li>
<li><a href="#legal-issues">Legal issues</a></li>
</ul>
</div>
<h1 id="installing-the-server"><a class="toclink" href="#installing-the-server">Installing the server</a></h1>
<p>See: <a href="https://posativ.org/isso/docs/quickstart/">https://posativ.org/isso/docs/quickstart/</a></p>
<p>I modified the Debian <code>isso</code> package's systemd init script, <code>/lib/systemd/system/isso.service</code>, to start Isso's Python server directly, and not through GUnicorn, which I could not get working easily. This loses performance (I just have one worker thread, this means ~40 requests/second tops; totally not enough for my high-traffic site).</p>
<p>(You can find the location of the systemd script and anything else installed by the Debian package using <code>dpkg -L isso | less</code>).</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Enable automatic startup on system boot using systemd</span>
$<span class="w"> </span>sudo<span class="w"> </span>systemctl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>isso
<span class="c1"># Start it right now also</span>
$<span class="w"> </span>sudo<span class="w"> </span>systemctl<span class="w"> </span>start<span class="w"> </span>isso
</code></pre></div>
<p>I preferred Debian's package because it conveniently creates an <code>isso</code> user. This was the only one with write rights to the comment DB. Still, e-mails are sensitive data, so I changed it so that no users other than <code>isso</code> and <code>root</code> have read rights either:
<code>sudo chmod 600 /var/lib/isso/comments.db</code></p>
<p>The Debian package config is in <code>/etc/isso.d/enabled/</code>. But that is an empty directory, somehow to be interpreted by GUnicorn. I also hammered the SystemD service to use a fixed file <code>/etc/isso.d/enabled/isso.cfg</code>. It's not like you have tens of configs.</p>
<p>My config, in order to have the /isso/ virtual subdir instead of a subdomain which would require a separate SSL certificate:</p>
<div class="highlight"><pre><span></span><code>[general]
; database location, check permissions, automatically created if not exists
dbpath = /var/lib/isso/comments.db
; your website or blog (not the location of Isso!)
host =
http://danuker.go.ro
https://danuker.go.ro
[server]
listen = http://localhost:1550
public-endpoint = https://danuker.go.ro/isso/
reload = off
profile = off
[guard]
enabled = true
ratelimit = 2
direct-reply = 5
reply-to-self = false
require-author = false
require-email = false
[hash]
salt = <censored>
algorithm = pbkdf2
[admin]
enabled = true
password = <censored>
</code></pre></div>
<h2 id="importing-old-comments"><a class="toclink" href="#importing-old-comments">Importing old comments</a></h2>
<p>I decided to import the old comments from Disqus. Given that the original users wanted to publish them, it is clear they would have wanted this as well.</p>
<div class="highlight"><pre><span></span><code>scp<span class="w"> </span>-P<span class="w"> </span><span class="m">22</span><span class="w"> </span>-r<span class="w"> </span>disqus-export.xml<span class="w"> </span>dan@danuker.go.ro:/home/dan/disqus-export.xml
mosh<span class="w"> </span>dan@danuker.go.ro
<span class="c1"># (enter password)</span>
sudo<span class="w"> </span>su
<span class="c1"># (enter password again)</span>
isso<span class="w"> </span>-c<span class="w"> </span>/etc/isso.d/enabled/isso.cfg<span class="w"> </span>import<span class="w"> </span>/home/dan/disqus-export.xml
</code></pre></div>
<h2 id="apache"><a class="toclink" href="#apache">Apache</a></h2>
<p>I use an Apache server instead of the illustrated Nginx. I had to learn to use the <code>ProxyPass</code> and <code>ProxyPassReverse</code> directives in my <code>/etc/apache2/sites-enabled/</code> configs. In case you want its endpoint to be on <code>/isso/</code> and run the isso server on port 1550, Here they are:</p>
<div class="highlight"><pre><span></span><code><span class="gh"># Proxy for Isso commenting</span>
ProxyPass /isso/ http://localhost:1550/
ProxyPassReverse /isso/ http://localhost:1550/
</code></pre></div>
<h1 id="installing-the-client"><a class="toclink" href="#installing-the-client">Installing the client</a></h1>
<p>Integration in your website (without NodeJS/Bower/NPM optimization crud). I just hammered this into the comment section of the <code>pelican-bootstrap3</code> theme:</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">script</span> <span class="na">data-isso</span><span class="o">=</span><span class="s">"isso/"</span>
<span class="na">data-isso-reply-to-self</span><span class="o">=</span><span class="s">"false"</span>
<span class="na">data-isso-require-author</span><span class="o">=</span><span class="s">"false"</span>
<span class="na">data-isso-require-email</span><span class="o">=</span><span class="s">"false"</span>
<span class="na">data-isso-avatar</span><span class="o">=</span><span class="s">"true"</span>
<span class="na">data-isso-avatar-bg</span><span class="o">=</span><span class="s">"#f0f0f0"</span>
<span class="na">data-isso-vote</span><span class="o">=</span><span class="s">"true"</span>
<span class="na">data-isso-feed</span><span class="o">=</span><span class="s">"false"</span>
<span class="na">src</span><span class="o">=</span><span class="s">"issojs/embed.js"</span><span class="p">></</span><span class="nt">script</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">id</span><span class="o">=</span><span class="s">"isso-thread"</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
</code></pre></div>
<h1 id="rant-on-js"><a class="toclink" href="#rant-on-js">Rant on JS</a></h1>
<p>I did not want to use any NodeJS/Bower/NPM package managers. I don't want Bower and NPM and browserify and whatnot for a script that validates 3 forms and sends a post request. To me they look like trojan horses. Sure, PyPi is not much different, but Python includes <a href="https://www.theregister.com/2016/03/23/npm_left_pad_chaos/">string padding</a> without needing to install another shady 3rd party dependency that could go rogue at any moment.</p>
<p>Luckily, the Debian and PyPi packages contain the standalone embeddable JS files, with all dependencies included.</p>
<h1 id="no-e-mail-notification-of-reply"><a class="toclink" href="#no-e-mail-notification-of-reply">No e-mail notification of reply</a></h1>
<p>I didn't bother setting up e-mail notifications. If you want to see what people reply to your discussion, bookmark the page and visit it later.</p>
<h1 id="legal-issues"><a class="toclink" href="#legal-issues">Legal issues</a></h1>
<p>See all I have to say on the <a href="pages/privacy-statement.html">Privacy statement page</a>.</p>TDD Revisited - pytest (updated 2020-09-03)2020-08-31T11:46:00+03:002020-09-05T13:12:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-08-31:/tdd-revisited-pytest-updated-2020-09-03.html<p><em>Updated 2020-09-03: <a href="#update-2020-09-03-keep-coupling-low">Statement on coupling</a></em></p>
<p>I was recently blown away by watching some of <a href="http://www.cleancoder.com/products">Uncle Bob</a>'s Clean Code presentations (make sure to check out his <a href="http://blog.cleancoder.com/">blog</a>). Of particular focus was <a href="https://www.youtube.com/watch?v=58jGpV2Cg50&t=2628s">his demo of Test-Driven Development</a>, which when done right, reduces defects, development cost, and maintenance cost of software.</p>
<p>In …</p><p><em>Updated 2020-09-03: <a href="#update-2020-09-03-keep-coupling-low">Statement on coupling</a></em></p>
<p>I was recently blown away by watching some of <a href="http://www.cleancoder.com/products">Uncle Bob</a>'s Clean Code presentations (make sure to check out his <a href="http://blog.cleancoder.com/">blog</a>). Of particular focus was <a href="https://www.youtube.com/watch?v=58jGpV2Cg50&t=2628s">his demo of Test-Driven Development</a>, which when done right, reduces defects, development cost, and maintenance cost of software.</p>
<p>In addition, it might even nudge the software architecture into a better one, by exposing awkward-to-handle points and letting you easily refactor.</p>
<div class="toc">
<ul>
<li><a href="#the-rules-of-tdd">The rules of TDD</a></li>
<li><a href="#uncle-bobs-observations-that-interest-me">Uncle Bob's observations that interest me</a></li>
<li><a href="#update-2020-09-03-keep-coupling-low">Update 2020-09-03: Keep coupling low</a></li>
<li><a href="#pytest">pytest</a></li>
</ul>
</div>
<h1 id="the-rules-of-tdd"><a class="toclink" href="#the-rules-of-tdd">The rules of TDD</a></h1>
<p><a href="https://web.archive.org/web/20200618002844/butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd">The rules</a> are as follows:</p>
<ol>
<li>You are not allowed to write any production code unless it is to make a failing unit test pass.</li>
<li>You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.</li>
<li>You are not allowed to write any more production code than is sufficient to pass the one failing unit test.</li>
</ol>
<h1 id="uncle-bobs-observations-that-interest-me"><a class="toclink" href="#uncle-bobs-observations-that-interest-me">Uncle Bob's observations that interest me</a></h1>
<ul>
<li>Only use as many brain cells as necessary each round. Beat around the golden standard (final implementation) instead of going straight for it: test everything outside it first.<ul>
<li>This results in comprehensive coverage, that would catch anything that may go wrong.</li>
</ul>
</li>
<li>At each TDD increment, make tests more specific, and implementation more general<ul>
<li>Failing to do so might result in coupling your tests with the implementation. This is bad because it makes for brittle tests: any time you change the implementation, you need to change the tests. Make sure to test through the interfaces - such as the less-changing public method signatures, and not private methods or fields of objects.</li>
</ul>
</li>
<li>You will often find that as you implement the "trivial" tests, you'd already done a lot of the work.</li>
<li>If a component is hard to test, you might find that it's also badly designed. Try to split it up, make sure to use small, testable units right up to the untestable UI / DB / Web / other I/O endpoints.<ul>
<li>DB and front-ends are notoriously hard to test. To work around this, make sure you push all the logic into testable chunks, and you only use the endpoints as dumb display/storage/input devices.</li>
<li>Uncle Bob gives an example about UIs: have "Presenters" that are testable, and whose only role is to format the data into "Response Models" to be given to a front-end view (plain objects with mostly strings to display, but also enabled/disabled flags, colors if they change, or coordinates if you're in a game) - so the view has so little code in it that it does not need automated testing.</li>
</ul>
</li>
<li>Use unit tests instead of integration tests.<ul>
<li>Integration tests are notoriously slow, requiring servers, database, heavy browsers (such as Selenium), or even a web framework (routing, middleware, complex models). Slow tests lead to you running them much less often - which make them much less useful. Unit tests which run in a few seconds at most and give you enough confidence in the code let you constantly refactor the code as needed - every TDD round.</li>
</ul>
</li>
</ul>
<p>If you use TDD and get enough coverage, you can fearlessly refactor any part of your system. "Testable" is a synonym for "decoupled", and decoupled code is easy to adapt and maintain.</p>
<h1 id="update-2020-09-03-keep-coupling-low"><a class="toclink" href="#update-2020-09-03-keep-coupling-low">Update 2020-09-03: Keep coupling low</a></h1>
<p>Only test the behavior of <strong>stable APIs, not methods and classes that are implementation details</strong>. Otherwise, you get <strong>brittle tests</strong> that need to be changed whenever the implementation needs changing.</p>
<p>When your tests are brittle, it's because they are coupled to the implementation. <strong>Coupling kills software.</strong></p>
<p>I remembered from my last job that we tested lots of things. The tests that came out were only on the (tiny) methods that did actual computation. Surely enough, we needed to replace them. This made me look more into what a "unit" is and where should tests be.</p>
<p>Testing like this is wrong, and methods and classes are not "units". <strong>A (business) use case is a unit. A module is a unit.</strong> A high-level class expressed in almost-English might be a unit.</p>
<p>You <strong>should</strong> test for the presence of <strong>detailed</strong> behavior that does the right thing in depth, but only <strong>through stable interfaces</strong>. This way, the tests will need modification only when the stable interfaces also need it - that is, rarely.</p>
<p>Still, as per "unit tests instead of integration tests", you should not hit the <strong>slow DB, Web, framework middleware, or UI</strong> during unit tests. You should isolate your code from DB/UI/Web frameworks, so that the machine can execute it quickly.</p>
<p>This isolation is not easy, especially in today's framework-does-all environments, but following <a href="#uncle-bobs-observations-that-interest-me">Uncle Bob's tips above</a> might be helpful: doing-all is machine-costly.</p>
<p>Test as far as you can reach on the edge of your system, while keeping tests fast and isolated. This would be almost like integration tests, but don't touch the endpoints (or touch them very lightly, such as through an in-memory DB that can be easily reset between cases, instead of a real DB server). </p>
<p>Update 2020-09-05: Still, if you find that the tests are not local enough to the errors, and half of your tests fail on a bug in a component, making it harder to find the component, you may need to reduce their depth. <a href="https://archive.org/details/pyvideo_422___units-need-testing-too">Gary Bernhardt</a> considers a unit to be at most a 100-line class.</p>
<p>Unit test speed is paramount. If your tests are too slow (more than, say a few seconds), then you will not be able to use them to <strong>make decisions while adding features and refactoring</strong>.</p>
<p>More is spoken <a href="https://www.youtube.com/watch?v=EZ05e7EMOLM">here by Ian Cooper - TDD, Where Did It All Go Wrong?</a>. Ian Cooper's talk is inspired from Kent Beck's book on TDD in 2002 - and Ian claims this book is all you need to understand TDD.</p>
<h1 id="pytest"><a class="toclink" href="#pytest">pytest</a></h1>
<p>All this excitement led me to try out various testing frameworks. The first one I tried was <code>unittest</code> which comes with the Python default library:</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">unittest</span>
<span class="k">class</span> <span class="nc">StackTest</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">s</span> <span class="o">=</span> <span class="n">Stack</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_new_stack_is_empty</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertTrue</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">s</span><span class="o">.</span><span class="n">is_empty</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">test_pushed_stack_is_not_empty</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">s</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertFalse</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">s</span><span class="o">.</span><span class="n">is_empty</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">test_empty_pop_error</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertRaises</span><span class="p">(</span><span class="n">Stack</span><span class="o">.</span><span class="n">Underflow</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">s</span><span class="o">.</span><span class="n">pop</span><span class="p">)</span>
</code></pre></div>
<p>I did not enjoy the verbosity of it: <code>self.assertTrue</code> and <code>self.assertFalse</code> calls. There must be a better way.</p>
<p>Also, while it does provide a <code>setUp</code> method, I then had to use <code>self.s</code> everywhere, because it was an instance variable. </p>
<p>I then tried out another framework: <code>pytest</code>:</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>pytest
$<span class="w"> </span>pytest<span class="w"> </span>stack.py
</code></pre></div>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">pytest</span>
<span class="k">def</span> <span class="nf">test_new_stack_is_empty</span><span class="p">():</span>
<span class="k">assert</span> <span class="n">Stack</span><span class="p">()</span><span class="o">.</span><span class="n">is_empty</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_stack_not_empty_after_push</span><span class="p">():</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">Stack</span><span class="p">()</span>
<span class="n">s</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="k">assert</span> <span class="ow">not</span> <span class="n">s</span><span class="o">.</span><span class="n">is_empty</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_stack_empty_after_push_pop</span><span class="p">():</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">Stack</span><span class="p">()</span>
<span class="n">s</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="n">s</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
<span class="k">assert</span> <span class="n">s</span><span class="o">.</span><span class="n">is_empty</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">test_can_not_pop_empty</span><span class="p">():</span>
<span class="n">s</span> <span class="o">=</span> <span class="n">Stack</span><span class="p">()</span>
<span class="k">with</span> <span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="n">Stack</span><span class="o">.</span><span class="n">Underflow</span><span class="p">):</span>
<span class="n">s</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span>
</code></pre></div>
<p>For this particular trial, I gave up on the <code>setUp</code> method. I preferred saying <code>s = Stack()</code> in each test case than having to create a class (like <code>StackTest</code>) with a constructor and then using <code>self.s</code> instead of just <code>s</code> everywhere.</p>
<p>I could also have avoided the <code>self.</code> prefixes in <code>unittest</code>, but I would still need to create the class (for the <code>self.assertTrue</code> stuff).</p>
<p>In any case, now I can just say <code>assert X</code> instead of <code>self.assertTrue(X)</code>. It prints out magnificently, showing you the exact source line, and its evaluated assertion values.</p>
<p>I vastly prefer <code>pytest</code> to <code>unittest</code> and warmly recommend it to you, if you use Python!</p>
<p>Cheers!</p>Cryptocurrency security2020-07-09T00:00:00+03:002020-07-09T00:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-07-09:/cryptocurrency-security.html<p>In my <a href="review-of-cryptocurrencies.html">review of cryptocurrencies</a>, I mentioned that Bitcoin Cash offers "relatively less security compared to Bitcoin".</p>
<p>What did I mean by that?</p>
<p>What are the various kinds of security that a cryptocurrency offers? How to enhance them and avoid compromising them?</p>
<p>I will address the following questions:</p>
<div class="toc">
<ul>
<li><a href="#what-does-security-mean-in-a-cryptocurrency">What does …</a></li></ul></div><p>In my <a href="review-of-cryptocurrencies.html">review of cryptocurrencies</a>, I mentioned that Bitcoin Cash offers "relatively less security compared to Bitcoin".</p>
<p>What did I mean by that?</p>
<p>What are the various kinds of security that a cryptocurrency offers? How to enhance them and avoid compromising them?</p>
<p>I will address the following questions:</p>
<div class="toc">
<ul>
<li><a href="#what-does-security-mean-in-a-cryptocurrency">What does security mean in a cryptocurrency?</a></li>
<li><a href="#counterparty-risk">Counterparty risk</a></li>
<li><a href="#private-key-security">Private key security</a></li>
<li><a href="#private-computer-security">Private computer security</a></li>
<li><a href="#systemic-security">Systemic security</a><ul>
<li><a href="#how-to-quantify-this-security">How to quantify this security?</a></li>
<li><a href="#is-it-worth-paying-the-transaction-fees-to-get-the-security">Is it worth paying the transaction fees to get the security?</a></li>
</ul>
</li>
<li><a href="#non-proof-of-work-currencies">Non proof-of-work currencies</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
</div>
<p><a href="https://www.flickr.com/photos/151691693@N02/38747838010"><img alt='"cryptocurrency" by stockcatalog is licensed under CC BY 2.0 ' class="img-fluid" src="images/cryptocurrency-stockcatalog.jpg"/></a></p>
<p>"cryptocurrency" by stockcatalog is licensed under CC BY 2.0 </p>
<h1 id="what-does-security-mean-in-a-cryptocurrency"><a class="toclink" href="#what-does-security-mean-in-a-cryptocurrency">What does security mean in a cryptocurrency?</a></h1>
<p>Among the disadvantages of cryptocurrencies (<a href="intro-to-cryptocurrencies.html">mentioned here</a>), there are security risks, and they come in many flavors.</p>
<p>The concept of security is only useful against specific risks. </p>
<p>Securing a piece of information is called "Operations Security", or <a href="https://en.wikipedia.org/wiki/Operations_security">OpSec</a>, and it involves thinking about who could or would like to find out that information, and how to prevent them.</p>
<p>Security against unknown risks is much harder to think about. You could enumerate many risks, but there's no guarantee that you'll think of the one that occurs.</p>
<h1 id="counterparty-risk"><a class="toclink" href="#counterparty-risk">Counterparty risk</a></h1>
<p>This is the risk that the person you're dealing with does not fulfill their end of the deal. Say, you send them your Dogecoin, but they don't send you the product you're buying. This is a form of fraud.</p>
<p>This can be mitigated by:</p>
<ul>
<li>Trading with people you trust</li>
<li>Trading with people you have repeat business with (so their interest is not to end the profitable relationship)</li>
<li>Breaking up a trade in small pieces (so that you only lose a small amount if the counterparty decides to defraud you)</li>
<li>Making sure both parties understand the deal clearly before anything changes hands</li>
<li>Using the services of a trusted mediator or arbiter<ul>
<li>For example, the Bisq decentralized exchange <a href="https://docs.bisq.network/trading-rules#dispute-resolution">offers decentralized dispute resolution</a>. The arbiter used to have the power to settle a transaction, but not anymore, because of <a href="https://en.wikipedia.org/wiki/Collusion">collusion</a>. As of now, in the case of a misunderstanding, the traders will have their transaction locked until they can agree who pays how much.</li>
</ul>
</li>
</ul>
<h1 id="private-key-security"><a class="toclink" href="#private-key-security">Private key security</a></h1>
<p>This is, perhaps, the riskiest part of owning a cryptocurrency. <a href="https://en.bitcoin.it/wiki/List_of_Major_Bitcoin_Heists,_Thefts,_and_Losses">People have lost a lot of cryptocurrency</a>.</p>
<p>Your private key is your password for spending your funds. Many wallet programs show you an ordered list of words ("passphrase") when creating a wallet. This is functionally equivalent to a cryptographic private key.</p>
<ul>
<li>If you <strong>lose</strong> your private key, you will not be able to spend your currency. This is by design, to avoid requiring trust in third-parties. Mitigation:<ul>
<li>Backing up your private key in multiple safe physical locations (consider floods, fires, even <a href="https://www.universityproducts.com/temperature-and-humidity/">temperature and humidity spoiling your paper</a>)</li>
<li>Trusting a third-party <a href="https://en.bitcoin.it/wiki/Multisignature">via a multisig</a> (but this is riskier and more complicated in my opinion).</li>
</ul>
</li>
<li>If <strong>someone else finds</strong> your private key, they are able to spend your funds. <ul>
<li>Keep your private key hidden at all times and do not send it anywhere. Note that taking a photo with a phone usually sends it to Google or Apple, and your data security is in <a href="https://en.wikipedia.org/wiki/2018_Google_data_breach">their</a> <a href="https://techcrunch.com/2021/09/13/apple-zero-day-nso-pegasus/">hands</a>.</li>
<li>Keep the computer you use firewalled, exposed as little as possible to the Internet and to USB devices from strangers, and only visit sites and download software you trust. Computer security is complex - do not store funds you can't afford to lose if you can't figure out attack surfaces.</li>
<li>Use an offline password manager; I can recommend <a href="https://keepassxc.org/">KeePassXC</a>.</li>
<li>Keeping your money on an exchange is equivalent to letting the exchange have the private key (and you not having it). This defeats the purpose of owning cryptocurrencies. Withdraw from the exchange after trading. <a href="https://www.youtube.com/watch?v=5mcYQpHDgXc">Learn more here.</a></li>
<li>Keeping your money in a closed-source wallet is equivalent to letting the developer have the private key. In the particular case of Jaxx, <a href="https://vxlabs.com/2017/06/10/extracting-the-jaxx-12-word-wallet-backup-phrase/">it also used to let any other program read your private key</a>.</li>
<li>Even keeping your money on an open-source web-based wallet <a href="https://www.coindesk.com/150k-stolen-myetherwallet-users-dns-server-hijacking">is riskier</a> when the server is not on your machine.</li>
<li>Even using an open-source wallet has risks. <a href="https://github.com/spesmilo/electrum/issues/3374#issuecomment-355726294">Make sure not to expose the RPC ports you're using to attackers.</a>.</li>
</ul>
</li>
</ul>
<p>When holding significant funds that would hurt you if lost, you may want to have a hardware wallet like Trezor or Ledger, or perhaps a dedicated, always-offline computer to transact using (even with the network card removed). If you are in this situation, read about the <a href="https://glacierprotocol.org/">Glacier protocol</a>, learn about the motivations behind every step, and consider following its steps.</p>
<h1 id="private-computer-security"><a class="toclink" href="#private-computer-security">Private computer security</a></h1>
<p>Some malware gets around being unable to fish for your private key by <a href="https://techcrunch.com/2018/07/03/new-malware-highjacks-your-windows-clipboard-to-change-crypto-addresses/">replacing the address to send money in the clipboard</a> with a similar-looking one from the attacker. Always double-check the address before hitting "send".</p>
<p>Other malware logs your keys and eventually might learn your password.</p>
<p>Learn how your wallet stores the keys (it is not just the passphrase in a file), and make sure they are encrypted with a password.</p>
<p>Consider a hardware wallet (though I am not very comfortable with that device having update capabilities, nor do I trust the cryptographic security of its random number generator - I used dice, albeit not casino-grade as the <a href="https://glacierprotocol.org/">Glacier protocol</a> suggests).</p>
<h1 id="systemic-security"><a class="toclink" href="#systemic-security">Systemic security</a></h1>
<p>After you're reasonably secure locally, there comes another dimension of risk, that in the cryptocurrency system itself.</p>
<p>Many cryptocurrencies work via proof-of-work, which is a mechanism for offering an incentive for processing transactions, and for avoiding <a href="https://en.wikipedia.org/wiki/Double-spending">double-spending</a>. I will present how to mitigate the double-spending risk step by step.</p>
<p>In order to have a "permanent" transaction, it has to be included in a block with enough "confirmations". The block with the most "work" proven behind it will be the one people and systems will trust.</p>
<p>"Work" means hashing - the amount of cryptographic puzzles of a specific difficulty you solve.
"Power" means hashes per second - or speed of work. <a href="https://en.wikipedia.org/wiki/Power_(physics)">This analogous to the definition in Physics.</a></p>
<p>In order to perform a double-spend attack, you must:</p>
<ol>
<li>Start mining a hidden chain with majority hash power (at least 51% of your and the public chain's combined powers).</li>
<li>In your hidden chain, move some money to another address of your own.</li>
<li>In public, send money to your victim, and wait for the victim to accept the validated transaction and send you whatever you are buying.</li>
<li>After receiving the bought product, publish your hidden chain. </li>
</ol>
<p>Your chain should be longer than the old public one, since you had 51% of power, which is more than the rest of the network. This means the transaction to your victim is now part of a chain containing less work, and it will be abandoned. Your transaction in the freshly-published chain sending money to yourself will be accepted instead.</p>
<p>Why don't people do this all the time? Find out next.</p>
<h2 id="how-to-quantify-this-security"><a class="toclink" href="#how-to-quantify-this-security">How to quantify this security?</a></h2>
<p>The cost of overpowering the entire network is easily calculable, for example:</p>
<ul>
<li>based on hash rate, power usage, and electricity price</li>
<li>based on the cost of renting computing power for a 51% attack, which is what <a href="https://www.crypto51.app/">Crypto51</a> does.</li>
</ul>
<p>As you can see, an attacker would only find it profitable to perform such an attack when they could gain more than the cost of the attack.</p>
<p>If you wait more than an hour after receiving $432000 in Bitcoin in order to accept that payment, and there is no blockchain reorganization, you should be safe to give the buyer their product - if the attacker only targeted one person.</p>
<p>It would still be profitable for a centralized attacker to target multiple people at the same time. So, if an attacker could simultaneously double-spend on 1000 users, they would only need to make $432 from each user.</p>
<p>But such coordination could only be pulled off by big actors, and it might be against their interests, since such an attack would surely make the news and the big actor might be legally punished.</p>
<p>One mitigation is to divide the rate by a security margin, say 10. This would imply that the doublespender buying from you is also attacking 9 other people. That way, to judge how fast you can receive funds, divide the <a href="https://www.crypto51.app/">Crypto51</a> rate by 10. That way, there is an organizational estimate of security, instead of just the proof-of-work cost.</p>
<p>Bitcoin's $432000/hr attack protection would become $43200/hr for being safely confirmed by a recipient.</p>
<p>This means a block every 10 minutes would cost $43200 / 6 = $7200. If you receive less than $7200, one blockchain confirmation should suffice in order to accept the payment.</p>
<p>In any case, <a href="https://thebitcoinnews.com/video-shows-how-easy-it-is-to-double-spend-btc-using-rbf/">you should wait for at least one blockchain confirmation</a> if you do not trust the sender, and care about losing the money.</p>
<p>Another mitigation is to deal with people you trust, and who do not seem as part of an organization that could pull off such an attack. But who do you trust when dealing with such large amounts?</p>
<h2 id="is-it-worth-paying-the-transaction-fees-to-get-the-security"><a class="toclink" href="#is-it-worth-paying-the-transaction-fees-to-get-the-security">Is it worth paying the transaction fees to get the security?</a></h2>
<p>Bitcoin may be quite secure in this respect, but it also has <a href="https://coinmetrics.io/charts/#assets=btc,bch,eth,xmr,ltc,etc,btg,bsv,dash,zec_roll=7_left=FeeMedUSD_zoom=1562630400000,1594166400000_includeZero=true_crosshair=true">quite the transaction fees (roughly $0.63 at time of writing)</a>. This might make it impractical for microtransaction applications; but there is hope with the Lightning Network (a whole other article).</p>
<p>Let's look at Bitcoin, and its less-popular, but more-spacious cousin, Bitcoin Cash (ABC). I choose these two because they use the same hash algorithm, and the "difficulty" on the chart is proportional to the attack cost.</p>
<p>We compare how much <a href="https://coinmetrics.io/charts/#assets=btc,bch_left=DiffMean_zoom=1279411200000,1593475200000">difficulty</a> backs the network vs. how much <a href="https://coinmetrics.io/charts/#assets=btc,bch_left=FeeTotUSD">fees are paid to the miners</a>.</p>
<p>Bitcoin:</p>
<ul>
<li>$458027 in fees per day</li>
<li>15784G difficulty units</li>
</ul>
<p>Bitcoin Cash:</p>
<ul>
<li>$97 in fees per day</li>
<li>389G difficulty units</li>
</ul>
<p>A difficulty unit is equivalent to <a href="https://en.bitcoin.it/wiki/Difficulty#What_network_hash_rate_results_in_a_given_difficulty.3F">"around 7 Mhashes per second."</a></p>
<p>So, Bitcoin's security costs $458027 / 15784 = ~$29.02 per Gdiff. unit, while Bitcoin Cash's costs $97 / 389 = ~$0.25.</p>
<p>This means Bitcoin Cash offers more "bang-for-your-buck"; but it does not have a security as strict as Bitcoin's.</p>
<p>As you can see, when not many transactions are competing for Bitcoin Cash's more spacious blocks, the cost of operating the network is much smaller, and fewer miners will secure the network.</p>
<p>If you can live with the network requiring an attacker to make $9,839/hr from victims in order to keep staying profitable, then Bitcoin Cash may be for you. <a href="https://blog.coinbase.com/a-deep-dive-into-the-recent-bch-hard-fork-incident-2ee14132f435?gi=88b7092b9666">There has probably been a double-spend attack on Bitcoin Cash in May 2019.</a>.</p>
<p>If you require more security than $983/hour (at the moment of writing, considering the 10-victim coefficient), then you might want to use Bitcoin or Ethereum.</p>
<p>Beware of much-lower-hash-rate coins. For example, a double-spend on Bitcoin Gold only costs $305, in spite of it having a market cap of $156 million. Dividing by our coefficient again, this means a safe rate to receive would be around $30/hr.</p>
<h1 id="non-proof-of-work-currencies"><a class="toclink" href="#non-proof-of-work-currencies">Non proof-of-work currencies</a></h1>
<p>My analysis of systemic security only applies to coins using Proof of Work.</p>
<p>I am not knowledgeable enough to evaluate the security of Proof-of-Stake currencies in any way. Also, I do not trust what I don't understand. If you find any article explaining proof-of-stake to the same level of clarity, please share it.</p>
<h1 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h1>
<p>I hope you now have a clearer picture of cryptocurrencies and the risks of using them. You might still find their benefits worth it, as I do.</p>
<p>Remember: not your keys, not your coins.</p>
<p>Have fun and enjoy responsibly!</p>Don't use VPNs2020-06-28T20:50:00+03:002020-06-28T20:50:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-06-28:/dont-use-vpns.html<p>I argue that VPNs are bad for the user, especially if the user is a dissenter from the status quo.</p><p>I argue that VPNs are bad for the user, especially if the user is a dissenter from the status quo.</p>
<div class="toc">
<ul>
<li><a href="#vpns-present-a-centralized-point-of-monitoring">VPNs present a centralized point of monitoring</a></li>
<li><a href="#vpns-use-proprietary-software">VPNs use proprietary software</a></li>
<li><a href="#real-options-tor-i2p-freenet">Real options: Tor, I2P, Freenet</a><ul>
<li><a href="#tor-the-onion-router">Tor (The Onion Router)</a></li>
<li><a href="#i2p-invisible-internet-protocol">I2P (Invisible Internet Protocol)</a></li>
<li><a href="#freenet">Freenet</a></li>
</ul>
</li>
<li><a href="#closing">Closing</a></li>
</ul>
</div>
<h1 id="vpns-present-a-centralized-point-of-monitoring"><a class="toclink" href="#vpns-present-a-centralized-point-of-monitoring">VPNs present a centralized point of monitoring</a></h1>
<p>Similarly to your ISP or a <a href="how-to-protect-your-personal-data.html">CDN</a>, a VPN could just get handed a <a href="https://www.aclu.org/other/national-security-letters">National Security Letter</a> (or your state's equivalent) to start logging your traffic without letting you know. Of course many of them say they don't log users.</p>
<p>The government would pay more attention when monitoring VPNs in addition to, say, ISPs, because those users are more interested in privacy.</p>
<p>In addition, if you pay a VPN using a credit card, the payment company now knows you are up to no good, and <a href="https://news.gab.com/2020/06/19/gab-blacklisted-by-visa/">might blacklist you</a>.</p>
<h1 id="vpns-use-proprietary-software"><a class="toclink" href="#vpns-use-proprietary-software">VPNs use proprietary software</a></h1>
<p>Some VPN providers offer <a href="https://github.com/OpenVPN/openvpn">OpenVPN</a>, or some other free-software solution; but many use proprietary software. This has quite some implications:</p>
<ol>
<li>It is much more difficult to inspect what the software does. The software could be buggy or could spy on you.</li>
<li>Constant updates might introduce new issues even after you inspect an old version of the software.</li>
<li>The software might install new root certificates on your system, allowing them to impersonate or Man-in-the-Middle any website, like <a href="https://arstechnica.com/information-technology/2015/02/lenovo-pcs-ship-with-man-in-the-middle-adware-that-breaks-https-connections/">Lenovo's Superfish</a> (same is true of any otherwise-legitimate certificate authority).</li>
</ol>
<p>Depending on your skill, wealth, the importance, and the sensitivity of the information you are transferring, you might want to audit or inspect the source code of the software you will use, as well as build it from the inspected source, and not use binaries.</p>
<p>In addition, you need to seriously consider the trustworthiness of your service provider. One that pushes binary software that is meant to somehow offer you protection raises a red flag.</p>
<h1 id="real-options-tor-i2p-freenet"><a class="toclink" href="#real-options-tor-i2p-freenet">Real options: Tor, I2P, Freenet</a></h1>
<p><a href="https://commons.wikimedia.org/wiki/File:Mixed_onions.jpg"><img alt="Onions in the dark are good for you; CC-BY-SA Credit: Colin @ wikimedia.org" class="img-fluid" src="images/1024px-Mixed_onions.jpg"/></a></p>
<p>Onions in the dark are good for you; CC-BY-SA Credit: Colin @ wikimedia.org</p>
<h2 id="tor-the-onion-router"><a class="toclink" href="#tor-the-onion-router"><strong>Tor</strong> (The Onion Router)</a></h2>
<p><a href="https://www.torproject.org/">Tor</a> works by layering encrypted instructions forming a route of volunteers of the Tor network, eventually "exiting" to the open Internet, or reaching a hidden service. Each volunteer relay will decrypt the instructions targeted for them, and forward what's left to the next relay as per the instructions, without knowing where the final destination or payload is. Such decryption is analogous to peeling layers of an onion - hence the name.</p>
<p>It would be impractical to send National Security Letters to relays around the world; and they would likely not be bound by them. I believe you should choose an exit node, correspondent, and/or relays outside of your current jurisdiction (i.e. US -> EU, Russia, China, Iran...). (Export control lists are handy here). Still, a lot of network relays or exit nodes might be run by governments.</p>
<p>WikiLeaks <a href="https://www.wikileaks.org/#submit">has a Tor address</a> for the disclosure of leaks; which means this institution that leaked information about US war crimes trusts Tor. </p>
<p>Note that they have a long address; be wary of short addresses, since, for example, Facebook's vanity onion address is made up entirely of English words - facebookcorewwwi.onion . <a href="https://trac.torproject.org/projects/tor/wiki/doc/NextGenOnions">Version 3 onion addresses are longer and more secure</a>.</p>
<p><a href="https://www.torproject.org/download/">Tor Browser</a> is an easy way to become a lot harder to track, and is pretty well-tested, being perhaps the largest anonymity network.</p>
<h2 id="i2p-invisible-internet-protocol"><a class="toclink" href="#i2p-invisible-internet-protocol">I2P (Invisible Internet Protocol)</a></h2>
<p><a href="https://i2p.net/en/">I2P</a> works similarly to Tor, except that it uses <a href="https://en.wikipedia.org/wiki/Garlic_routing">garlic routing</a> - multiple messages ("bulbs" or "cloves") get grouped together, to make it supposedly harder to deanonymize.</p>
<p>I2P is a bit harder to use, since it is <a href="https://i2p.net/en/download">not so nicely-packaged as Tor Browser</a>, requiring you to run the Router manually, in addition to your browser.</p>
<p>Also, in order to access the open Internet, as opposed to anonymous I2P sites, you need an "Outproxy" of which there are quite few; whereas hidden services have better support.</p>
<p>Both of these networks have a vulnerability, because ISPs might statistically correlate the endpoints of traffic, at the request of government. For instance, they could single you out based on the exact timestamps of your requests, and notice that another computer has network activity with sufficiently similar timestamps (serving your requests).</p>
<p>This kind of surveillance can be performed by malicious intermediary network relays as well - perhaps that is why there was <a href="https://i2p-metrics.np-tokumei.net/router-distribution">a significant uptick in Chinese I2P routers starting June 2020</a>, right before US elections, for instance.</p>
<h2 id="freenet"><a class="toclink" href="#freenet">Freenet</a></h2>
<p><a href="https://freenetproject.org/"><strong>Freenet</strong></a> was created for more difficult situations, and does not have this specific weakness. Not only is Freenet supposed to offer some anonymity, but nodes also act as a data cache. This way, your content is available from many nodes, and you do not have to be online in order to serve it, and any surveillance will find it hard to pinpoint the original source of the content.</p>
<p>Most importantly, this ensures that popular-enough content is very censorship-resistant. But it has the disadvantage that dynamic content is much harder to implement.</p>
<h1 id="closing"><a class="toclink" href="#closing">Closing</a></h1>
<p>It is up to you to figure out the most appropriate technology for your speech.</p>
<p>It is a noble mission to expose corrupt governments that want you silenced, but it is also very risky. I hope you find these tools useful, and that you use them in a well-thought-out <a href="https://en.wikipedia.org/wiki/Operations_security">Operations Security</a> context. You also need a trustworthy operating system, hardware, network, and penpal.</p>
<p>Good luck!</p>Bitcoin price predictions2020-04-21T00:00:00+03:002020-04-21T00:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2020-04-21:/bitcoin-price-predictions.html<p>Knowing that the next Bitcoin "halvening" will shrink inflation to half yet again <a href="https://en.bitcoin.it/wiki/Controlled_supply#Projected_Bitcoins_Short_Term">next month</a> (I estimate it'll be the 23rd of May), I was curious to see what's in store for the price of Bitcoin.</p>
<p>Currently, the supply inflation is <a href="https://charts.woobull.com/bitcoin-inflation/">below 4% per year, according to this pretty chart …</a></p><p>Knowing that the next Bitcoin "halvening" will shrink inflation to half yet again <a href="https://en.bitcoin.it/wiki/Controlled_supply#Projected_Bitcoins_Short_Term">next month</a> (I estimate it'll be the 23rd of May), I was curious to see what's in store for the price of Bitcoin.</p>
<p>Currently, the supply inflation is <a href="https://charts.woobull.com/bitcoin-inflation/">below 4% per year, according to this pretty chart.</a> This means starting next month it'll be less than 2% per year. </p>
<p>This means it'll be quite better than say, holding Euros, which <a href="https://www.investing.com/economic-calendar/m3-money-supply-198">show a roughly 5% M3 supply increase per year</a>.</p>
<p>I discovered two analyses. I reproduced them and will plot both in the same chart, then explain them briefly. <a href="mailto:danuthaiduc@gmail.com">Mail me</a> and ask for the spreadsheet if you want it!</p>
<h3 id="toc"><a class="toclink" href="#toc">TOC</a></h3>
<div class="toc">
<ul>
<li><a href="#historic-price-versus-the-two-predictions">Historic price versus the two predictions</a></li>
<li><a href="#first-prediction-time-based-power-law">First prediction: Time-based power law</a></li>
<li><a href="#second-prediction-stock-to-flow-based-power-law">Second prediction: Stock-to-Flow-based power law</a></li>
<li><a href="#more-predictions">More predictions</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
</div>
<h1 id="historic-price-versus-the-two-predictions"><a class="toclink" href="#historic-price-versus-the-two-predictions">Historic price versus the two predictions</a></h1>
<p><a href="images/btc-predictions.svg"><img alt="Historic Bitcoin price versus the two predictions" class="img-fluid" src="images/btc-predictions.svg"/></a></p>
<h1 id="first-prediction-time-based-power-law"><a class="toclink" href="#first-prediction-time-based-power-law">First prediction: Time-based power law</a></h1>
<p>The <a href="https://medium.com/quantodian-publications/bitcoins-natural-long-term-power-law-corridor-of-growth-649d0e9b3c94">first prediction</a> fits a straight line to the log-log plot of Bitcoin price vs. time.</p>
<p>There's not a lot to it; it looks accurate, but is quite loose.</p>
<p>Fitting to the extreme points gives a "corridor", which predicts a price of $100k between 2021 and 2028, and a price between $6k and $90k, most likely about $10-20k, at the end of 2020.</p>
<h1 id="second-prediction-stock-to-flow-based-power-law"><a class="toclink" href="#second-prediction-stock-to-flow-based-power-law">Second prediction: Stock-to-Flow-based power law</a></h1>
<p>I learned about the Stock-to-Flow (S2F) ratio yesterday.</p>
<p>It is defined as Stock (total supply) divided by Flow (newly created supply in the past year).</p>
<p>This is the inverse of inflation, and it computes as the number of years it would take to re-create the supply, at current production.</p>
<p>Investors are attracted to stores of value. Stores of value are investments with high S2F, <a href="https://en.wikipedia.org/wiki/Store_of_value">among other properties</a>.</p>
<p>When inflation is zero, it means the current stock is all there is, and creation has stopped. This means the stock becomes rare, and perhaps more valuable. The simple S2F model predicts an infinite price here (which is not realistic). Still, it may be useful.</p>
<p>For comparison, there are about <a href="https://materials-risk.com/the-gold-stock-to-flow-model/">190k tons of gold mined, with 2.9k tons mined every year</a>. This means a S2F of ~65.5 years.</p>
<p>Bitcoin's inflation in the past year was about 3.87%, so the S2F was 1/0.0387 ~= 25.8 years. But next month, the S2F will roughly double, to roughly 51.6 years.</p>
<p>What's great about an <a href="https://en.bitcoin.it/wiki/Controlled_supply">algorithmic supply</a> is that inflation (and therefore S2F) is dictated, in the long run, by hard rules, making it predictable, and making the investment safer.</p>
<p>Lo and behold, <a href="https://medium.com/@100trillionUSD/modeling-bitcoins-value-with-scarcity-91fa0fc03e25">when fitting a power law of Price vs. S2F, you get a good fit</a>.</p>
<p>This model predicts a Bitcoin price of roughly <a href="https://digitalik.net/btc/">$100k in the next 4 years (May 2020 - May 2024)</a>. But my chart only shows about $50k; I guess it depends a lot on the fit.</p>
<h1 id="more-predictions"><a class="toclink" href="#more-predictions">More predictions</a></h1>
<p>Here's a <a href="https://blockonomi.com/bitcoin-price-prediction-2020">plethora of other predictions</a> people have made for the current year. Their <a href="https://en.wikipedia.org/wiki/Interquartile_range">IQR</a> is $14K to $250k.</p>
<p>(John McAfee <a href="http://dickening.com/">reneged on eating his own dick</a>, haha).</p>
<h1 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h1>
<p>While these models might not be very precise, they might be accurate. If you want to gamble your money away, do it informed!</p>
<p>Today's price of $7200 strikes me as very cheap compared to what seems to be quite the upside in the next years.</p>
<p>Still, the models diverge quite a lot past 2024. My S2F shows upwards of $500k, while the Time model shows just $100k in 2026. I am very curious what will happen.</p>
<p>In any case, your decisions are your sole responsibility, and not mine.</p>
<p>If you've got any other ideas of evaluating Bitcoin quantitatively, do send them to <a href="mailto:danuthaiduc@gmail.com">my e-mail</a>!</p>The story of Hong Kong: mapping search engine bias2020-03-18T00:00:00+02:002020-03-18T00:00:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2020-03-18:/the-story-of-hong-kong-mapping-search-engine-bias.html<p>A search engine has an immense influence upon the searcher. Let's explore the story of Hong Kong.</p>
<p>Hong Kong was a UK colony before being <a href="https://www.hklii.hk/eng/hk/legis/instrument/A301/declaration.html">ceded</a> back to China, who declared it will let Hong Kong "enjoy a high degree of autonomy, except in foreign and defence affairs which are …</p><p>A search engine has an immense influence upon the searcher. Let's explore the story of Hong Kong.</p>
<p>Hong Kong was a UK colony before being <a href="https://www.hklii.hk/eng/hk/legis/instrument/A301/declaration.html">ceded</a> back to China, who declared it will let Hong Kong "enjoy a high degree of autonomy, except in foreign and defence affairs which are the responsibilities of the Central People’s Government".</p>
<p>What's not to like? The human rights inspired by UK law, and the military defence of China. Standing on the shoulders of giants, Hong Kong became <a href="https://en.wikipedia.org/wiki/List_of_countries_by_Human_Development_Index#Countries">one of the best places to live in</a>.</p>
<p>But then, there came trouble.</p>
<h2 id="google"><a class="toclink" href="#google">Google</a></h2>
<p>Here is the first result I get right now, when Googling "hong kong protest". A BBC article from 2019-11-28:</p>
<blockquote>
<p><a href="https://www.bbc.com/news/world-asia-china-49317695">The Hong Kong protests explained in 100 and 500 words</a>.</p>
<p>Critics feared <em>[extradition to China]</em> could undermine judicial independence and endanger dissidents. [...]</p>
<p>Clashes between police and activists have become increasingly violent, with police firing live bullets and protesters attacking officers and throwing petrol bombs.</p>
</blockquote>
<p>The article is written by the BBC, the public UK broadcaster. Its focus is on the extradition bill being controversial, and causing people to protest violently.</p>
<h2 id="yandex"><a class="toclink" href="#yandex">Yandex</a></h2>
<p>Let's use a different search engine: the Russian one, <a href="https://yandex.com/">Yandex</a>.</p>
<p>The first result is the <a href="https://en.wikipedia.org/wiki/2019%E2%80%9320_Hong_Kong_protests">Wikipedia article</a>, which I believe is the most relevant page. This was the second result on Google as well, and it has about the same focus as the BBC article.</p>
<blockquote>
<p><a href="https://en.wikipedia.org/wiki/2019%E2%80%9320_Hong_Kong_protests">2019–20 Hong Kong protests</a></p>
<p>[...] ongoing protests in Hong Kong triggered by the introduction of the Fugitive Offenders amendment bill by the Hong Kong government.</p>
</blockquote>
<p>Most importantly, the protests are "ongoing", meaning you can still take part if you're a Hongkonger.</p>
<h2 id="baidu"><a class="toclink" href="#baidu">Baidu</a></h2>
<p>Now, let's see what a Chinese search engine shows me as the first result: <a href="http://baidu.com/">Baidu</a>.</p>
<blockquote>
<p><a href="https://www.telegraph.co.uk/news/worldnews/asia/hongkong/11129765/Hong-Kong-pro-democracy-protests-watch-live.html">'We'll be back' Hong Kong protesters vow</a></p>
<p>Hong Kong authorities demolished a protest camp at the heart of the city's pro-democracy movement, but scores of activists taken away by police vowed their fight for genuine elections was not over</p>
<p>Groups of up to four police arrested holdout protesters one by one</p>
</blockquote>
<p>What is happening here? Some protesters making empty promises, while going to prison?</p>
<p>No. It's an article from 2014, depicting the authorities' supression of a protest camp. Here, the protesters are the losing ones. The message is "Go home or go to jail". "We're in charge".</p>
<p>The article is old, but it marks a victory for the government.</p>
<p>The second result is some <a href="https://www.npr.org/tags/352993831/hong-kong-protests">NPR</a> filler <a href="https://en.wikipedia.org/wiki/Red_herring">red-herring</a>, showing people smile and that everything is OK.</p>
<p>The third result is still interesting, another old 2014 article, this time from a Chinese newspaper, again depicting a victory for the government, and the surrender of some protest leaders:</p>
<blockquote>
<p><a href="http://www.globaltimes.cn/content/894923.shtml">Hong Kong protest leaders surrender</a></p>
</blockquote>
<h2 id="everyone-is-loyal-to-their-tribe"><a class="toclink" href="#everyone-is-loyal-to-their-tribe">Everyone is loyal to their tribe</a></h2>
<p>The moral is, every news agency has its interests, and every search engine as well. You have to pay attention to the dates, and read articles with conflicting interests in order to get a more complete picture.</p>
<p>I ask you to think for yourself, and challenge authority. Even the western media may be manipulating information lately; and you can learn to spot fake news coming from anywhere.</p>Price gouging during disasters2020-03-03T16:45:00+02:002020-03-03T16:45:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2020-03-03:/price-gouging-during-disasters.html<p>When disaster strikes, like a fire, flood, hurricane, or an <a href="https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6">epidemic</a>, and utilities and supply chains get disrupted, you might see much higher prices for goods that yesterday were cheap. This is called <strong>price gouging</strong>.</p>
<p><a href="https://pixabay.com/photos/thunderstorm-flash-crane-hurricane-1761849/"><img alt="A crane struck by lightning in a thunderstorm" class="img-fluid" src="images/thunderstorm-crane.jpg"/></a></p>
<p>For instance, in 2017 after hurricane Harvey struck Texas, in some locations <a href="https://www.washingtonpost.com/news/business/wp/2017/08/30/99-for-a-case-of-bottled-water-texas-stores-accused-of-price-gouging-in-wake-of-harvey/">gas prices shot …</a></p><p>When disaster strikes, like a fire, flood, hurricane, or an <a href="https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6">epidemic</a>, and utilities and supply chains get disrupted, you might see much higher prices for goods that yesterday were cheap. This is called <strong>price gouging</strong>.</p>
<p><a href="https://pixabay.com/photos/thunderstorm-flash-crane-hurricane-1761849/"><img alt="A crane struck by lightning in a thunderstorm" class="img-fluid" src="images/thunderstorm-crane.jpg"/></a></p>
<p>For instance, in 2017 after hurricane Harvey struck Texas, in some locations <a href="https://www.washingtonpost.com/news/business/wp/2017/08/30/99-for-a-case-of-bottled-water-texas-stores-accused-of-price-gouging-in-wake-of-harvey/">gas prices shot up to $20 per gallon, water went to $8.50 per bottle, and hotel nights almost tripled</a>.</p>
<p>Even franchisors washed their hands of the stores that did this, severing ties and calling them "egregious and unethical".</p>
<hr/>
<h3 id="why-do-people-revolt-from-price-gouging"><a class="toclink" href="#why-do-people-revolt-from-price-gouging">Why do people revolt from price gouging?</a></h3>
<p>Well, I suppose that's viscerally clear. When you need the stuff most, that's when it gets 10x as expensive.</p>
<p>Keep reading!</p>
<hr/>
<h3 id="what-do-people-do-to-prevent-price-gouging"><a class="toclink" href="#what-do-people-do-to-prevent-price-gouging">What do people do to prevent price gouging?</a></h3>
<p>When the hurricane struck, Texas <a href="https://www.texasattorneygeneral.gov/consumer-protection/disaster-and-emergency-scams/how-spot-and-report-price-gouging">already had state laws forbidding price gouging</a>. Possibly not everyone followed them, as per the <a href="https://www.washingtonpost.com/news/business/wp/2017/08/30/99-for-a-case-of-bottled-water-texas-stores-accused-of-price-gouging-in-wake-of-harvey/">article</a>.</p>
<p>What the article does not say: those that did follow the laws had their shelves emptied in record time, using "first come, first serve", at much less profit.</p>
<p><a href="https://www.publicdomainpictures.net/en/view-image.php?image=37956&picture=empty-grocery-store"><img alt="Almost empty grocery store" class="img-fluid" src="images/empty-grocery-store.jpg"/></a></p>
<hr/>
<h3 id="what-might-happen-if-price-gouging-were-let-loose"><a class="toclink" href="#what-might-happen-if-price-gouging-were-let-loose">What might happen if price gouging were let loose?</a></h3>
<ol>
<li>Suppose you didn't buy the $8.50 bottle of water, not needing it too much, and being able to live for a day without water, while you think some more.</li>
<li>After that, suppose a father came and bought the last 5 bottles for his sick daughter that might die if she didn't get some.<ul>
<li>If the store had sold the water cheaply, there would have been none left for the father.</li>
</ul>
</li>
<li>As for you, perhaps you think about the high price, and figure you can drive somewhere less affected, where it's cheaper, and buy some there. Hmm, perhaps, even bring some back and sell it - people will surely need it, and you might price it cheaper than the current profiteers, but still make non-negligible money.</li>
<li>Others might reach the same conclusion, and do the same, and get rewarded by profit while the shortage lasts.</li>
</ol>
<p>Of course, you can't just start doing this when a hurricane strikes - commerce in many countries requires a license, and they couldn't just let <em>regular people do it</em>, <strong>gasp</strong>!</p>
<p>Still, if you were an authorized merchant, and could take advantage of the situation, you might do so, instead of your regular business. This would alleviate the shortage, and reward you for your endeavour through profit.</p>
<p>On the other hand, if you were a particularly greedy merchant, you might set your prices too high - such that very few people, if any, buy your products. In that case, it's your loss - you did not make the most money from the disaster.</p>
<ul>
<li>If the price is too low, you sell all of your stock, but some people still need it badly and might have paid more</li>
<li>If the price is too high, you are left with your stock at the end of the crisis, and have lost potential clients at lower prices</li>
</ul>
<p>A business gets the most profit by setting the price right, so that <code>number of sales * profit per item</code> is maximized.</p>
<hr/>
<h3 id="my-impressions"><a class="toclink" href="#my-impressions">My impressions</a></h3>
<p>Banning price gouging ensures the disappearance of the profit, and therefore, the resources. People will be forced to go somewhere else for water, and will have no financial motive to bring any back - it is not worth transporting heavy, cheap, and voluminous water for long distances.</p>
<p>People are forced to rely on themselves, friends, charity, or on government aid, which is what the government wants - more control over everyone.</p>
<p>That is how price signalling (sometimes known by the dirty word "gouging") works. It directs resources from where they're plentiful to where they're needed, while rewarding people prudent enough to stockpile what might be needed direly (sometimes known by the dirty word "speculators").</p>
<hr/>
<h3 id="my-suggestions"><a class="toclink" href="#my-suggestions">My suggestions</a></h3>
<p>I believe that, during a catastrophe:</p>
<ol>
<li>Price gouging should be only limited when selling to sick/disabled people, and</li>
<li>there should be no taxes for goods or services sold during a catastrophe, and</li>
<li>any natural person should be allowed to sell anything without requiring neither a license nor any reporting.</li>
</ol>
<p>These measures would <strong>mobilize the most people</strong> and ensure a distributed, spontaneous response to any disaster, instead of making people wait for someone to come and rescue them, like sitting ducks.</p>
<ul>
<li>Stores might be able to pay a premium for emergency delivery of goods, instead of waiting empty-shelved.</li>
<li>Without the fridge running, your food will go bad soon. You could sell it to someone hungry nearby instead.</li>
<li>Anyone could taxi people to their loved ones for money, without needing a taxi medallion from the city.</li>
<li>Your neighbor could, say, hook you up to his car charger, and let you charge your phone</li>
</ul>
<p>Also, there are some pitfalls you need to be aware of:</p>
<ul>
<li>fraudsters could sell you contaminated water as drinking water - make sure the bottle of water is factory-sealed</li>
<li>same with medicine; make sure it's in factory-sealed packaging before buying</li>
<li>only buy prepared food from people you trust</li>
<li>make sure produce looks good and smells good</li>
<li>make sure home-canned food is at least under vacuum. This means it is sealed, but not necessarily that it is sterile.</li>
</ul>
<p>In general, you won't get the same experience from a private person as from a licensed store: storage temperature and humidity might not be respected, for example. But the state should allow you to buy at your own risk, at least while it can not guarantee your survival.</p>
<p>Poor people will be affected more strongly by these price variations. Suppose they could barely afford food at the usual price - now it's many times worse. Fortunately, this coin has two sides: they can get rich quick by doing valuable work during the crisis, instead of their usual work, or instead of doing nothing.</p>
<h3 id="people-that-are-poor-and-sickdisabled"><a class="toclink" href="#people-that-are-poor-and-sickdisabled">People that are poor AND sick/disabled</a></h3>
<p>People that are poor AND sick would have a lot to suffer. They would be <strong>unable to shop around</strong> looking for smaller prices, and <strong>unable to work</strong> to take advantage of the situation. But let's say they could reach their usual store, the nearest one.</p>
<ul>
<li>If there is price gouging, they will likely find the product they need, but it would be too expensive.<ul>
<li>Potential solution: Legal exceptions for sick/poor people, limiting the price increase, similar to current legislation applying to ALL people.<ul>
<li>Perhaps the state should later reward the store for this, especially if the store is assaulted by lots of such people. This is to prevent businesses from refusing to serve sick customers (as sociopathic as that would be)</li>
<li>Alternatively, the store could price-in the impact of potential disasters, and refusals should be punished, as is done typically with current legislation.</li>
</ul>
</li>
</ul>
</li>
<li>If there is no price gouging, they will probably find an empty shelf when they reach the store.<ul>
<li>Potential solution: mandating that stores hold back some products; similar to parking for disabled; but for water, basic foods, and medicine.</li>
</ul>
</li>
</ul>
<p>While every resource diverted towards helping the poor sick/disabled slows down the overall disaster recovery, I personally feel this would strike a fair balance.</p>
<hr/>
<h3 id="should-people-revolt-from-price-gouging"><a class="toclink" href="#should-people-revolt-from-price-gouging">Should people revolt from price gouging?</a></h3>
<p>As for the feeling of disgust when you see someone price-gouge: <strong>you should feel disgust at the scarcity, not the merchant trying to conserve the resource</strong>.</p>
<p>People are not your slaves and should not be forced to sell to you for a cheap price. The high price shows you that the resource is limited - and invites you to personally help out or to compete.</p>
<p>When the self-interested profit motive is aligned with the common good, people will hopefully tend towards better behavior.</p>
<hr/>
<h4 id="video-game-recommendation"><a class="toclink" href="#video-game-recommendation">Video game recommendation</a></h4>
<p>Perhaps you can tell I've played some <a href="https://bluebottlegames.com/games/neo-scavenger">NEO Scavenger</a> lately. I recommend this game, if you want to see what post-apocalyptic life might look like: sterilizing your own water, hunting, gathering, cooking, transporting goods and selling them. Check out the demo!</p>
<p>As much as I dislike the fact that it's proprietary software, I enjoyed it. I didn't notice it doing anything strange to my computer, and you can safely contain it in a virtual machine if you are worried.</p>
<hr/>
<p>How's this for a disaster plan? Any thoughts?</p>Frequency of bug/fix commits2019-12-11T14:10:00+02:002019-12-11T14:10:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2019-12-11:/frequency-of-bugfix-commits.html<p>I shared my <a href="/programming-languages.html">previous post</a> on Reddit, and I got some <a href="https://www.reddit.com/r/programming/comments/e6gyl3/programming_language_terseness_vs_popularity/f9r2yee/?context=3">very useful feedback</a> which pointed out the irrelevance of what I was measuring (and also a <a href="https://arxiv.org/abs/1911.11894">scientific paper</a> pointing out many statistical errors in a "real" GitHub study).</p>
<p>So, I figured I'd measure something more relevant: bugs.</p>
<h2 id="caveats"><a class="toclink" href="#caveats">Caveats</a></h2>
<p>Sadly …</p><p>I shared my <a href="/programming-languages.html">previous post</a> on Reddit, and I got some <a href="https://www.reddit.com/r/programming/comments/e6gyl3/programming_language_terseness_vs_popularity/f9r2yee/?context=3">very useful feedback</a> which pointed out the irrelevance of what I was measuring (and also a <a href="https://arxiv.org/abs/1911.11894">scientific paper</a> pointing out many statistical errors in a "real" GitHub study).</p>
<p>So, I figured I'd measure something more relevant: bugs.</p>
<h2 id="caveats"><a class="toclink" href="#caveats">Caveats</a></h2>
<p>Sadly, there is no cheap way of measuring bugs due to language. What I did was similar to the paper <strong>criticized</strong> in the paper above, but with much fewer projects (roughly 20 per language), and a much simpler pipeline.</p>
<ul>
<li>This is not a scientific study. It is an informal hobby analysis, for discovering fun facts and/or languages. Use at your own risk!</li>
<li>Correlation is not causation. Just because there are more bugfix commits in a language, doesn't mean the language is at fault.<ul>
<li>The difference might be caused by something else, such as developer experience, or the use cases of the language, or the way the language is used.</li>
</ul>
</li>
<li>I did not correct for developer experience, number of developers, project size, or many other factors. When you create a model taking such variables into account, and holding them constant, you might get less overlap between languages.</li>
<li>The detection of bug/fix commits is very primitive: whatever is being returned by the GitHub query "fix OR fixed OR bug". This could be improved by using neural networks and somesuch (I would recommend character-level, in the case of commit messages). Also, the linked paper uses <a href="https://en.wikipedia.org/wiki/Bonferroni_correction">Bonferroni adjustment</a> to mitigate the error rate, but I didn't, because I don't know what the error rate is in my case.</li>
<li>There may still be unfixed bugs in the analyzed repositories (at least those reported as issues, but not yet fixed).</li>
</ul>
<h2 id="what-did-i-do"><a class="toclink" href="#what-did-i-do">What did I do</a></h2>
<ul>
<li>Search for the programming language's name, and then click on the programming language itself as a filter</li>
<li>Get top GitHub projects sorted by "most forked"</li>
<li>Eliminate projects which are about learning or collections of other software</li>
<li>Try to eliminate projects which, as a commit message policy, use "Fix" or "Bug" even for new features added</li>
<li>Eliminate projects with less than 200 commits</li>
<li>Eliminate projects with less than 75% of it being the main programming language</li>
<li>Only keep one project per language from each GitHub account</li>
<li>For each project, divide number of bug commits by total commits</li>
</ul>
<p>I have breached a few rules with Mathematica and Jupyter Notebook, because it was very difficult to find code with these constraints. Example: <a href="https://github.com/InsightSoftwareConsortium/SimpleITK-Notebooks">this</a> is a learning/tutorial project with both R and Python.</p>
<h2 id="boxplot"><a class="toclink" href="#boxplot">Boxplot</a></h2>
<p>In spite of the aforementioned limitations, I produced a pretty <a href="https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.boxplot">Matplotlib boxplot</a>.</p>
<p>The boxplot boxes show the interquartile range (IQR - 25th percentile to 75th percentile). The median (50th percentile, or value that splits the data in two halves) is in red. The whiskers are not modified from the default <code>whis</code> behavior, covering data 1.5 IQRs above and below (right and left actually, in this image) the box ends themselves. Anything more than 1.5 IQRs away are outliers (marked with circles).</p>
<p><img alt="Boxplot of bugfix frequency" class="img-fluid" src="images/prog-lang-bugfixes.png"/></p>
<h2 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h2>
<p>While this analysis was fun, it was not very informative. As you can see, there is significant overlap, even if we used the IQR which covers 50% of data, much less than the 95% confidence interval required for a typical statistical test. Even JavaScript's lower whisker is lower than any language's 75% percentile.</p>
<p>This means there is a lot of variation between projects within the same language, so much so that we can not tell which languages are better.</p>
<p>I added some Jupyter Notebook repositories, because I suspected Mathematica and Clojure use the REPL a lot, which I suspect leads to more testing of the code and fewer bugs committed. It seems using a REPL <em>might</em> have a positive influence, seeing that the median gets between Clojure and Mathematica, though again, that might not mean much - there are many "suspect" links there, and many Jupyter projects are for learning and/or presentation purposes, which might influence bugfix rates (I couldn't get around that, because there're so few with sufficient commit counts).</p>
<p>Still, I think I will try out Clojure. The low fix rate (~14% of commits compared to Python's ~23%) might mean I'll learn something.</p>
<p>As usual, you can find the <a href="https://github.com/danuker/Explorations/tree/master/bugfix-ratios">Jupyter Notebook on my GitHub repository</a>.</p>Programming languages2019-11-26T13:14:00+02:002019-11-26T13:14:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2019-11-26:/programming-languages.html<p>I decided to review some programming languages, by comparing their popularity with their <a href="http://rosettacode.org/">Rosetta Code</a> implementation terseness.</p>
<p>I measured terseness as the average of implementations' estimated zipped code filesize.</p>
<p><strong>Update 2021-01-17</strong>: After another <a href="https://www.reddit.com/r/programming/comments/kyu8ze/why_haskell_is_our_first_choice_for_building/gjkibf6/">enlightening Reddit discussion</a>, I realized an even worse caveat of just looking at Rosetta Code snippets is …</p><p>I decided to review some programming languages, by comparing their popularity with their <a href="http://rosettacode.org/">Rosetta Code</a> implementation terseness.</p>
<p>I measured terseness as the average of implementations' estimated zipped code filesize.</p>
<p><strong>Update 2021-01-17</strong>: After another <a href="https://www.reddit.com/r/programming/comments/kyu8ze/why_haskell_is_our_first_choice_for_building/gjkibf6/">enlightening Reddit discussion</a>, I realized an even worse caveat of just looking at Rosetta Code snippets is the fact that some languages, like Clojure, need unit tests to reach the reliability offered by Haskell, which makes you define types upfront. The unit tests are not in Rosetta Code, but the type definitions are. Therefore, this analysis is even less relevant.</p>
<p><strong>Update 2019-12-11</strong>: I want to add a few caveats that people notified me in <a href="https://www.reddit.com/r/programming/comments/e6gyl3/programming_language_terseness_vs_popularity/">this Reddit post</a>:</p>
<ul>
<li>The analyzed source code <strong>includes comments</strong>, which might make up a lot of the code, like <a href="https://rosettacode.org/wiki/Eban_numbers#REXX">REXX</a>.</li>
<li>The tasks might use <a href="https://www.reddit.com/r/programming/comments/e6gyl3/programming_language_terseness_vs_popularity/f9tcstp/?context=3"><strong>different algorithms</strong></a> in different programming languages. Hopefully this averages out, though.</li>
<li>The compression did not do much. What I thought was it would be a better proxy for "difficulty" than raw source code, but it's the same. <a href="https://github.com/danuker/Explorations/blob/nomath/rosetta-code/Compare-programming-languages.ipynb">Here is the analysis with raw file sizes instead of zipped sizes</a>.</li>
<li>I'm not measuring what matters. Of course I'm not, what matters is different for everyone. I encourage you to use <a href="https://github.com/danuker/Explorations/blob/master/rosetta-code/">my Jupyter Notebook</a> and create your own analysis. Also, send a comment over, I'm very curious what you thought of! Check out the "nomath" branch also.</li>
</ul>
<p>I averaged various metrics of popularity (zeros when they were missing):</p>
<ul>
<li>The <a href="https://www.tiobe.com/tiobe-index/">TIOBE index</a></li>
<li>What languages people tend to <a href="https://web.archive.org/web/20190906151905/https://blog.sourced.tech/post/language_migrations/">migrate to on GitHub</a></li>
<li>Number of <a href="http://rosettacode.org/">Rosetta Code</a> examples themselves (people must have had lots of dedication)</li>
</ul>
<p>Because not all languages implement all Rosetta Code tasks, I had to estimate the length of the missing ones. I did this using the SVD++ matrix factorization algorithm in <a href="http://surpriselib.com/">Surprise</a>.</p>
<p>Also, many language-task pairs have multiple source code files. Sometimes there were multiple implementations, but other times it was just one implementation, but split across multiple files. In all cases, I ignored the pair (as if no examples existed).</p>
<p><strong>Update 2019-11-27</strong>: I also filtered the tasks to non-math ones that I handpicked.</p>
<p>I could then compute statistics about the longest programs, and the shortest languages.</p>
<p>Still, I didn't want <a href="https://code-golf.io/">code golf craziness from here</a> (nor <a href="https://codegolf.stackexchange.com/">code golf craziness from there</a>). So, I also looked at the popularity of languages.</p>
<h2 id="non-math-map"><a class="toclink" href="#non-math-map">Non-Math Map</a></h2>
<p><strong>Update 2019-11-27:</strong> Here is a map of the languages, in popularity vs. code size on <strong>non-math tasks handpicked by me</strong>, with the <a href="https://en.wikipedia.org/wiki/Pareto_efficiency">Pareto</a>-nondominated space in green:</p>
<p><img alt="Map of programming language popularity vs. non-math code size" class="img-fluid" src="images/prog-pop-v-lang-size-nomath.png"/></p>
<h2 id="overall-map"><a class="toclink" href="#overall-map">Overall Map</a></h2>
<p>Here is the old map, with <strong>all the Rosetta Code tasks</strong>, including lots of numerical/math ones:</p>
<p><img alt="Map of programming language popularity vs. code size" class="img-fluid" src="images/prog-pop-v-lang-size.png"/></p>
<h2 id="comments"><a class="toclink" href="#comments">Comments</a></h2>
<p>My analysis found quite a few interesting languages. The most condensed were:</p>
<table class="table table-striped">
<thead>
<tr>
<th>Language</th>
<th>Average estimated code (zipped bytes)</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="http://rosettacode.org/wiki/Category:Burlesque">Burlesque</a> - crazy code golf language</td>
<td>110.36</td>
</tr>
<tr>
<td>Excel - sadly, not usable outside spreadsheets</td>
<td>115.15</td>
</tr>
<tr>
<td>IDL - "Interactive Data Language"; commercial but has <a href="https://github.com/gnudatalanguage/gdl">a free clone</a></td>
<td>141.71</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/GUISS">GUISS</a> - a Windows GUI automation language</td>
<td>144.67</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/TI-83_BASIC">TI-83-BASIC</a> - a Texas Instruments calculator language</td>
<td>144.97</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/PARI/GP">PARI/GP</a> - a computer algebra system</td>
<td>145.51</td>
</tr>
</tbody>
</table>
<p>Not all of these are useful programming languages (they are very domain-specific).</p>
<p>But <a href="http://pari.math.u-bordeaux.fr/"><strong>PARI/GP</strong></a>, while not very popular, had <a href="https://rosettacode.org/wiki/Category:PARI/GP">440 samples in Rosetta Code</a> when my dataset was created. A pretty comprehensive set if you ask me! It has <a href="http://pari.math.u-bordeaux.fr/Events/PARI2018b/talks/prog.pdf">programming capabilities</a>. The fact that it was written by mathematicians for mathematicians intrigues me; I will check it out (I could easily install the <code>pari-gp</code> Debian package then run <code>gp</code>). If it is flexible enough, it should be much easier to write code in! I will set the unpopularity threshold a bit below this language.</p>
<p><strong>Edit 2019-11-27</strong>: After looking at non-math tasks, I chose to study R and Clojure first, instead of this algebra system; it is no longer on the Pareto frontier when including Mathematica (but it is still close).</p>
<p>It seems <a href="https://www.wolfram.com/language/"><strong>Mathematica's language</strong></a>, a proprietary one, is also quite condensed. I don't want to learn it because it's proprietary and quite locked-in - even their tutorials show pictures instead of copyable code.</p>
<p>The third one that caught my eye was <strong>MATLAB</strong>. I did not expect it to be so terse, I did not think it so when I was using it during college. The reason might have been not knowing its vast library of in-built functions. A libre version is called <a href="https://www.gnu.org/software/octave/"><strong>Octave</strong></a>, which is also slightly terser, but was not on any popularity list. Still, many MATLAB scripts run as they are in Octave.</p>
<p><a href="https://clojure.org/"><strong>Clojure</strong></a> (<strong>new on the Pareto frontier as of 2019-11-27</strong>) - After considering just non-math tasks, Clojure reaches the frontier, albeit not much better than R. Clojure is a lisp-like language on the JVM.</p>
<p><a href="https://www.r-project.org/"><strong>R</strong></a> is a free-software statistical computing language. I am liking it a lot, and it is very popular among data scientists. If I can not learn or use PARI/GP, I will try my brain at R.</p>
<p><a href="https://www.ruby-lang.org/en/"><strong>Ruby</strong></a> and <a href="https://www.php.net/"><strong>PHP</strong></a> are very popular languages, built with the Web in mind. I have used Ruby professionally and I love it, it really is perceptibly denser than Python. Here is an example in an <code>irb</code> session:</p>
<div class="highlight"><pre><span></span><code><span class="o">></span><span class="w"> </span><span class="mi">3</span><span class="o">.</span><span class="n">even?</span>
<span class="o">=></span><span class="w"> </span><span class="kp">false</span>
</code></pre></div>
<p>Here, I am "asking" the number 3 whether it is an even number. Notice the question mark, a useful convention to remind yourself that the return value is a boolean. Contrast to the Python way, which has no support for this:</p>
<div class="highlight"><pre><span></span><code><span class="o">>>></span> <span class="mi">3</span> <span class="o">%</span> <span class="mi">2</span> <span class="o">==</span> <span class="mi">0</span>
<span class="kc">False</span>
</code></pre></div>
<p>Tiny cases like this add up, and I am not surprised Ruby is objectively terser.</p>
<p>The reason I didn't stick with Ruby is because its numerical computing ecosystem is not as developed as Python's. But it <a href="https://github.com/ruby-numo/numo-narray">is growing</a>.</p>
<p>As for <strong>PHP</strong>, I have not had much experience with it, and I'm surprised that it's shorter than Python (albeit only slightly).</p>
<p><a href="https://www.python.org/"><strong>Python</strong></a> is my current workhorse. This analysis is done using Python (with <a href="https://pandas.pydata.org/">Pandas</a> and the Mathematica-inspired <a href="https://jupyter.org/try">Jupyter-Notebook</a>, which can actually be used with many languages, including <a href="https://github.com/jdemeyer/pari_jupyter">PARI/GP</a>). It has a large and fast-growing user base, great support for a wide range of applications, and can be sped up via C extensions (as many computing libraries do).</p>
<p><a href="https://www.java.com/en/"><strong>Java</strong></a> is a more verbose language. I remember code "patterns" that led to something similar to <code>StudentFactoryManagerObserverFacadeController</code>, and don't want to go back there. Still, it is arguably the most popular language (though Python ~~will most likely outgrow it~~ <a href="https://insights.stackoverflow.com/trends?tags=java%2Cpython">might have outgrown it in some areas</a>).</p>
<h2 id="hardest-tasks"><a class="toclink" href="#hardest-tasks">Hardest tasks</a></h2>
<p><strong>Update 2019-12-11</strong></p>
<p>The factorization was also useful for detecting the tasks that took the most code, independent of programming language.</p>
<p>They might serve as inspiration for "extreme" difficulty coding interview tasks.</p>
<table class="table table-striped">
<thead>
<tr>
<th>Task</th>
<th>Average expected code length</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://rosettacode.org/wiki/Minesweeper_game">Minesweeper-game</a></td>
<td>1128.487693</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/MD5/Implementation">MD5-Implementation</a></td>
<td>1056.812754</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Stable_marriage_problem">Stable-marriage-problem</a></td>
<td>1001.869361</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/The_ISAAC_Cipher">The-ISAAC-Cipher</a></td>
<td>992.001283</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Tic-tac-toe">Tic-tac-toe</a></td>
<td>963.968912</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Honeycombs">Honeycombs</a></td>
<td>952.617747</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Zebra_puzzle">Zebra-puzzle</a></td>
<td>950.222265</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/K-means%2B%2B_clustering">K-means++-clustering</a></td>
<td>831.377990</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Total_circles_area">Total-circles-area</a></td>
<td>828.344111</td>
</tr>
<tr>
<td><a href="https://rosettacode.org/wiki/Sudoku">Sudoku</a></td>
<td>820.059907</td>
</tr>
</tbody>
</table>
<h2 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h2>
<p>Do not take this analysis too seriously. It has inaccuracies, and was done as a hobby/curiosity, not a formal scientific experiment.</p>
<p>Still, I hope you learned something from this analysis. Sure, code length is not an end-all be-all metric, but short code might be easier to work with (at the expense of being harder to learn initially). I would appreciate your answer about any of the following:</p>
<ul>
<li>What do you think of your programming language? Is it close to the Pareto frontier?</li>
<li>Is the Rosetta dataset too artificial of a comparison?</li>
<li>Does my analysis reflect reality?</li>
<li>Did it help you choose a new language?</li>
</ul>
<p>The Jupyter notebook and popularity data for this analysis are available <a href="https://github.com/danuker/Explorations/blob/master/rosetta-code/">here</a>. Check out the "nomath" branch also.</p>How to protect your personal data2019-11-18T21:10:00+02:002019-11-18T21:10:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2019-11-18:/how-to-protect-your-personal-data.html<p>On the surface, <a href="https://en.wikipedia.org/wiki/Content_delivery_network">CDNs</a> (Content Delivery Networks) present a great way of speeding up the loading of websites.</p>
<p>Without a CDN, requests <a href="https://wondernetwork.com/pings/">take longer for users that are far away network-wise</a>, but not a lot longer if users are near the server (perhaps on the same continent). <a href="https://wheresitfast.com/results/5dd2a36351a54069486c9cc2">Here are some …</a></p><p>On the surface, <a href="https://en.wikipedia.org/wiki/Content_delivery_network">CDNs</a> (Content Delivery Networks) present a great way of speeding up the loading of websites.</p>
<p>Without a CDN, requests <a href="https://wondernetwork.com/pings/">take longer for users that are far away network-wise</a>, but not a lot longer if users are near the server (perhaps on the same continent). <a href="https://wheresitfast.com/results/5dd2a36351a54069486c9cc2">Here are some results about this site</a>: Izmir (Turkey) is close to Romania and loads my page quickly (0.974 seconds), but Medellín (Colombia, South America) is quite slower (my page loads in a bit over 3 seconds).</p>
<p>This doesn't just have to do with the speed of light through fiber optic and/or electrons through copper, but also with the amount and performance of network devices along the way.</p>
<p>With a CDN, you rely on a company which has servers in many parts of the world, thereby reducing the time it takes to serve clients.</p>
<p>However, CDNs have quite the caveats, as we'll explore in this post.</p>
<p>Another practice that has similar impact is websites tracking their visitors using a <strong>third-party analytics site</strong>, which may be more convenient than using a self-hosted solution.</p>
<div class="toc">
<ul>
<li><a href="#privacy">Privacy</a></li>
<li><a href="#security">Security</a></li>
<li><a href="#censorship">Censorship</a></li>
<li><a href="#what-you-can-do-as-a-user">What you can do as a user</a></li>
<li><a href="#what-you-can-do-as-a-site-owner">What you can do as a site owner</a></li>
</ul>
</div>
<h2 id="privacy"><a class="toclink" href="#privacy">Privacy</a></h2>
<p>The most important caveat is loss of privacy. When a website tells a browser to request a resource from a CDN, the CDN usually gets the <a href="https://en.wikipedia.org/wiki/HTTP_referrer">referrer</a> field as well. So, the CDN can see what IPs visit what other websites.</p>
<p>If you're particularly unlucky, the CDN <a href="https://techcrunch.com/2015/09/09/google-partners-with-cloudflare-fastly-level-3-and-highwinds-to-help-developers-push-google-cloud-content-to-users-faster/">partners with a data broker</a>. Quote:</p>
<blockquote>
<p>Google says the idea here is to “encourage the <strong>best practice</strong> of regularly distributing content originating from Cloud Platform out to the edge close to your end-users. Google provides a private, high-performance link between Cloud Platform and the CDN providers we work with, allowing your content to travel a low-latency, reliable route <strong>from our data centers out to your users</strong>.”</p>
</blockquote>
<p>So, your <a href="https://www.eff.org/deeplinks/2015/01/healthcare.gov-sends-personal-data">search for a health insurance scheme, for instance</a>, might result in personal health data disclosures.</p>
<p>This is a blatant violation of privacy, even more proof that the GDPR is a joke designed to give people cookie warnings (and a false sense of privacy). See my <a href="gdpranoia.html#edit-2019-11-18-implementation-and-impact">recent edit on my GDPR article</a>.</p>
<h2 id="security"><a class="toclink" href="#security">Security</a></h2>
<p>If a CDN gets hacked, which luckily hasn't happened <a href="https://thehackernews.com/2017/02/cloudflare-vulnerability.html">too</a> <a href="https://www.zdnet.com/article/jquery-hacked-site-was-hit-but-not-the-library/">often</a>, the users could get a malicious version of, say, a JS library. This can lead to arbitrary stealing of data, personal of financial.</p>
<p>A technique for validating such scripts <a href="https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity">has been developed</a>, but it's an extra burden on a website developer.</p>
<h2 id="censorship"><a class="toclink" href="#censorship">Censorship</a></h2>
<p>In addition to the surveillance introduced by these ubiquitous centralized middlemen, relying on their infrastructure introduces an extra point of failure in a website: namely, the website can be effectively and easily censored by the middleman.</p>
<p><a href="https://www.cloudflare.com/cloudflare-criticism/">CloudFlare</a> admits and "recognizes" this power, and seems to even brag about it. Please don't give them more power.</p>
<h2 id="what-you-can-do-as-a-user"><a class="toclink" href="#what-you-can-do-as-a-user">What you can do as a user</a></h2>
<p>It is important to know there are two kinds of CDNs:</p>
<ol>
<li>Content hosting; where the webmaster sends a website to a CDN to host on many servers; these are usually paid services, and there is not much you can do, if you want to see the site properly (since the CDN has the customer billing data, and the site won't serve you directly, you have to go through the CDN tracking you).</li>
<li>"Generic" library hosting, like Bootstrap or various Javascript frameworks. These are noticeably missing from the <a href="https://en.wikipedia.org/wiki/Content_delivery_network">Wikipedia article</a>. These are available for "free" (as in, you pay with private data), via Google Hosted Libraries, <code>cdnjs</code>, jsDelivr, Microsoft AJAX CDN, jQuery CDN and so on (I won't link to them). These can be blocked more conveniently, because many sites work in spite of blocking surprisingly many scripts. And even if you do request them, stripping their referrer renders them relatively blind (they won't know <strong>which</strong> site asked for Bootstrap).</li>
</ol>
<p>First of all, you should <strong>strip cross-origin referrers</strong>. <a href="https://blog.mozilla.org/security/2018/01/31/preventing-data-leaks-by-stripping-path-information-in-http-referrers/">Firefox in private mode tries to prevent this</a>, and it might be effective against CDNs.</p>
<p>Firefox (or the more private <a href="https://www.gnu.org/software/gnuzilla/">IceCat</a>) lets you configure it in regular browsing mode also, <a href="https://wiki.mozilla.org/Security/Referrer">by setting <code>network.http.referer.XOriginPolicy</code> to 1 or 2</a>, but I can't find a setting or a plugin I would trust for Chrome.</p>
<p>Second of all, stripping referrers still can't prevent an analytics script from encoding something before sending it home. To solve this, you can <strong>block third-party requests</strong> as well, with a tool I warmly recommend: <a href="https://github.com/gorhill/uBlock/wiki/Blocking-mode:-medium-mode"><strong>uBlock Origin</strong></a>, but this might affect site functionality).</p>
<p>I usually use "hard" mode, and choose which 3rd-party resources to unblock (if any) the first time I visit any site (then click the "lock" icon if I want to save them). But for not-so-hardcore users, I recommend "medium" mode, in addition to the stripped referrer.</p>
<p>In uBlock's "medium" mode, CDNs still track you via images and CSS, if you don't strip it.</p>
<hr/>
<h4 id="using-ublock-origins-advanced-user-panel-one-can-block-red-or-permit-green-requests"><a class="toclink" href="#using-ublock-origins-advanced-user-panel-one-can-block-red-or-permit-green-requests">Using uBlock Origin's "advanced user" panel, one can block (red) or permit (green) requests:</a></h4>
<p><a href="https://github.com/gorhill/uBlock/blob/master/README.md"><img alt="uBlock Origin: interface crash course" class="img-fluid" src="images/ublock.png"/></a></p>
<hr/>
<p>Third, you can use Firefox (and therefore IceCat) to comfortably read pages without any CSS loaded; simply click the <strong>"Reader View"</strong> button which arranges everything for a magical experience:</p>
<p><img alt="Firefox Reader view; shortcut is Ctrl+Alt+R" class="img-fluid" src="images/firefox-magic.png"/></p>
<h2 id="what-you-can-do-as-a-site-owner"><a class="toclink" href="#what-you-can-do-as-a-site-owner">What you can do as a site owner</a></h2>
<p>It is simple to respect users' privacy: do not use any 3rd party resources. Host everything yourself: CSS, minified JS, and media.</p>
<p>Usually, the resource sizes are on the same order of magnitude as HTML pages anyway (at least CSS, JS, and typically-sized images). Sure, you'll get more requests, but you can cache the resources way longer than the HTML (for instance, my site asks your browser to cache them for a week).</p>
<p>As for advertising, a site owner can make old-fashioned deals with people who want to advertise, and can host the ads on the site itself, and not through a third-party ad platform. Admittedly, this is more difficult to do, if you rely on the income. But if you're catering to a privacy-savvy audience, they likely have ad blockers anyway.</p>
<p>I am proud that my website does not make unrequested external requests. The only external requests your browser might make are the Disqus ones, and only if you click on the "Load Disqus" button below, to see the comments. But very few people do that, and I am glad that I respected their wish!</p>
<p>Still, I'd love to hear more from you (if not through Disqus, then via plain <a href="mailto:danuthaiduc@gmail.com">e-mail</a>)! What should I write about next?</p>Why Android is falling apart2019-11-18T21:00:00+02:002019-11-18T21:00:00+02:00Dan Gheorghe Haiductag:danuker.go.ro,2019-11-18:/why-android-is-falling-apart.html<p>If you're an Android user, you've probably seen more and more "glitches" lately, where you get delayed notifications, events, or even alarms.</p>
<div class="toc">
<ul>
<li><a href="#my-experience-with-a-samsung-galaxy-s6">My experience with a Samsung Galaxy S6</a></li>
<li><a href="#app-killing">App killing</a></li>
<li><a href="#firebase-notifications">Firebase & notifications</a></li>
<li><a href="#blocking-wifi-scanning-for-other-apps-while-google-location-services-still-scans-everything">Blocking WiFi scanning for other apps while Google Location Services still scans everything</a></li>
<li><a href="#open-source-is-not-freelibre-software">Open Source is not …</a></li></ul></div><p>If you're an Android user, you've probably seen more and more "glitches" lately, where you get delayed notifications, events, or even alarms.</p>
<div class="toc">
<ul>
<li><a href="#my-experience-with-a-samsung-galaxy-s6">My experience with a Samsung Galaxy S6</a></li>
<li><a href="#app-killing">App killing</a></li>
<li><a href="#firebase-notifications">Firebase & notifications</a></li>
<li><a href="#blocking-wifi-scanning-for-other-apps-while-google-location-services-still-scans-everything">Blocking WiFi scanning for other apps while Google Location Services still scans everything</a></li>
<li><a href="#open-source-is-not-freelibre-software">Open Source is not Free/Libre Software</a></li>
<li><a href="#why-all-this">Why all this?</a></li>
<li><a href="#actionable-advice">Actionable advice</a></li>
</ul>
</div>
<h2 id="my-experience-with-a-samsung-galaxy-s6"><a class="toclink" href="#my-experience-with-a-samsung-galaxy-s6">My experience with a Samsung Galaxy S6</a></h2>
<p>I have a second-hand Chinese-made "Samsung" Galaxy S6 (there's a whole other post to write about buying second hand phones; at least make sure the phone is built in the same country as the manufacturer says, and that it works with your SIM card - make sure to try calling someone, and browsing the web).</p>
<p>The default clock widget is called "Weather and Clock", which leads you to a weather app even if you tap on the clock.
There is one way to disable this: use the weather-free clock widget.
However, when you tap that, you get <em>absolutely nothing</em>.</p>
<p>Usually, Android offers easy access to the alarms, stopwatch, and timer by tapping on the clock widget. But to have this behaviour on an S6, you have to add a shortcut to the Clock app, using up an extra icon spot on the home screen like so:</p>
<p><img alt="clock icon next to the factory clock" class="img-fluid" src="images/glitchy-clock.png"/></p>
<p>I could not get used to tapping the Clock app. I wanted the default Android behaviour that would not waste my home screen real estate.</p>
<p>So, I installed <a href="https://f-droid.org/en/packages/com.simplemobiletools.clock/">a libre clock app from F-Droid</a>. Lo and behold, I can get the tapping behaviour I want. But now comes another "glitch":</p>
<p>When I steep tea and try to use a timer, the process freezes 5 seconds after the screen turns off, rendering this free-as-in-freedom clock application almost useless.</p>
<p>I have turned off Power saving, as well as removed the app from being "battery optimized" (a.k.a. killed) in the Battery -> tiny Battery Usage button -> tiny 3-dots menu button -> Optimize battery usage -> tap "Apps not optimized" and choose "All apps" instead -> Find your app manually in a huge list and disable the button on its right.</p>
<p><em>Note: Because this is so hard to perform, it is a <a href="https://www.darkpatterns.org/">Dark Pattern</a>. I believe it is meant to discourage you from using third-party background/automated apps. I can not explain this simply by incompetence of developers; I am quite convinced the Android or Samsung developers are prevented from fixing this.</em></p>
<p>But even after performing the steps in this dark pattern, the only benefit I got is an additional 5 to 15 seconds of timer time, before it being killed.</p>
<h2 id="app-killing"><a class="toclink" href="#app-killing">App killing</a></h2>
<p>There are some people gathering data about this excessive app killing (even foreground ones!), turning smartphones into dumbphones, and Samsung is among the offenders: <a href="https://dontkillmyapp.com/">Don't kill my app!</a></p>
<h2 id="firebase-notifications"><a class="toclink" href="#firebase-notifications">Firebase & notifications</a></h2>
<p>I tried using libre-software communication apps <a href="https://nextcloud.com/talk/">Nextcloud Talk</a> and <a href="https://delta.chat/en/">Delta Chat</a>. But they don't work satisfactorily because notifications are iffy.</p>
<p>Trying to find out why this is the case, I was shocked to find out that <strong>Google manages every notification on your phone</strong>.</p>
<p>How exactly?</p>
<p>The only way to get a notification from a background service to your screen (waking up the device) is to use Google Firebase / Cloud Messaging (formerly just Google Cloud Messaging). This piece of work is needed even if your application would otherwise be offline - all for "conserving battery".</p>
<ul>
<li>Delta Chat <a href="https://github.com/deltachat/deltachat-android/issues/82">gets around this</a> by having an always-on notification, keeping the process in the foreground, but in my experience it doesn't work that great either. It could have to do with battery optimizers putting it to sleep (see "App killing" above).</li>
<li>Nextcloud Talk installed from F-Droid has no notifications. They are thinking about offering notification support <a href="https://github.com/nextcloud/talk-android/issues/257">without Firebase</a>. Only the Google Play version has notifications, and they are through Firebase.</li>
</ul>
<p>Here are more details about Cloud Messaging, <a href="https://firebase.google.com/docs/cloud-messaging/concept-options">straight from the horse's mouth</a>.</p>
<h2 id="blocking-wifi-scanning-for-other-apps-while-google-location-services-still-scans-everything"><a class="toclink" href="#blocking-wifi-scanning-for-other-apps-while-google-location-services-still-scans-everything">Blocking WiFi scanning for other apps while Google Location Services still scans everything</a></h2>
<p>There exists an Android Pie <a href="https://issuetracker.google.com/issues/79906367">bug report</a>, which is starred by 297 people as of writing this.
The report is asking for Google to be less aggressive with blocking Wi-Fi scanning for apps.
Some apps used it for indoor localization, others for helping users optimize radio channel usage, and others to detect rogue access points.</p>
<p>Google took away their ability to function by severely limiting the number of scans, again for "battery usage" reasons. This is in spite of the fact that users clearly wanted those apps to use battery.</p>
<p>I call <strong>bullshit</strong> on this, and here are two reasons why.</p>
<h3 id="1-the-missing-permission-for-frequent-wi-fi-scans"><a class="toclink" href="#1-the-missing-permission-for-frequent-wi-fi-scans">1. The (missing) permission for frequent Wi-Fi scans</a></h3>
<p>It can be done via the <a href="https://issuetracker.google.com/issues/112688545#comment6">"NETWORK_SETTINGS" permission</a>, but that is only available to apps signed with the same key as the OS itself (this is Google, for most people). It is not accessible to users even with developer tools turned on. Why so secure?</p>
<h3 id="2-google-records-wi-fi-access-points-constantly"><a class="toclink" href="#2-google-records-wi-fi-access-points-constantly">2. Google records Wi-Fi access points constantly</a></h3>
<p><a href="https://www.howtogeek.com/211186/how-to-disable-google-location-wi-fi-scanning-on-android/">In this article</a> it is shown how to disable Google's constant Wi-Fi scanning.</p>
<p>Notice the wording: "Let Google's location service and other apps scan for networks, even when Wi-Fi is off". This means <strong>Google can screw with your "battery usage", but competitors can't</strong>.</p>
<p>Why would Google throttle everyone else's Wi-Fi scanning, but not their own?
One explanation I found is to prevent <a href="https://location.services.mozilla.com/">any competitors</a> or <a href="https://wigle.net">community efforts</a> from gathering data that would threaten <a href="https://developers.google.com/maps/documentation/geolocation/usage-and-billing">Google's Geolocation API moolah</a>.</p>
<p>Perhaps the <a href="https://www.theverge.com/2018/7/18/17580694/google-android-eu-fine-antitrust">Chrome integration anti-trust fine</a> was not enough for Google, and they are begging for more.</p>
<p>Eh, at least you can turn it off for Google as well, and improve your "battery usage".</p>
<p><img alt='Google scans Wi-Fi and Bluetooth hotspots even when the features are "off"' class="img-fluid" src="images/android-wifi-bluetooth.png"/></p>
<h2 id="open-source-is-not-freelibre-software"><a class="toclink" href="#open-source-is-not-freelibre-software">Open Source is not Free/Libre Software</a></h2>
<p>As you know, Android is "Open Source".</p>
<p>"Open Source" means "you can look at the source". But this is not enough, because it's pointless if you can't change the actual software running on the device. The device itself is heavily locked-down.</p>
<p>This is why <a href="https://www.gnu.org/philosophy/open-source-misses-the-point.html">GNU is against the "Open Source" term</a>. I agree, and I will try to use "libre software" or "free software".</p>
<h2 id="why-all-this"><a class="toclink" href="#why-all-this">Why all this?</a></h2>
<p>Google's and other manufacturer's actions in the market should tell you that the mobile industry is not a hardware business anymore, but one about "Internet Services" - i.e. subscription payments and/or watching ads. <a href="https://www.youtube.com/watch?v=esUOQpKNLsE">Check out TechAltar's YouTube video on the subject, focusing on Xiaomi.</a>.</p>
<p>This involves cheaper hardware, but with strings attached - <strong>you do not have software freedom on it</strong>.</p>
<p>The providers claim reducing battery usage as an excuse, but reducing battery usage can justify killing anything on a phone. Same as banning cars to prevent car accidents.</p>
<p>Sadly, many consumers fell into these traps, with their money. But you don't have to follow them.</p>
<h2 id="actionable-advice"><a class="toclink" href="#actionable-advice">Actionable advice</a></h2>
<p>If you find yourself bugged by your phone working unreliably, especially on background processes, the most effective way to send a signal to the manufacturers is to avoid buying it, or to return it, provide the reasons, and ask for your money back. If there's anything the manufacturers care about, it's money.</p>
<hr/>
<p>If you have an older device, check out if it is by any chance supported by <a href="https://lineageos.org/">Lineage OS</a>. That is a freer-software operating system, and is a step in the right direction; though it still includes some vendor-specific binary blobs. I've had a much better experience on old phones with Lineage than with the vendor's distro (except you will lose your warranty, and I overheated my Moto G to death within 8 months: Lineage had no thermal throttling/protection in my case; be warned!).</p>
<p>If you're looking to buy a new device, consider whether <a href="https://wiki.lineageos.org/devices/">LineageOS supports it</a>. This way, in the event the manufacturer pushes glitchy apps, and the seller won't take back the phone, at least you can flash Lineage on it (but be aware of the risks).</p>
<p>Or you can try different ROMs. Check out <a href="https://www.xda-developers.com/">XDA-Developers</a>, a mobile modding site. Of course, these come with higher risks.</p>
<hr/>
<p><strong>Apple</strong> is not an alternative; they offer even <a href="https://www.cnet.com/news/apple-sorry-iphone-battery-slowdown-ios-update-official/">less control of your own device</a>. While the article gives an idea of what Apple did, it also claims something I disagree with:</p>
<blockquote>
<p>Of course, when a chemically aged battery is replaced with a new one, iPhone performance returns to normal when operated in standard conditions.</p>
</blockquote>
<p>That is not true, because of software bloat. Every update packs more and more features, and is designed for newer and newer devices. It seems the lawyered-up people didn't know that, hehe.</p>
<p>Also, if you don't update, <a href="https://discussions.apple.com/thread/1156889">bad things might happen</a>.</p>
<hr/>
<p>Another alternative is to use a tablet + <strong>a dumbphone</strong>. This is what I do; the tablet is my cheap "Samsung" that does not work on a mobile network. Paradoxically, <a href="https://www.cel.ro/telefoane-mobile/telefon-mobil-allview-m9-join-dual-sim-black-pMyc0Mz0n-l/">the cheap 3G dumbphone I own</a> keeps my alarms and calendar events more reliably than the Galaxy I mentioned (sadly it doesn't have a timer feature; I use <a href="https://kde.org/applications/utilities/org.kde.kteatime">KTeaTime</a> on my laptop now). I use the dumbphone as my main phone. As a bonus, the Calculator app starts up much faster. This is in spite of the price difference. Why pay for a smartphone, and get a dumb-crippled one?
For maps I use <a href="https://f-droid.org/en/packages/net.osmand.plus/">OsmAnd</a>, which is based on OpenStreetMap, and I warmly recommend it - even has offline navigation.</p>
<p>Lastly, I'd appreciate if you shared this post with your friends. Friends don't let friends pay for smartphones but get dumbphones!</p>Review of Cryptocurrencies2019-09-02T00:00:00+03:002019-09-02T00:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-09-02:/review-of-cryptocurrencies.html<p>Hello again, dear reader!</p>
<p>As I promised you <a href="intro-to-cryptocurrencies.html">last post</a>, I have reviewed some cryptocurrencies, and I will publish my fact-based heavily-biased opinions on them. Take this post with a grain of salt, and mind that investment (or speculation) is risky. However, I hope you find it useful.</p>
<h3 id="bitcoin"><a class="toclink" href="#bitcoin">Bitcoin</a></h3>
<p><a href="https://bitcoin.org">Bitcoin …</a></p><p>Hello again, dear reader!</p>
<p>As I promised you <a href="intro-to-cryptocurrencies.html">last post</a>, I have reviewed some cryptocurrencies, and I will publish my fact-based heavily-biased opinions on them. Take this post with a grain of salt, and mind that investment (or speculation) is risky. However, I hope you find it useful.</p>
<h3 id="bitcoin"><a class="toclink" href="#bitcoin">Bitcoin</a></h3>
<p><a href="https://bitcoin.org">Bitcoin</a> is the first of decentralized cryptocurrencies. It still has the most users <a href="https://bitinfocharts.com/">(most addresses with >$1 USD, most Reddit subscribers, most GitHub stars)</a>, and since <a href="https://en.wikipedia.org/wiki/Metcalfe%27s_law">it lets you connect with more people</a>, it provides more value than the other networks.</p>
<p>Bitcoin's current supply is around 17.9 million BTC, with a mathematically-enforced eventual maximum of 21 million (but there may be hard forks of the network, as happened several times already).</p>
<p>However, the <a href="https://bitinfocharts.com/comparison/transactionfees-btc-bch-eth-xmr-zec.html#log&2y">transaction fees</a> are high, at time of writing being nearly $1 USD. This is more expensive than many bank tranfers.</p>
<p>Therefore, the currency is not as useful as a payment system anymore, but as a <a href="https://www.newsbtc.com/2019/08/05/bitcoin-store-of-value-narrative-turning-toward-safe-haven-asset/">"store of value" or "safe haven"</a> (hard to transact, but hopefully still of resilient value, like gold, silver, petrol and real estate). This discourages payments but encourages speculation (however, that is not necessarily a bad thing, if you have the stomach).</p>
<blockquote>
<p>@SquawkCNBC</p>
<p><a href="https://www.newsbtc.com/2019/08/05/bitcoin-store-of-value-narrative-turning-toward-safe-haven-asset/">"Humanity has now created a non sovereign, highly secure mechanism to store value that can exist anywhere that the internet exists," says @jerallaire on <strong>#btc surge amid #TradeWar</strong></a></p>
</blockquote>
<p>The reason for the fees is in part, its popularity, and in part, the limited transaction processing speed. Bitcoin <a href="https://bitinfocharts.com/comparison/bitcoin-size.html">limits blocks to 1MB</a> (though using <a href="https://en.wikipedia.org/wiki/SegWit">SegWit</a> one can do more transactions, at the expense of having to withdraw them later in another transaction. SegWit transactions are not on the blockchain; it's complicated.).</p>
<p>A tyical blockchain transaction takes about <a href="https://bitcoinfees.earn.com/">244 bytes</a>, so a 1-MB block can only hold about 4100 of them. Also, a block is targeted to be generated every 10 minutes, which means you can only get 410 transactions per minute. You can see why this is a problem.</p>
<p>In spite of this limitation, the blockchain size has surpassed 277GB, storing transaction history since 2009-01-09. The size is important for judging how expensive it would be to host a mining or non-mining (just validation) node yourself.</p>
<p>Some free-software wallets (free-software are the only ones deserving review) are the <a href="https://bitcoin.org/en/bitcoin-core/">official one</a>, and <a href="https://electrum.org/">Electrum</a>.</p>
<table class="table table-striped">
<thead>
<tr>
<th>Trait</th>
<th>Value</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Market Cap</td>
<td>$171 B</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Fee per transaction</td>
<td>$0.94</td>
<td><a href="https://bitinfocharts.com/comparison/transactionfees-btc-bch-eth-xmr-zec.html#log&2y">BitInfoCharts chart</a></td>
</tr>
<tr>
<td>Transactions per day</td>
<td>336k</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Blockchain Size</td>
<td>277.12 GB</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
</tbody>
</table>
<h3 id="bitcoin-cash"><a class="toclink" href="#bitcoin-cash">Bitcoin Cash</a></h3>
<p><a href="https://www.bitcoincash.org/">Bitcoin Cash</a> is a fork of Bitcoin that decided to increase the block size limit. This will hopefully drive transaction prices lower, which would lead to easier and more transactions. But this comes at a cost of increased workload for the miners - which might lead to centralization and relatively less security compared to Bitcoin.</p>
<p>As opposed to Bitcoin, which aims to be a "store of value", Bitcoin Cash has the main goal of being "cash", or a means of payment, not just of speculation. Bitcoin Cash argues it is staying true to the original <a href="https://bitcoin.org/bitcoin.pdf">Bitcoin whitepaper - "Bitcoin: A Peer-to-Peer Electronic Cash System"</a>".</p>
<p>Indeed, the transactions are much cheaper at least for now (with much fewer transactions than Bitcoin, perhaps because of lower popularity), at less than $0.01 (1 cent).</p>
<p>Transactions being so cheap led to the development of on-blockchain chat protocols, like Twitter. These protocols are hard to censor, unlike Twitter. <a href="https://memo.cash/">Memo.cash</a> is an example (but note that the website itself is censorable, as any centralized service; do not place any meaningful money on a website that has your private key).</p>
<p>Bitcoin Cash's supply is meant to be the same as Bitcoin's.</p>
<p>A free-software wallet I could find is <a href="https://electroncash.org/">Electron Cash</a>, which is forked from Electrum for Bitcoin, and it also supports Android (useful for a "cash" cryptocurrency).</p>
<table class="table table-striped">
<thead>
<tr>
<th>Trait</th>
<th>Value</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Market Cap</td>
<td>$4.96 B</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Fee per transaction</td>
<td>$0.003</td>
<td><a href="https://bitinfocharts.com/comparison/transactionfees-btc-bch-eth-xmr-zec.html#log&2y">BitInfoCharts chart</a></td>
</tr>
<tr>
<td>Transactions per day</td>
<td>38k</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Blockchain Size</td>
<td>170.56 GB</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
</tbody>
</table>
<h3 id="ethereum"><a class="toclink" href="#ethereum">Ethereum</a></h3>
<p><a href="https://www.ethereum.org/">Ethereum</a> is more of a developer-friendly cryptocurrency. Its main feature is a flexible "virtual machine" that runs on miners' computers, allowing you to program applications - called "DApps" for "distributed apps".</p>
<p>Among my favorites are:</p>
<ul>
<li><a href="https://www.cryptokitties.co/">CryptoKitties</a>, which lets you trade collectible kitties on the blockchain as non-fungible tokens</li>
<li><a href="https://aragon.org/">Aragon</a>, which lets you create and manage "decentralized organizations"</li>
<li><a href="https://www.augur.net/">Augur</a>, which lets you <a href="https://predictions.global/">bet on future events or values</a>.</li>
</ul>
<p>Many others can be seen on <a href="https://www.stateofthedapps.com/rankings">StateOfTheDApps</a>.</p>
<p>Of course, this flexible programming allows for complicated software - which allows for bugs. There was an error in a crowdfunding app called "The DAO", <a href="https://www.coindesk.com/understanding-dao-hack-journalists">which allowed draining funds</a>.</p>
<p>Ethereum has plenty of transactions, surpassing even Bitcoin. Its blockchain size is 428 GB, in its shorter history since 2015-07-30. As a consequence, an Ethereum node or miner needs to be more beefy.</p>
<p>Transaction fees are around $0.12, which is still not cheap. Ethereum includes processing power into the transaction cost ("gas"), not just the storage space like Bitcoin. I think that is being more fair.</p>
<p>As for its supply, I am not too knowledgeable about the algorithm that emits Ethereum, but it seems the original supply <a href="https://etherscan.io/stat/supply">still makes up 67% of it</a>, so the inflation was not too bad (50% since July 2015, roughly 11% compounded per year; comparable with Romania's 11.5% inflation of the <a href="https://bnr.ro/Masa-monetara-M3-si-contrapartida-acesteia-5171.aspx">M2 money supply in 2017</a>).</p>
<p>Still, if this fiat-currency-level inflation were to last, what would be the point of holding Ethereum?</p>
<p>That is the point! Ethereum wants you to use it and develop DApps, not just hold it forever. They do not promise any supply cap à la Bitcoin, and the inflation is quite high.</p>
<p>Ethereum is investigating sharding - splitting the blockchain in 100, and having nodes only keep 1 part of it. <a href="https://github.com/ethereum/wiki/wiki/Sharding-roadmap">See here</a>. But this is hard, not only for Bitcoin <a href="https://petertodd.org/2015/why-scaling-bitcoin-with-sharding-is-very-hard">as this article says</a>, but for all cryptocurrencies.</p>
<table class="table table-striped">
<thead>
<tr>
<th>Trait</th>
<th>Value</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Market cap</td>
<td>$18 B</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Fee per transaction</td>
<td>$0.12</td>
<td><a href="https://bitinfocharts.com/comparison/transactionfees-btc-bch-eth-xmr-zec.html#log&2y">BitInfoCharts chart</a></td>
</tr>
<tr>
<td>Transactions per day</td>
<td>668 k</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
<tr>
<td>Blockchain Size</td>
<td>428 GB</td>
<td><a href="https://bitinfocharts.com/">BitInfoCharts</a></td>
</tr>
</tbody>
</table>
<h3 id="monero-zcash"><a class="toclink" href="#monero-zcash">Monero, ZCash</a></h3>
<p>These coins have innovated in another direction: allowing for private, untraceable transactions. Coupled with techniques for anonymizing your network, they might offer immense financial anonymity (and freedom).</p>
<p>One thing to mention is that these currencies are definitely not bug-free (and neither is any software, actually): ZCash fixed a <a href="https://z.cash/blog/zcash-counterfeiting-vulnerability-successfully-remediated/">catastrophic vulnerability</a> earlier this year; luckily, they did not see any traces of it being exploited.</p>
<p>Why is financial anonymity important, aside from tax evasion (which I deem risky and foolish, by the way)?</p>
<h4 id="financial-anonymity"><a class="toclink" href="#financial-anonymity">Financial anonymity</a></h4>
<p>Well, when citizens are transparent to the state, the state might be tempted to abuse that power. For example, if they see you are funding independent journalism, exposing the state's corrupt gears, they might freeze your accounts for some unrelated excuse, or they might persecute you.</p>
<p><a href="https://en.wikipedia.org/wiki/Buckley_v._Valeo">Here</a> is a very difficult decision the United States made, to reconcile their First Amendment (freedom of speech) with limiting political spending (in an election campaign). They found that limiting the spending also limits the <em>quantity of speech</em>, and the spending limit was deemed unconstitutional.</p>
<p>Another fun point I make personally: the implementation of a cryptocurrency (which is money, some say) only requires sending messages (which is speech).</p>
<p>An even funner point: banks do the same with money in bank accounts - if you see past the fancy suits and grandiose buildings, it's all electronic messaging. <a href="https://en.wikipedia.org/wiki/International_Payments_Framework">Here's one for instance.</a></p>
<p>A <a href="https://democracyisforpeople.org/page.cfm?id=19">differring opinion</a> is that "A system that allows corporations and the wealthiest among us to drown out the voices of others, [...] undermines the First Amendment’s core purpose – to foster and protect a flourishing marketplace of democratic ideas."</p>
<p>This is a valid point, but there are technologies that can let you filter out the well-funded corporate "spam":</p>
<ul>
<li><a href="https://github.com/gorhill/uBlock">ad blockers</a></li>
<li>critical thinking</li>
</ul>
<p>So, speech should be free, and money at least lets you speak louder, if it is not speech in itself.</p>
<p>I think that is fair (as long as that speech is not blasted with a megaphone into my home, without my consent).</p>
<p>This is why I am in support of Monero and ZCash, in principle. You become free to support anyone whatever they may do or say, without Big Brother watching.</p>
<p>Just please don't sponsor terrorists (which are violent by definition)!</p>
<p>In any case, these two currencies are a long way from drowning out other people's speech - their market cap and adoption is very small.</p>
<p>Still, if these currencies get reliable enough, they would be the only form of electronic voting that I <em>might</em> trust. <a href="https://www.youtube.com/watch?v=w3_0x6oaDmI">See this YouTube video</a> where Tom Scott explains why E-Voting is a bad idea - in essence, it's much easier to rig electronic elections.</p>
<p>When you look at the <code>Transaction fee / Market cap</code> ratio, Monero is the most expensive (2.72 UScents fee per billion USD market cap), while ZCash is the cheapest (0.014 cents/billion).</p>
<h4 id="monero"><a class="toclink" href="#monero"><a href="https://www.getmonero.org/">Monero</a></a></h4>
<table class="table table-striped">
<thead>
<tr>
<th>Trait</th>
<th>Value</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Market cap</td>
<td>$1.1 B</td>
<td><a href="https://bitinfocharts.com/monero/">BitInfoCharts on Monero</a></td>
</tr>
<tr>
<td>Fee per transaction</td>
<td>$0.03</td>
<td><a href="https://bitinfocharts.com/monero/">BitInfoCharts on Monero</a></td>
</tr>
<tr>
<td>Transactions per day</td>
<td>5k</td>
<td><a href="https://bitinfocharts.com/monero/">BitInfoCharts on Monero</a></td>
</tr>
<tr>
<td>Blockchain Size</td>
<td>63.75 GB</td>
<td><a href="https://bitinfocharts.com/monero/">BitInfoCharts on Monero</a></td>
</tr>
</tbody>
</table>
<h4 id="zcash"><a class="toclink" href="#zcash"><a href="https://z.cash/">ZCash</a></a></h4>
<table class="table table-striped">
<thead>
<tr>
<th>Trait</th>
<th>Value</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Market cap</td>
<td>$0.3 B</td>
<td><a href="https://bitinfocharts.com/zcash/">BitInfoCharts on ZCash</a></td>
</tr>
<tr>
<td>Fee per transaction</td>
<td>$0.000043 USD (amazing!)</td>
<td><a href="https://bitinfocharts.com/zcash/">BitInfoCharts on ZCash</a></td>
</tr>
<tr>
<td>Transactions per day</td>
<td>3.5k</td>
<td><a href="https://bitinfocharts.com/zcash/">BitInfoCharts on ZCash</a></td>
</tr>
<tr>
<td>Blockchain Size</td>
<td>22.98 GB</td>
<td><a href="https://bitinfocharts.com/zcash/">BitInfoCharts on ZCash</a></td>
</tr>
</tbody>
</table>
<h3 id="stellar"><a class="toclink" href="#stellar">Stellar</a></h3>
<p><a href="https://www.stellar.org/">Stellar</a> looks interesting, but I haven't researched it enough yet. What I can say is that they use a <a href="https://www.stellar.org/papers/stellar-consensus-protocol.pdf">much cheaper algorithm</a> to determine valid transactions and protect from double spending. I have not spent enough time with it to find out if it's safe enough.</p>
<p>Stellar has <a href="https://www.stellar.org/explainers/decentralized-trading">native trading</a> on the platform. This means the system stores bid and ask requests on the blockchain, which makes it a decentralized exchange. You can create tokens, which might represent your promise to send real things, and sell them for "lumens" or other tokens. Of course, that promise would not mean much without escrow, <a href="https://medium.com/wearetheledger/stellar-escrow-smart-contract-development-4c43ef32ac4b">which seems possible</a>.</p>
<p>At one point, <a href="https://www.fxstreet.com/cryptocurrencies/news/stellar-xlm-suffered-and-fixed-an-inflation-bug-in-2017-messari-201903280449">there was a bug</a> being exploited in the wild, which created an unintended $10 million worth of Lumens (the unit of the Stellar network). <a href="https://blockonomi.com/stellar-attacked-inflation-bug/">They burnt their own funds in an equivalent of that amount</a>, in order to protect Lumen owners from unintended inflation. Would an organization voluntarily forfeit $10 million? I hope there is more to gain; otherwise this smells fishy to me.</p>
<p>To prevent spam, you need to hold a <a href="https://www.stellar.org/developers/guides/concepts/fees.html#minimum-account-balance">minimum balance of 2.5 XLM</a> (worth about $0.15 as of today) in order to do anything.</p>
<h3 id="ripple"><a class="toclink" href="#ripple">Ripple</a></h3>
<p><a href="https://www.ripple.com/">Ripple</a> is not open in the sense that anyone can join the network to validate transactions (or even to create an account, for that matter). They have <a href="https://www.ripple.com/ripplenet/join-the-network/">"Contact Us" buttons</a> when you want to get down to business. Their main purpose is doing financial settlement, letting <a href="https://www.ripple.com/use-cases/banks/">banks save on payment processing costs</a>.</p>
<p>There is no distributed wallet, and you have to rely on Ripple-approved validators.</p>
<h3 id="libra"><a class="toclink" href="#libra">Libra</a></h3>
<p>Any article I could find on it either:</p>
<ul>
<li>reads like an advertisement, or:</li>
<li>points out the obvious centralization, being controlled by the status quo (among others, Facebook, PayPal, Visa, Mastercard).</li>
</ul>
<p>Since I don't trust any of those companies, which constantly lobby <em>against</em> individual financial freedom, and <em>for</em> their ever-increasing control over parts of my life, I will not trust Libra.</p>
<p><a href="https://qz.com/1649526/facebook-is-begging-us-to-trust-libra-but-should-we/">No</a>, <a href="https://www.youtube.com/watch?v=7S6506vkth4"><strong>just no!</strong></a></p>
<hr/>
<p>Hope you enjoyed my post. Do comment, so that I know what you think of it! Also, feel free to suggest any topic for a new post. I could write forever.</p>Intro to Cryptocurrencies2019-08-13T00:00:00+03:002019-08-13T00:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-08-13:/intro-to-cryptocurrencies.html<p>A friend of mine recently asked me about cryptocurrencies. I gave him the short story, but I'll post the long story here as well, for anyone interested.</p>
<h3 id="whats-the-point"><a class="toclink" href="#whats-the-point">What's the point?</a></h3>
<p>In its most basic form, a cryptocurrency is a list of balances in accounts. Only people that have the password …</p><p>A friend of mine recently asked me about cryptocurrencies. I gave him the short story, but I'll post the long story here as well, for anyone interested.</p>
<h3 id="whats-the-point"><a class="toclink" href="#whats-the-point">What's the point?</a></h3>
<p>In its most basic form, a cryptocurrency is a list of balances in accounts. Only people that have the password to an account can spend the balance in it, using a "wallet" application.</p>
<p>However, a bank account offers the same function so far.</p>
<ul>
<li>One difference is that the bank account uses a <strong>government currency</strong> as a unit, but a cryptocurrency account uses just <strong>made-up numbers</strong>.</li>
<li>Another difference is that anyone can begin validating transactions for cryptocurrencies, not just a central authority middleman (the bank).</li>
</ul>
<h3 id="why-would-anyone-trust-made-up-numbers"><a class="toclink" href="#why-would-anyone-trust-made-up-numbers">Why would anyone trust made-up numbers?</a></h3>
<p>The history of money is long, and I won't pretend I can explain all of it, but here's the gist of it.</p>
<p>A long time ago, people used gold and silver coins to trade. While this allowed great financial freedom, it is not very convenient to carry chunks of metal around with you all the time. Then, banks appeared, which let you use bank notes instead of the heavy chunks, and they would store your metals.</p>
<p>Then, fractional reserve banks were invented - banks would print more bank notes than they had gold, and lend it to people. See a comprehensive clarification <a href="https://www.khanacademy.org/economics-finance-domain/macroeconomics/monetary-system-topic/macro-banking-and-the-expansion-of-the-money-supply/v/overview-of-fractional-reserve-banking">in these Khan Academy videos</a>.</p>
<p>But more recently, the government and the central bank decided to <a href="https://www.thetimes.co.uk/article/chancellor-alistair-darling-on-brink-of-second-bailout-for-banks-n9l382mn62h">just print pretty much unlimited money and do bad things with it</a>.</p>
<p>This rewards the reckless risk-takers and the government, while it punishes diligent and responsible people, and <a href="https://mises.org/library/currency-debasement-and-social-collapse">can lead to the fall of civilizations</a>. Rome fell in part because of its <a href="https://en.wikipedia.org/wiki/Roman_currency#Value_and_composition">currency debasement</a>.</p>
<p><a href="https://en.wikipedia.org/wiki/Roman_currency#Value_and_composition"><img alt="Progression of Roman coins showing less and less silver content" class="img-fluid" src="images/Decline_of_the_antoninianus.jpg"/></a></p>
<blockquote>
<p>Without a constant influx of precious metals from an outside source, and with the expense of continual wars, it would seem reasonable that coins might be debased to increase the amount that the government could spend. A simpler possible explanation for the debasement of coinage is that it allowed the state to spend more than it had. By decreasing the amount of silver in its coins, Rome could produce more coins and "stretch" its budget. As time progressed, the trade deficit of the west, because of its buying of grain and other commodities, led to a currency drainage in Rome.</p>
</blockquote>
<p>There are two kinds of inflation:</p>
<ul>
<li>Money supply inflation, which is the increase in amount of money existing (the money printing I mentioned earlier)</li>
<li>Price inflation, which is the increase in prices as a result of increased spending; it directly erodes purchasing power, and is a consequence of the money supply inflation.</li>
</ul>
<p>The bigger the money supply, the more people have available to spend, and the more prices are driven up (unless people save the money). In general, since central banks love to print money, $100 today will buy you fewer goods and services than $100 a year ago.</p>
<p>A chart <a href="https://observationsandnotes.blogspot.com/2011/04/100-year-declining-value-of-us-dollar.html">here</a> shows that $100 today can be used to buy goods worth only $3.48 in 1900 dollars - though you still pay the same $100 face value. This is quite a drop in purchasing power.</p>
<p><a href="https://observationsandnotes.blogspot.com/2011/04/100-year-declining-value-of-us-dollar.html"><img alt="Chart of US Dollar purchasing power drop" class="img-fluid" src="images/Purchasing Power of U.S. Dollar.jpg"/></a></p>
<p>In Romania, the country I'm from, the central bank decided to print a lot of money following the 1989 revolution. The official consumer price index is <a href="http://www.insse.ro/cms/ro/content/indicele-preturilor-de-consum">only available in RAR archives and only from 2002</a>, which makes it pretty useless, but we can compute a more useful approximation of price inflation for ourselves.</p>
<p>A kilogram of bread was around <a href="https://economie.hotnews.ro/stiri-finante_banci-21668081-isi-mai-aduce-cineva-aminte-preturile-dinainte-1989.htm">4.4 Lei in 1970</a>, and today, 2019, 2 loaves of 500 grams be worth around <a href="https://www.numbeo.com/cost-of-living/in/Cluj-napoca"><code>2 * 2.82 =</code> 5.64 RON</a>. But note the <strong>difference in the unit</strong>: a RON (Romanian New Leu) represents 10000 ROL (Romanian Leu - the old one).</p>
<p>This translates to a 12818.18-fold devaluation over 49 years, or an average geometric price increase of <code>12818.18 ^ (1/49) - 1 =</code> <strong>21.3% per year</strong>. Any saver would have seen their savings wiped out of existence. Clearly, saving or holding Lei or RON should not be done. Perhaps this is why Romania suffered (or is still suffering?) a <a href="https://www.forbes.com/sites/stephenmcgrath/2017/01/30/losing-your-mind-romanias-attempts-to-counter-the-brain-drain/">brain drain</a>.</p>
<p>However, with cryptocurrencies, you can mathematically determine the money supply, and have a safe guarantee of what it will be in the future. For instance, there will <a href="https://en.bitcoin.it/wiki/Controlled_supply">never be more than 21 million Bitcoin</a>, and the monetary inflation will reach essentially zero by around 2040. However, the creator(s?) of Bitcoin decided to use some inflation, in order to stimulate adoption.</p>
<p>Of course, the actual market price of cryptocurrencies will not only depend on the supply, but also on the wildly varying demand.</p>
<p>Still, in a way, the "made-up" numbers proposed by cryptocurrencies are <strong>less made-up than the ones in people's bank accounts</strong>. One can use cryptocurrencies as a hedge against inflation - similar to storing oil, gold, copper, paintings, or houses, instead of holding money that is losing its purchasing power constantly.</p>
<h3 id="yet-more-advantages-of-cryptocurrencies"><a class="toclink" href="#yet-more-advantages-of-cryptocurrencies">Yet more advantages of cryptocurrencies</a></h3>
<p>Suppose you are a journalist involved in exposing powerful, corrupt politicians in your country. They have frozen your bank accounts.</p>
<p>You are having trouble paying for food, let alone paying for a bodyguard which you desperately need. What do you do?</p>
<p>Well, cryptocurrencies are the newest answer. Using cryptocurrencies, you can generate an account using a wallet application on your computer or phone, and accept donations by publishing your account's address. Your patrons can send you cryptocurrencies there.</p>
<p>The government would find it very difficult to block your cryptocurrency account, since you could transact even on paper (using QR codes or just writing down hex code).</p>
<h3 id="disadvantages-of-cryptocurrencies"><a class="toclink" href="#disadvantages-of-cryptocurrencies">Disadvantages of cryptocurrencies</a></h3>
<p>Not everything is rosy, though.</p>
<ol>
<li>
<p>Price variation</p>
<p>Since cryptocurrencies are in their infancy, demand for them is wildly fluctuating. One should be prepared to lose all their money - for no other reason than people rejecting them as payment. For instance, <a href="https://www.coinfairvalue.com/coins/bitcoin/">Bitcoin lost 92% of its value between June and November 2011</a>.</p>
</li>
<li>
<p>Technical/security risks</p>
<p>People need to be apt at computer security in order to hold significant amounts. Wallet files are magnets for viruses. In addition to one's password being copied off one's backups and <a href="https://bitcoinist.com/electrum-wallet-phishing-bitcoin/">fake wallets sending money to attackers</a>, there may also be some bug in actual cryptocurrency network implementations - like one found <a href="https://www.coindesk.com/zcash-team-reveals-it-fixed-a-catastrophic-coin-counterfeiting-bug">last year in ZCash that allowed counterfeiting</a> (but was fortunately not exploited in the wild).</p>
</li>
<li>
<p>Increased costs</p>
<p>Right now, fees for Bitcoin transactions are big, over 1 USD. While there are <a href="https://bitinfocharts.com/comparison/transactionfees-btc-eth-ltc-bch-etc-dash-xmr-zec.html#log">other cryptocurrencies with smaller fees</a>, it's hard to know how they will behave as they scale. It's fundamentally more difficult to keep a distributed network than to just have one central authority that processes payments (like a credit card company).</p>
</li>
<li>
<p>Politics / legal</p>
<p>There may be missing or even <strong>hostile legislation</strong> for how one should treat cryptocurrencies. There might be power-hungry central banks banning them (such as <a href="https://www.foxnews.com/tech/russia-bans-bitcoins">Russia's</a>, <a href="https://www.bbc.com/news/world-asia-india-43669730">India's</a>, and <a href="https://www.businessinsider.com/bitcoin-price-september-14-2017-9">China's</a>), for various reasons including terrorism, money laundering, and price instability.</p>
<p>Opinion: I think these reasons are quite baseless, and should not lead to banning.</p>
<ul>
<li>Terrorism means expressing your point of view using violence. While I sternly condemn any violence, I am a strong supporter of freedom of speech. Just because terrorists use cryptocurrencies doesn't mean we should ban them - terrorists also use roads, trains, planes, and <a href="https://www.youtube.com/watch?v=9VDvgL58h_Y">spoons</a>.</li>
<li>Money laundering: of course the government won't like people performing tax evasion. But that doesn't mean anonymous financial payments should be forbidden. A government should do its job without infringing upon financial privacy (see the journalist example above). If a government, say, imposes a ceiling on cash transactions, not only does it fail to meet its purpose (criminals will transact illegally and invisibly anyway), but it significantly reduces financial freedom for honest citizens.</li>
<li>Price instability: I think this is a joke. Anyone buying anything should be aware of the risks of what they are buying. The government should not ban people from assuming risks - it would lead to much greater risks (similar to <a href="https://en.wikipedia.org/wiki/Prohibition_in_the_United_States">clandestine breweries during the Prohibition</a>). Of course, the government should not "save" people from the risks either.</li>
</ul>
</li>
</ol>
<h3 id="what-cryptocurrency-should-i-choose"><a class="toclink" href="#what-cryptocurrency-should-i-choose">What cryptocurrency should I choose?</a></h3>
<p>In case you're insane enough to want some of this, stay tuned. <a href="review-of-cryptocurrencies.html">Another article is coming up soon.</a> I will review some currencies:</p>
<ul>
<li>Bitcoin</li>
<li>Bitcoin Cash</li>
<li>Ethereum</li>
<li>Monero, ZCash</li>
<li>Ripple, Stellar</li>
</ul>
<p>Have fun!</p>
<p><strong>Edit 2019-09-02: <a href="review-of-cryptocurrencies.html">Here is the article!</a></strong></p>Caring for your hardware2019-06-09T00:00:00+03:002019-06-09T00:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-06-09:/caring-for-your-hardware.html<h3 id="thermal-shutdown"><a class="toclink" href="#thermal-shutdown">Thermal shutdown</a></h3>
<p>I started playing some games on my Debian laptop. After some time of intense fan blowing, the laptop shut down.
I figured this was likely a thermal shutdown - the device turning itself off to protect itself.</p>
<p>I started monitoring the temperature - and it was getting to 100ºC at …</p><h3 id="thermal-shutdown"><a class="toclink" href="#thermal-shutdown">Thermal shutdown</a></h3>
<p>I started playing some games on my Debian laptop. After some time of intense fan blowing, the laptop shut down.
I figured this was likely a thermal shutdown - the device turning itself off to protect itself.</p>
<p>I started monitoring the temperature - and it was getting to 100ºC at times, while I was playing my game. This is terrible; I like my hardware cool.
The game did what it was supposed to - use up all available resources to give me the most fluid gameplay. But the laptop did not - it was supposed to throttle itself to prevent overheating before it simply shuts down for a thermal emergency. Instead, the default Intel drivers were pushing it into constant Turbo Boost.</p>
<h3 id="heat-and-electronics"><a class="toclink" href="#heat-and-electronics">Heat and electronics</a></h3>
<p>Why do I like my hardware to stay cool?</p>
<p>Because it prolongs its life. There are many electronics that suffer <a href="https://en.wikipedia.org/wiki/Electromigration#Thermal_effects">accelerated wear when hot</a>, so, the cooler they run, the better.</p>
<p>A <a href="https://www.ti.com/lit/an/sprabx4a/sprabx4a.pdf">report by Texas Instruments</a> gives the <a href="http://reliawiki.com/index.php/Arrhenius_Relationship">Arrhenius relationship</a> for their components designed to run at 105ºC for 10 years.</p>
<p>This is used to calculate an "acceleration factor" which predicts how long a component survives at different temperatures. If you run the component at 95ºC instead of 105ºC (just 10ºC cooler), you get a whopping <strong>79% increase</strong> (so, the component will work for <strong>17.9 years</strong> instead of 10).</p>
<p><a href="https://www.ti.com/lit/an/sprabx4a/sprabx4a.pdf"><img alt="Graph showing the relationship between temperature and acceleration factor" class="img-fluid" src="images/acceleration-factor.png"/></a></p>
<p>(Note the logarithmic vertical AF scale)</p>
<p>Another chart is offered by <a href="https://www.overclockers.com/overclockings-impact-on-cpu-life/">Overclockers.net</a>, and it's useful to get ballpark values for a CPU's life expectancy <a href="https://en.wiktionary.org/wiki/WRT#English">wrt</a>. temperature. It should really be taken with a grain of salt - your mileage may vary.</p>
<h3 id="software-solutions"><a class="toclink" href="#software-solutions">Software solutions</a></h3>
<p>I have researched software solutions for this problem. I have found the best Linux tool for the job: <code>cpufreqd</code>.
It allows you to specify custom rules for changing CPU states.</p>
<p><a href="pages/caring-for-your-hardware-cpufreqd-settings.html"><strong>Here</strong></a> are my settings for the laptop and desktop I use, as well as useful commands to deal with this tool.</p>
<p>I have had no more thermal problems since using <code>cpufreqd</code>, however, the caveat is that my game plays with a noticeably lower framerate.</p>
<p>Cool beans! :)</p>
<p>So far, I use only the CPU for playing games. I need to figure out how to use <code>cpufreqd</code> with my closed-source and not-very-usable-under-open-source Nvidia video card before I can use it (I got thermal crashes before). There is no open-source driver support for the GTX 850M yet. Hopefully I can still use cpufreqd on it. If you have any tips, please mail me, and you can get featured on this blog if you'd like!</p>
<h3 id="cleaning-and-thermal-compounds"><a class="toclink" href="#cleaning-and-thermal-compounds">Cleaning, and thermal compounds</a></h3>
<p>I thought cleaning my laptop was the right course of action. I took it to a laptop cleaning place, and asked them to replace the "thermal paste", but it did not seem to improve much.</p>
<p>I have since learned a bit about thermal pastes, and I can recommend you ones with a high thermal conductivity.
<a href="https://www.youtube.com/watch?v=hdTsra-uLBI">Here</a> is the video that made me interested in this topic. A Linus other than Mr. Torvalds shows that you can lower your laptop CPU temperature by 20ºC by replacing the stock thermal compound with a possibly dangerous aluminum-dissolving and electrically-conducting liquid metal paste from "Thermal Grizzly".</p>
<p>This is because the stock thermal paste is, paradoxically, among the least thermally conductive points in the heat dissipation pipeline. Replacing it with a better performing one is, perhaps, more important than having a better heatsink or air flow.</p>
<p>I have not yet replaced my thermal paste, since I am still reading about how to mitigate the risks of a metal paste, as well as researching non-conductive pastes. Also, very few seem to be available in Romania. (<a href="http://www.shop4pc.ro/pasta-termoconductoare-thermal-grizzly-kryonaut-1g-p-43676.html">Here</a> is one I found so far, but I have not bought it yet).</p>
<p>On liquid metal ones:</p>
<ul>
<li>They have the best thermal conductivity, usually, <a href="https://en.wikipedia.org/wiki/Thermal_grease#Composition">up to 13 W/(m*K)</a>.</li>
<li>You have to protect the electrical components near the CPU or GPU for which you're replacing the paste, such as by using a skirt around them made of <a href="https://en.wikipedia.org/wiki/Kapton">Kapton tape</a>.</li>
<li>Also, you have to be extra careful with the material compatibility and not to spill it on some other places on the motherboard.</li>
</ul>
<p>On others:</p>
<ul>
<li>They are not as risky as liquid metal</li>
<li>Some may have satisfactory thermal conductivity, of <a href="https://www.nrel.gov/docs/fy08osti/42972.pdf">4 W/(m*K), according to this 2008 study</a>.</li>
</ul>
<p>Note, however, that many vendors overstate their thermal conductivity. For instance, the study measured Arctic Silver 5 at 0.94 W/(m<em>K), but the vendor claims 8.7 W/(m</em>K) - that is an overstatement of <a href="https://knowyourmeme.com/memes/its-over-9000">over 900%</a>.</p>
<hr/>
<p>Hope you find this post useful. If so, drop me a comment or an e-mail.</p>
<h4 id="have-a-cool-day"><a class="toclink" href="#have-a-cool-day">Have a cool day!</a></h4>On Zeitgeist, the Venus Project, and the "Resource Based Economy"2019-04-09T14:40:00+03:002019-04-09T14:40:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-04-09:/on-zeitgeist-the-venus-project-and-the-resource-based-economy.html<p>The <a href="https://www.thevenusproject.com/">Venus Project</a> is a vague "solution" to current problems of civilization. They claim that through a <a href="https://www.thevenusproject.com/resource-based-economy/">"Resource Based Economy"</a>, coupled with advanced science and engineering, everyone can reach a very high standard of living.</p>
<p>This "Resource Based Economy" claims that "all resources must be declared as the common heritage …</p><p>The <a href="https://www.thevenusproject.com/">Venus Project</a> is a vague "solution" to current problems of civilization. They claim that through a <a href="https://www.thevenusproject.com/resource-based-economy/">"Resource Based Economy"</a>, coupled with advanced science and engineering, everyone can reach a very high standard of living.</p>
<p>This "Resource Based Economy" claims that "all resources must be declared as the common heritage of all Earth’s inhabitants".</p>
<p>The way I interpret this is as follows: "Abolish private property, and declare legal that anyone can use any one else's resources". If you have a better interpretation, please leave a comment or e-mail me, and I will update this post with further thought.</p>
<p>In other words, I interpret it as <a href="https://en.wikipedia.org/wiki/Communism">communism</a>, which is "structured upon the common ownership of the means of production and the absence of social classes, money, and the state".</p>
<h2 id="the-problem"><a class="toclink" href="#the-problem">The problem</a></h2>
<p>The problem with <strong>communism</strong>, or letting everyone use everyone else's stuff, is that the incentive to create stuff is practically eliminated.</p>
<p>Suppose you're a farmer, and you work hard and have a plentiful harvest. Then, your harvest is confiscated and shared equally among everyone (even people who have not lifted a finger, though are perfectly capable). Would you be just as eager to work hard next year?</p>
<p>Contrast to your neighbor, who is incredibly lazy, got up at noon, and did not work at all, yet was rewarded for this with the fruits of your labor.</p>
<p>Well, in the end, nobody wants to work anymore, or at least to give away their production. This happened in Soviet Ukraine, leading to the Great Famine (called <a href="https://en.wikipedia.org/wiki/Holodomor">"Holodomor"</a>).</p>
<h2 id="the-solution"><a class="toclink" href="#the-solution">The solution</a></h2>
<p>Letting people keep what they have created is essential to keep them motivated to keep creating stuff. This is called <a href="https://en.wikipedia.org/wiki/Capitalism"><strong>capitalism</strong></a> - (the "capital" is allowed to accumulate, and people are allowed to profit from their work).</p>
<p>If everyone produces, and then they are allowed to trade between themselves, even the poorest people in this model will quickly become richer than the richest in communism.</p>
<h2 id="the-problem-with-the-solution"><a class="toclink" href="#the-problem-with-the-solution">The problem with the solution</a></h2>
<p>Under capitalism, people that are unable to produce and have no resources accumulated will go very poor (and become dependent on charity, which might be inadequate). While this provides extraordinary motivation for being thoughtful and productive, it can be <strong>cruel</strong> to people unable to do so (for example, disabled people).</p>
<p>For this reason, most countries today are <a href="https://en.wikipedia.org/wiki/Mixed_economy">mixed economies</a>, allowing people to keep a portion of their capital, but redistributing the other portion (through taxation).</p>
<p>What's bad about that is that the redistributors keep a big portion for themselves; and they usually choose the beneficiaries in a manner advantageous to them (i.e. creating or enlarging a <a href="https://en.wikipedia.org/wiki/Welfare_dependency">dependent class</a> which votes them loyally).</p>
<h2 id="conclusion"><a class="toclink" href="#conclusion">Conclusion</a></h2>
<p>Now you know why I disagree with the Venus project - it would be no different than Soviet Russia, or today's North Korea. What would prevent people from slacking off? Authoritarianism? That is not a good solution and can't motivate them in the right way.</p>GDPRanoia2019-04-05T11:39:00+03:002019-04-05T11:39:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-04-05:/gdpranoia.html<p><em>Updated 2019-11-18.</em></p>
<p>The GDPR is legally convenient, in case you are a private person, and want the illusion of control over data gathered from you.</p>
<p>However, when considering its actual impact, and looking at it from different perspectives, it can appear strange, and perhaps even scary.</p>
<h2 id="broad-definitions"><a class="toclink" href="#broad-definitions">Broad definitions</a></h2>
<p>I had …</p><p><em>Updated 2019-11-18.</em></p>
<p>The GDPR is legally convenient, in case you are a private person, and want the illusion of control over data gathered from you.</p>
<p>However, when considering its actual impact, and looking at it from different perspectives, it can appear strange, and perhaps even scary.</p>
<h2 id="broad-definitions"><a class="toclink" href="#broad-definitions">Broad definitions</a></h2>
<p>I had doubts about certain data being labeled "personal", such as IP addresses. Sometimes, people share an Internet connection, which means they would share an IP address.</p>
<p>But according to <a href="https://gdpr-info.eu/art-4-gdpr/">Article 4</a>, "‘personal data’ means any information relating to an identified or <strong>identifiable</strong> natural person". This means an IP address is personal data, even if a processor can't by itself identify a user.</p>
<p>This definition is very broad. Perhaps one can use the shoe size to identify a person (say, pinpoint someone from a household). This means shoe size can be personal data, and you are liable if you share your friends' shoe sizes!</p>
<h2 id="kafkaesque-requirements"><a class="toclink" href="#kafkaesque-requirements">Kafkaesque requirements</a></h2>
<p>If you record the phone numbers of your friends, say, on your phone, then you are processing personal data.</p>
<p>Fortunately, according to <a href="https://gdpr-info.eu/art-2-gdpr/">Article 2</a>, the GDPR does not apply to you, if you are a "natural person" and you process the data "in the course of a purely personal or household activity".</p>
<p>There are two relevant court cases about this which <a href="https://law.stackexchange.com/questions/28582/is-it-possible-for-a-publicly-accessible-personal-blog-to-make-use-of-the-perso">I found here</a>:</p>
<ul>
<li><a href="http://curia.europa.eu/juris/document/document.jsf?text=&docid=48382&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=322419">One may not record a public space with a webcam, as a "purely personal or household" activity</a></li>
<li><a href="http://curia.europa.eu/juris/document/document.jsf?text=&docid=48382&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=322419">One may not transmit the name or contact details of other people as a "purely personal or household" activity either</a></li>
</ul>
<p>You can search for more cases on the CURIA website by clicking on "Search form" at the top, then removing the "ECLI:EU:" field, then filling in your terms in the "Text" field.</p>
<p>So, suppose you want to introduce two friends over the phone (they are not physically near you). There are two ways to do this:</p>
<ul>
<li>Give friend A's number to friend B -> sharing friend A's number is not "purely personal" (because friend B is an outsider).</li>
<li>Give friend B's number to friend A -> sharing friend B's number is not "purely personal" (because friend A is an outsider).</li>
</ul>
<p>The only way to make this legal is to ask for one of the friends' permission (as per <a href="https://gdpr-info.eu/art-6-gdpr/">Art. 6 1. (a)</a>) to share their number, and to <a href="https://gdpr-info.eu/art-30-gdpr/">keep the required records</a> for a processor and controller (you are both, in this case).</p>
<p>This is ridiculous. <a href="https://en.wikipedia.org/wiki/GDPR_fines_and_notices">Laugh, I tell you!</a></p>
<h2 id="balancing-act"><a class="toclink" href="#balancing-act">Balancing act</a></h2>
<p><a href="https://gdpr-info.eu/art-6-gdpr/">Article 6</a> says processing is lawful when 1. f):</p>
<blockquote>
<p>processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child.</p>
</blockquote>
<p>This pits the "legitimate interests" of the controller against the "fundamental rights" of the data subject. These two rights are contradictory.</p>
<p>Suppose it is in my interest to identify scammers on the Internet. This necessarily infringes the scammer's right to privacy.
What is "legitimate"? That's a word on which the entire GDPR rests on, but it's quite vague.</p>
<p><a href="https://logrhythm.com/blog/gdpr-legitimate-interest/">Here is a long exploration someone else made on the subject.</a> But it's not very conclusive.</p>
<h2 id="fines"><a class="toclink" href="#fines">Fines</a></h2>
<p>Fines can be <a href="https://gdpr-info.eu/art-83-gdpr/">whichever is higher</a> between €20M and 4% of financial turnover.</p>
<p>This means:</p>
<ul>
<li>If a small company got a fine, they might get a fine greater than their company's worth, putting them out of business.</li>
<li>If a very large company got a fine, then the 4% limit would apply: say, of Facebook's €50 billion, they would only lose 4% (or €2 billion). This would be a slap on the wrist for them, and would let them keep most of the money resulting from the most egregious of infringements.</li>
</ul>
<p>As you can see, the Big Tech lobbying worked. They have arranged so that newcomers and smaller competitors are exterminated. Justice much? (<a href="eu-copyright-directive.html#what-you-can-do-about-it">See the last point here.</a>)</p>
<h2 id="edit-2019-11-18-implementation-and-impact"><a class="toclink" href="#edit-2019-11-18-implementation-and-impact">Edit 2019-11-18: Implementation and impact</a></h2>
<p>Technically, any site can embed content from any other site, say, from Google.</p>
<p>Your browser then sends Google a request, perhaps with personal data in the HTTP Referrer, and you typically don't get a say in this.</p>
<p>This results in a blatant violation of your privacy. Clearly, this law is enforced <a href="https://en.wikipedia.org/wiki/GDPR_fines_and_notices">only for some players</a>. Sure, Google got a fine as well, but €50 million is a slap on the wrist, compared to their net income in the billions. Also, where is Facebook on this list, with their Like buttons plastered on every site, collecting data without consent? Where is Amazon, hosting tons of sites on their S3 servers, more than likely analyzing their data?</p>
<p>Luckily, you might still get the <strong>impression of privacy</strong>, with every cookie warning pestering you on every tiny site, EVEN IF you know how to turn off cookies IN YOUR BROWSER, because YOU KNOW HOW TO BROWSE THE WEB.</p>
<p>In <a href="how-to-protect-your-personal-data.html">this post</a>, you can find tips for keeping your data to yourself.</p>
<h2 id="what-you-can-do"><a class="toclink" href="#what-you-can-do">What you can do</a></h2>
<p>Ask the GDPR authority in your country before you do something with other people's data.
If you do not get a response within 30 days, tell the Ombudsman / People's Advocate, and/or the media.</p>
<p>Hopefully pestering the GDPR authority will give you at least an idea of what they expect.</p>
<p>An alternative to the above is to perform civil disobedience: risk fines in exchange for demonstrating the absurdity of laws.</p>
<p><strong>Do pester other people with this post!</strong></p>Don't buy Apple products.2019-04-03T23:20:00+03:002019-04-03T23:20:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-04-03:/dont-buy-apple-products.html<p>Apple denies any possibility of data recovery after your device is not bootable. This is plainly false.</p>
<p>On top of that, they ban people who know how to recover your data from their forums. Jessa Jones was banned from the Apple forum for offering support to people. It's almost as …</p><p>Apple denies any possibility of data recovery after your device is not bootable. This is plainly false.</p>
<p>On top of that, they ban people who know how to recover your data from their forums. Jessa Jones was banned from the Apple forum for offering support to people. It's almost as if they don't want you to recover your data.</p>
<p>Watch <a href="https://www.youtube.com/watch?v=LrILfIE9IB4">Jessa Jones' interview in a documentary, reposted by Louis Rossman</a></p>
<p>Also, <a href="https://www.youtube.com/watch?v=_b3tnN7AtkI">check out Louis' comments</a>.</p>
<p>If you want to get data recovered off an iThing, check out Jessa's company, <a href="http://www.ipadrehab.com/">iPad Rehab</a>.</p>
<p>"In a time of universal deceit — telling the truth is a revolutionary act" - George Orwell</p>EU Copyright Directive2019-04-03T14:00:00+03:002019-04-03T14:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-04-03:/eu-copyright-directive.html<p>The EU Parliament <a href="https://saveyourinternet.eu/latest-developments/">recently adopted</a> a new copyright directive which makes a site owner responsible for what the site's users post (Article 17 AKA former Article 13).</p>
<p>As with most legislation nowadays, it must have been initiated by some group which would benefit from it - as opposed to "good" legislation …</p><p>The EU Parliament <a href="https://saveyourinternet.eu/latest-developments/">recently adopted</a> a new copyright directive which makes a site owner responsible for what the site's users post (Article 17 AKA former Article 13).</p>
<p>As with most legislation nowadays, it must have been initiated by some group which would benefit from it - as opposed to "good" legislation, which would benefit everyone. My guess is that this group is the group of copyright owners.</p>
<h2 id="the-implications"><a class="toclink" href="#the-implications">The implications</a></h2>
<ul>
<li>Small-sized websites have a new burden: they will have a difficult job sifting through user content, making sure none of it infringes copyright.</li>
<li>Medium-sized websites will experience <a href="https://en.wikipedia.org/wiki/Chilling_effect">chilling effects</a> - and will severely limit media (say, any images like memes, video, or audio), because of being unable to verify what their users upload.</li>
<li>Big Tech websites (i.e. Google, Facebook etc), who can afford an automated filter will suffer a burden, but will be the only ones able to keep up with many people uploading to their site. This regulation gives them a competitive advantage.<ul>
<li>Note: Big Tech can also implement business deals with the copyright owners.</li>
</ul>
</li>
<li>Copyright / Intellectual Property ("IP") owners will have more rights to make citizens more copyright-compliant (aka censorship).</li>
</ul>
<h2 id="what-you-can-do-about-it"><a class="toclink" href="#what-you-can-do-about-it">What you can do about it</a></h2>
<ul>
<li>Vote at the next EU Parliament election. Check out <a href="https://saveyourinternet.eu/latest-developments/">which Members of Parliament</a> have sold you out, and vote with the other guys.<ul>
<li>While voting might not do much (since you're just one person), you can share this idea to your friends.</li>
</ul>
</li>
<li>Boycott whoever gains from this arrangement:<ul>
<li>Don't go to/buy movies</li>
<li>Don't watch music videos online (such as on YouTube, or Facebook), since they make money from the ads<ul>
<li>an alternative YouTube client you can use is <a href="https://newpipe.schabi.org/">NewPipe</a>), though it might be against YouTube ToS (and it doesn't cure YouTube addictions)</li>
</ul>
</li>
<li>Don't buy music or streaming licenses</li>
<li>Listen to free music on <a href="http://freemusicarchive.org/curator/creative_commons">Creative Commons</a>, which does not trample on your rights.<ul>
<li>The second song I clicked there was <a href="http://freemusicarchive.org/music/Kriss/nomad_ep/unfound38_03_-_kriss_-_jazz_club">"Jazz Club" by "Kriss"</a>, and it sounds amazing!</li>
</ul>
</li>
<li>Only buy books from individuals (which could not have possibly lobbied to the EU); not from "publishers" (such as Google Books, or Amazon/Kindle).</li>
<li>While an individual boycotting a huge company does not have much effect, you are being more effective than voting: namely, voting with your money and time, which you can know are not financing this travesty!</li>
<li>Become aware and boycott top tech lobbyists in the EU, in the <strong>last financial year</strong>:<ul>
<li>Google: <a href="https://lobbyfacts.eu/representative/1d40cdaf822941888d1e6121858bb617/google">over €6M</a> (they sell books, music streaming, and gain from ads on YouTube videos)</li>
<li>Microsoft: <a href="https://lobbyfacts.eu/representative/60239386204445e2b0fb38cada46b204/microsoft-corporation">over €5M</a> (they declared they used some of it "on certain aspects concerning contracts for the supply of digital content")</li>
<li>Facebook: <a href="https://lobbyfacts.eu/representative/64755e0fc2a14e46aa9d8646df6f8f19/facebook-ireland-limited">over €3.5M</a> (gains from this by crushing competitor social media sites)</li>
<li>Amazon: <a href="https://lobbyfacts.eu/representative/5615fc9a365b4e0f9e9c0d7929a73f17/amazon-europe-core-sarl">over €1.75M</a> (they also sell DRM music, video, and other IP)</li>
<li>See other big lobbyists to generally boycott <a href="https://lobbyfacts.eu/reports/lobby-costs/all/0/1/1/1/0/0?sort=lob&order=desc">here</a> (not just tech).</li>
</ul>
</li>
</ul>
</li>
</ul>My new Pelican blog2019-04-02T23:00:00+03:002019-04-02T23:00:00+03:00Dan Gheorghe Haiductag:danuker.go.ro,2019-04-02:/my-new-pelican-blog.html<p>The more I think about tech giants, the more I want to use self-hosted services.</p>
<p>On this blog, I will talk about what I want to do with my life.</p>
<p>The first and most important item on my list is <strong>self-expression</strong>.</p>
<p>When you use a private service like Facebook, Google …</p><p>The more I think about tech giants, the more I want to use self-hosted services.</p>
<p>On this blog, I will talk about what I want to do with my life.</p>
<p>The first and most important item on my list is <strong>self-expression</strong>.</p>
<p>When you use a private service like Facebook, Google, YouTube and such, their interest is not to provide free speech, nor promote the development of society, nor improve your well-being. But it is in their interest to rob you of your attention, time, money, and eventually your entire community.</p>
<p>As people become addicted to these services, the very fabric of society starts to fall apart. We only interact through a middleman, who only shows us what we like to see, to keep us hooked and wanting for more.</p>
<p>In order to have free speech, you have to counter this influence. You have to break out of the bubble, and listen to other people, even of opposing viewpoints. People are smart in general, and their opinions are there for a reason. Find the reason, and understand it!</p>
<p>I urge you to become aware of Big Tech's influence over your life, and to fight back.</p>
<h3 id="write-your-own-website-and-host-it-yourself"><a class="toclink" href="#write-your-own-website-and-host-it-yourself">Write your own website, and host it yourself!</a></h3>
<p>P.S. I personally recommend <a href="https://blog.getpelican.com/">Pelican</a>. It just works out-of-the-box! Initially I tried <a href="https://github.com/oz123/blogit">blogit</a> for its small codebase, but less code also means less functionality, not just fewer bugs :)</p>